Click here to Skip to main content
15,867,141 members
Articles / Multimedia / DirectX
Article

Prototyping DirectShow filters in C#

Rate me:
Please Sign up or sign in to vote.
4.42/5 (6 votes)
14 Dec 20054 min read 281.6K   3.7K   66   97
This article shows how to implement a DirectShow filter in C#.

Sample Image

Introduction

DirectShow filters are the basic building blocks of multimedia applications on the Windows platform. Normally, they are written in C++. In this article, I'll show how to implement a couple of DirectShow transformation filters in C#. Since Microsoft doesn't recommend using managed code for filters, I'll call them prototypes but they perform quite well in most circumstances. We'll develop Sobel transformation as a DirectShow filter. Sobel transformations are often used for video segmentation which is the process of identifying objects in a scene.

Background

The strategy we use to implement a DirectShow filter in C# relies on having a C++ class that contains a pointer to a managed object which implements our filter. The C++ class just forwards all the processing to the managed object. The C++ class is generated using Cutler's DirectShow Filters Wizard. We just clean up the wizard-generated code to include a pointer to our C# object and implement all the filter processing as methods of the managed object.

I've written this Sobel filter to have an idea of the kind of performance we'd get with C#. If you want to "take your CPU for a spin", this is a good test. As written (i.e., with no optimization effort), there are around 20 integer multiplications per pixel. Usually, you'd transform an image to black and white before applying Sobel transformation to the image. So I've also written a transformation filter that converts a RGB24 bitmap to black and white.

Using the code

If you just want to have a filter that implements Sobel transformations, you can download and install the managed DLL and Ax files (from the download link). Then after registering the Ax file and installing the managed DLL in the Global Assembly Cache (GAC), you can start the GraphEdit utility and add the "CsSobelV2" filter (found in the DirectShow filters category of GraphEdit). Then render a media file to see a black and white "sketch" of the frames in the video. (Note that the open-source DirectShowLib must also be in the GAC.) More information can be found in the "readme.txt" file that is included in the download.

If you want to implement your own filter using these filters as a starting point, you can modify the Transform and ProcessFrame methods of the MySobelV2 class. This is sufficient if you are satisfied with a single media type for the video frame, here Rgb24, and you can live with the default memory allocator of the C++ DirectShow base classes (this covers a lot of situations).

Sobel transformation

The Sobel transformation uses the following two matrices:

C#
// define the horizontal Sobel mat
int [,] hx = new  int [,] { {-1, 0, 1},
                  {-2, 0, 2}, {-1, 0, 1} };

// define the vertical Sobel mat
int[,] hy = new int [,]  { { 1,  2,  1},
                { 0,  0,  0}, {-1, -2, -1} };

And consists of the following code, found in ProcessFrame, which is repeated for every pixel in every frame of a video:

C#
gradX = 0;
gradY = 0;
for (int row = -1; row <= 1; row++)
{
  for (int col = -1; col <= 1; col++)
  {
    position = GetPosition( x + col, y + row );
    intensity = pwSource[position];
    gradX += (intensity * hx[col + 1,row + 1]);
    gradY += (intensity * hy[col + 1,row + 1]);
  }
}

// compute the square of the gradient
gradMag = gradX * gradX + gradY * gradY;

// threshold default at 128
gradMag = (gradMag < threshold*threshold) ? 255 : 0;

// assign the pixel to the destination
position = GetPosition( x, y );
pwTarget[position] = pwTarget[position+1] =
      pwTarget[position+2] = (byte)gradMag;

The intuition behind the transformation is relatively simple. Since the contour of objects consists of a region in a picture where the color changes rapidly, the Sobel transformation amplifies these differences and simply sets to white those pixels whose changes are too small (and black, the other ones). The horizontal Sobel transformation looks at the left and right neighboring pixels and multiplies these values by 2 or -2, it also looks at the upper right and left, and lower right and left pixels. After summing all these values, you compute the square of the Euclidian distance and use an arbitrary threshold to divide the values.

Points of Interest

I've also included the code for a black and white transformation filter for uncompressed Rgb24 video frames. As mentioned, you'd normally perform this transformation before feeding the result to the Sobel filter. So you can test the performances of a filter graph with two filters written in managed code.

Property pages for filters are handy when used in a utility like GraphEdit. But being able to access custom filter properties from C# (easily) is even more useful. So I've modified Cutler's Wizard-generated C++ code to include a property that can be changed in C#. Similar changes could be done to implement different properties.

Limitations and known issues

The media type accepted by the filters is uncompressed Rgb24 video. Since this is one of the most "natural" media types, it's not a big limitation but the filters were not designed to be very flexible. They are just prototypes to experiment with.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


Written By
United States United States
You can read my blog entries at:
http://wwww.informikon.com/blog/

Comments and Discussions

 
Questioncan your .ax and .dll files register in windows 7? Pin
Member 1377552415-Apr-18 19:40
Member 1377552415-Apr-18 19:40 
NewsDirectshow filters completely in .NET Pin
Maxim Kartavenkov15-Aug-12 23:32
Maxim Kartavenkov15-Aug-12 23:32 
SuggestionGreat article, but why not do it all in C++? Pin
MikeDC25-Jun-11 13:01
MikeDC25-Jun-11 13:01 
QuestionWhere did you get .ax files? Pin
h.ahmadi8619-Dec-09 23:35
h.ahmadi8619-Dec-09 23:35 
GeneralGreat idea Pin
Steven Nagy6-Oct-09 16:47
Steven Nagy6-Oct-09 16:47 
GeneralMSIL error at compiling CsSobelV2.ax Pin
patkoxx4-Jun-09 3:44
patkoxx4-Jun-09 3:44 
GeneralRe: MSIL error at compiling CsSobelV2.ax Pin
tsylverberg30-Jul-09 20:50
tsylverberg30-Jul-09 20:50 
GeneralDllRegisterServer in CsSobel.ax Failed. Pin
roboticEDAR11-May-09 12:31
roboticEDAR11-May-09 12:31 
QuestionVisual studios 2008 Pin
Member 47309906-May-09 6:45
Member 47309906-May-09 6:45 
AnswerRe: Visual studios 2008 Pin
roboticEDAR7-May-09 11:19
roboticEDAR7-May-09 11:19 
I was going too, did you get it to build in 2005 ?
AnswerRe: Visual studios 2008 Pin
roboticEDAR27-May-09 12:31
roboticEDAR27-May-09 12:31 
GeneralThere Are Several NON-Commercial RTSP Procol Source Filters! Pin
Bill SerGio, The Infomercial King26-Oct-08 4:37
Bill SerGio, The Infomercial King26-Oct-08 4:37 
QuestionAbstract Class Error Pin
nickmaggie31-Aug-07 3:56
nickmaggie31-Aug-07 3:56 
AnswerRe: Abstract Class Error Pin
daniel0491-Sep-07 9:44
daniel0491-Sep-07 9:44 
GeneralCompiling on VS 2003 version 7.1 Framework 1.1 Pin
Russ531-Aug-07 8:01
Russ531-Aug-07 8:01 
GeneralRe: Compiling on VS 2003 version 7.1 Framework 1.1 Pin
daniel0492-Aug-07 2:43
daniel0492-Aug-07 2:43 
Generalnot working Pin
yair_coh26-May-07 4:23
yair_coh26-May-07 4:23 
GeneralRe: not working Pin
redpirate13-Jun-07 2:40
redpirate13-Jun-07 2:40 
GeneralRe: not working Pin
ToolmakerSteve217-Jul-07 20:29
ToolmakerSteve217-Jul-07 20:29 
GeneralRe: not working Pin
ppaijwar28-Nov-07 17:34
ppaijwar28-Nov-07 17:34 
GeneralRe: not working Pin
roboticEDAR12-May-09 13:39
roboticEDAR12-May-09 13:39 
GeneralRe: not working Pin
roboticEDAR7-May-09 11:23
roboticEDAR7-May-09 11:23 
Questiona cry for help from FDaSilva Pin
Sean Ewington9-May-07 13:59
staffSean Ewington9-May-07 13:59 
AnswerRe: a cry for help from FDaSilva Pin
daniel04917-May-07 10:33
daniel04917-May-07 10:33 
GeneralVS Express Solution and Projects file can be downloaded Pin
daniel04917-Nov-06 6:04
daniel04917-Nov-06 6:04 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.