Click here to Skip to main content
15,867,568 members
Articles / Multimedia / DirectX
Article

Capture Sample with DirectX and .NET

Rate me:
Please Sign up or sign in to vote.
4.70/5 (50 votes)
10 Aug 20035 min read 486.2K   45.6K   120   99
A solution to capture video and frames simultaneously

Image 1

Introduction

This article tries to explain how to capture video and frames simultaneously. It is based on SampleGrabber program done by NETMaster and DirectX.Capture by Brian Low. The root of all of this was the amazing work of NETMaster, DirectShow.NET, SampleGrabber is just a sample application. I said amazing because I'm close to understand completely the other projects or classes, but DirectShow.NET is far of my knowledge.

Before begin with the explanation, I have to say that I am not an expert programmer, this work is one of the first serious programs that I have ever typed, so probably someone could find a best solution to do the same.

At this point you could probably guess that my mother tongue is not English, it is Spanish, sorry for my grammatical mistakes. I have write the code in Visual Studio .NET Spanish version, therefore the automatically generated comments are in Spanish, but my comments are in English.

The VB code

First of all, to run the demo program you will need to have installed NET Framework 1.1. check windows update if you don't have it.

Here I want to explain how the project works in VB, I will explain the sampleGrabber filter in the next section. The main project is CapSample, it has three forms; MW, AddCam and CW, and a module ModCap. Let's begin with the description.

Image 2

MW stands for "Main Window", it's the initial form. When it is created, it create a new form, AddCam, that is responsible to the selection of an available camera. If OK button is clicked, AddCam will connect to the camera. The connection is done by using the capture class of DirectX.Capture, as I said before in the next section I will discuss my modifications to this class.

Image 3

Once you have the camera connected (AddCam finished, if you click Cancel an error will happen), MW continues its initialization and set up the preview board (I am not sure about this word, in Spanish this control it's call "Panel"), then it add a handler to the FrameCaptureComplete event of the capture class and initialize the counters. Finishing with the creation of CW (It stands for "Configuration Window"), which is responsible of the configuration of the camera. By clicking OK ConfParamCam() will set the selected parameters, then it will set the capture directory (If it doesn't exist an error will happen), and capture a frame and render the preview stream by CaptureInformation.CaptureInfo.CaptureFrame().

Once you have configured the camera, you are able to capture frames by clicking the Frame button and start the video capture by clicking the Start button. When you click Stop, it stops the capture but it calls ConfParamCam() to set the previous selected parameters and PrepareCam() to increment the name of the captured file. And that is all I have to say about the VB code. Let's begin with the capture class.

Modifications in capture class of DirectX.Capture

My capture class it is a mixed class between capture class of Brian Low and SampleGrabberNET program of NETMaster.

In createGraph(), I have added the next code

C#
AMMediaType media = new AMMediaType();
media.majorType= MediaType.Video;
media.subType = MediaSubType.RGB24;
media.formatType = FormatType.VideoInfo;
hr = sampGrabber.SetMediaType( media );
if( hr<0 ) Marshal.ThrowExceptionForHR( hr );

and

C#
mediaEvt    = (IMediaEventEx)    graphBuilder;
baseGrabFlt = (IBaseFilter) sampGrabber;
hr = graphBuilder.AddFilter( baseGrabFlt, "DS.NET Grabber" );
if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

The first block is to can use the filter and the second bock is the filter itself. The important things come with renderGraph(), before show you the code I have to say that all WDM devices have two PINs, capture and preview, sometimes only one (capture) but the smart tee filter convert the capture PIN into capture and preview, if it is needed Intelligent Connect will add it for us. So, I will use the capture PIN to capture video to a file and the preview PIN to capture frames. RenderGraph() is divided in two ifs, one to prepare all the filters to capture video to a file, it is controlled by wantCaptureRendered, and the other it is to render the preview stream, I have added the variable renderStream to avoid the normal functioning of the original class of Brian Low, because when baseGrabFlt is set-up I don´t know how to configure the cameras and when a capture was stopped, camera lost its configuration parameters, and I stop render until this parameters are updated with renderStream.

The first if is equal to the first if in Brian Low class, and the second looks like this.

C#
// Render preview stream and launch the baseGrabFlt to capture frames
// ==================================================================
if ( wantPreviewRendered && renderStream && !isPreviewRendered )
{
    /// Render preview (video.PinPreview -> baseGrabFlt -> renderer)
    /// At this point intelligent connect is used, because my 
    /// webcams don't have a preview pin and
    /// a capture pin, so Smart Tee filter will be used. 
    /// I have tested it using GraphEdit.
    /// I can type hr = captureGraphBuilder.RenderStream( ref cat, 
    /// ref med, videoDeviceFilter, null, baseGrabFlt); 
    /// because baseGrabFlt is a transform filter, 
    /// like videoCompressorFilter.
    
    cat = PinCategory.Preview;
    med = MediaType.Video;
    hr = captureGraphBuilder.RenderStream( ref cat, ref med, 
        videoDeviceFilter, baseGrabFlt, null ); 
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    // Get the IVideoWindow interface
    videoWindow = (IVideoWindow) graphBuilder;

    // Set the video window to be a child of the main window
    hr = videoWindow.put_Owner( previewWindow.Handle );
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    // Set video window style
    hr = videoWindow.put_WindowStyle( WS_CHILD | WS_CLIPCHILDREN 
       | WS_CLIPSIBLINGS);
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    // Position video window in client rect of owner window
    previewWindow.Resize += new EventHandler( onPreviewWindowResize );
    onPreviewWindowResize( this, null );

    // Make the video window visible, now that it is properly positioned
    hr = videoWindow.put_Visible( DsHlp.OATRUE );
    if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

    hr = mediaEvt.SetNotifyWindow( this.Handle, 
         WM_GRAPHNOTIFY, IntPtr.Zero );
    if( hr < 0 )
        Marshal.ThrowExceptionForHR( hr );

    isPreviewRendered = true;
    didSomething = true;

    // Begin Configuration of SampGrabber    <<<<<<-----------------
                
    AMMediaType media = new AMMediaType();
    hr = sampGrabber.GetConnectedMediaType( media );
    if( hr < 0 )
        Marshal.ThrowExceptionForHR( hr );
    if( (media.formatType != FormatType.VideoInfo) || 
              (media.formatPtr == IntPtr.Zero) )
        throw new NotSupportedException( "Unknown Grabber Media Format" ); 

    videoInfoHeader = (VideoInfoHeader) Marshal.PtrToStructure( 
               media.formatPtr, typeof(VideoInfoHeader) );
    Marshal.FreeCoTaskMem( media.formatPtr ); 
    media.formatPtr = IntPtr.Zero;

    hr = sampGrabber.SetBufferSamples( false );
    if( hr == 0 )
        hr = sampGrabber.SetOneShot( false );
    if( hr == 0 )
        hr = sampGrabber.SetCallback( null, 0 );
    if( hr < 0 )
        Marshal.ThrowExceptionForHR( hr );    
                
    // Finish Configuration of SampGrabber    <<<<<<----------------
}
            
if ( didSomething )
    graphState = GraphState.Rendered;

In this four lines is when the baseGrabFlt is set-up, note that I use baseGrabFlt as a transform filter.

C#
cat = PinCategory.Preview;
med = MediaType.Video;
hr = captureGraphBuilder.RenderStream( ref cat, ref med, 
    videoDeviceFilter, baseGrabFlt, null ); 
if( hr < 0 ) Marshal.ThrowExceptionForHR( hr );

About the configuration of sampleGrabber, I only comment next lines.

C#
hr = sampGrabber.SetBufferSamples( false );
if( hr == 0 )
    hr = sampGrabber.SetOneShot( false );
if( hr == 0 )
    hr = sampGrabber.SetCallback( null, 0 );
if( hr < 0 )
    Marshal.ThrowExceptionForHR( hr );

SampleGrabber has three different methods to capture a frame, I choose SetCallBack because I found it more useful, sampleGrabber will capture a frame when a call to a function was done.

We need more things to complete the capture of a frame, and it is done by the next block of code.

C#
void OnCaptureDone()
{
    int hr;
    if( sampGrabber == null )
        return;
    hr = sampGrabber.SetCallback( null, 0 );

    int w = videoInfoHeader.BmiHeader.Width;
    int h = videoInfoHeader.BmiHeader.Height;
    if( ((w & 0x03) != 0) || (w < 32) || (w > 4096) 
            || (h < 32) || (h > 4096) )
        return;
    int stride = w * 3;

    GCHandle handle = GCHandle.Alloc( savedArray, GCHandleType.Pinned );
    int scan0 = (int) handle.AddrOfPinnedObject();
    scan0 += (h - 1) * stride;
    Bitmap b = new Bitmap( w, h, -stride, 
        PixelFormat.Format24bppRgb, (IntPtr) scan0 );
    handle.Free();
    savedArray = null;
    ImageCaptured.Image = b;

    //Launch the event

    FrameCaptureComplete(ImageCaptured);

    return;
        
}

protected override void WndProc( ref Message m )
{
    if( m.Msg == WM_GRAPHNOTIFY )
    {
        if( mediaEvt != null )
            OnGraphNotify();
                return;
    }
    base.WndProc( ref m );
}

// graph event (WM_GRAPHNOTIFY) handler
void OnGraphNotify()
{
    DsEvCode    code;
    int p1, p2, hr = 0;
    do
    {
        hr = mediaEvt.GetEvent( out code, out p1, out p2, 0 );
        if( hr < 0 )
            break;
        hr = mediaEvt.FreeEventParams( code, p1, p2 );
    }
    while( hr == 0 );
}

int ISampleGrabberCB.SampleCB( double SampleTime, 
    IMediaSample pSample )
{
    return 0;
}
        
int ISampleGrabberCB.BufferCB(double SampleTime, 
    IntPtr pBuffer, int BufferLen )
{
    if( captured || (savedArray == null) )
    {
        return 0;
    }
    captured = true;
    bufferedSize = BufferLen;
    if( (pBuffer != IntPtr.Zero) && (BufferLen > 1000)
            && (BufferLen <= savedArray.Length) )
        Marshal.Copy( pBuffer, savedArray, 0, BufferLen );
    try
    {
        this.BeginInvoke( new CaptureDone( this.OnCaptureDone ) );
    }
    catch (ThreadInterruptedException e)
    {
        MessageBox.Show(e.Message);
    }
    catch (Exception we)
    {
        MessageBox.Show(we.Message);
    }
    return 0;
}

Beginning at the end, ISampleGraberCB.BufferCB(...) is a function that is filling a buffer, and when the buffer is full it have the frame, and call OnCaptureDone() that is the function that create the image. When the image was built it issues an event to send it to the receiver of that event.

ISampleGrabberCB.SampleCB(...) have to be defined instead it was not be used. For me the other two functions are black magic, I can guess what they are doing but I don't understand it.

And finally the method invoked to capture a frame.

C#
public void CaptureFrame()
{
    int hr;

    if(firstFrame)
    {
        assertStopped();

        // Re-render the graph (if necessary)
        renderStream = true;
        renderGraph();

        // Start the filter graph: begin capturing
        hr = mediaControl.Run();
        if ( hr != 0 ) Marshal.ThrowExceptionForHR( hr ); 

        firstFrame = false;
    }

    captured = false;

    if(savedArray == null )
    {
        int size = videoInfoHeader.BmiHeader.ImageSize;
        if( (size<1000) || (size > 16000000) )
            return;
        savedArray = new byte[ size + 64000];
    }
    hr = sampGrabber.SetCallback( this, 1 );
}

firstFrame is to avoid execute more than once next line.

C#
hr = mediaControl.Run()

I think the description is completed. I hope it will be useful to you :-)

System Requirements

I have been working with an AMDXP@1500, 512Mb RAM and WIN2k. I have installed DirectX 9.0 but, I think, it could work with DirectX 8.1.

As cameras I have been using a Philips ToUcam Pro and a Creative Video Blaster Web Cam 5 without problems. I have also installed a Pinacle PCTV Pro and it doesn't work capture a video to a file. I don't know why, but at the moment I don't have enough time to search the source of the problem.

Feedback and Improvements

I will try to keep track of the forum, if you need help I will be glad to give you a hand. I think we have to share code to improve the performance of our programs, and to help people do new things, for example, if I have not read the code of Brian Low I would never have written this code. Thank you!

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


Written By
Web Developer
Spain Spain
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
BugThe original project is much better than this Pin
Elmue11-Oct-14 5:23
Elmue11-Oct-14 5:23 
QuestionHow do I rotate a camera view got with DirectX in Vb.Net ? Pin
Olayèmi Ouabi12-May-14 23:10
Olayèmi Ouabi12-May-14 23:10 
Questionsir why is this code Pin
Member 1021801526-Feb-14 16:26
Member 1021801526-Feb-14 16:26 
QuestionException Pin
tweber201225-Feb-14 10:08
tweber201225-Feb-14 10:08 
AnswerRe: Exception Pin
Member 1588474530-Jun-23 3:45
Member 1588474530-Jun-23 3:45 
QuestionIs it possible to Rotate live view 90 degree to use webcam in portrait view? Pin
sakda12313-Feb-13 22:00
sakda12313-Feb-13 22:00 
QuestionException has been thrown by the target of an invocation Pin
Suman Lata Gupta22-Jan-13 19:54
Suman Lata Gupta22-Jan-13 19:54 
AnswerRe: Exception has been thrown by the target of an invocation Pin
Rajesh M Panchal17-May-23 4:04
Rajesh M Panchal17-May-23 4:04 
GeneralMy vote of 5 Pin
manojIITKGP21-Jan-13 8:38
manojIITKGP21-Jan-13 8:38 
QuestionDraw rectangle Pin
manganat17-Jun-12 23:03
manganat17-Jun-12 23:03 
GeneralMy vote of 5 Pin
morteza fakoor9-Dec-11 20:16
morteza fakoor9-Dec-11 20:16 
Generalthe video displayed mirrored Pin
li5715-May-11 19:56
li5715-May-11 19:56 
GeneralMy vote of 5 Pin
li5715-May-11 19:51
li5715-May-11 19:51 
GeneralMy vote of 1 Pin
johnson37511-Feb-11 13:46
johnson37511-Feb-11 13:46 
GeneralRe: My vote of 1 Pin
Nasenbaaer14-Mar-11 6:06
Nasenbaaer14-Mar-11 6:06 
Generalpause video Pin
niks231418-Nov-10 0:29
niks231418-Nov-10 0:29 
Generalmouse events Pin
Dafnwa31-Oct-10 2:09
Dafnwa31-Oct-10 2:09 
GeneralRe: mouse events Pin
959AVA13-May-12 21:36
959AVA13-May-12 21:36 
Questionwhy check for (w & 0x03) != 0) in OnCaptureDone() ? Pin
RiversideWalruses13-Oct-10 12:50
RiversideWalruses13-Oct-10 12:50 
Generalsetup project for DirectShow applications Pin
JoJo-jOjO19-Jun-10 2:17
JoJo-jOjO19-Jun-10 2:17 
GeneralMultiple Cams causing Error Pin
robin2421-Jan-10 4:46
robin2421-Jan-10 4:46 
GeneralMy vote of 2 Pin
BytePower11-Dec-09 0:05
BytePower11-Dec-09 0:05 
GeneralRe: My vote of 2 Pin
manojIITKGP21-Jan-13 8:45
manojIITKGP21-Jan-13 8:45 
Questionmultiple webcam - one avi Pin
solyom6-Aug-09 4:23
solyom6-Aug-09 4:23 
Generalcrashes when choose comressor Pin
katran.ek10-Mar-09 23:39
katran.ek10-Mar-09 23:39 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.