Click here to Skip to main content
15,867,488 members
Articles / Desktop Programming / MFC
Article

Video Preview and Frames Capture to Memory with SampleGrabber in Buffered Mode.

Rate me:
Please Sign up or sign in to vote.
4.91/5 (45 votes)
26 Oct 2007GPL34 min read 350.7K   21.6K   163   107
This article demonstrates video preview and frames capture to memory from external video devices using ISampleGrabber interface in buffered mode.
Screenshot - vidcap.jpg

Introduction

I looked around for a project capturing incoming video from external video devices to memory buffer for image processing applications and could not find the one I needed. The existing articles on The Code Project, An Easy Video Processing Framework by Grabbing Frames as Bitmaps using DirectShow, CAviCap and CFrameGrabber - Wrappers for AVICap Window, Real-time Video Image Processing / Frame Grabber using a Minimalistic Approach, Simultaneous Previewing & Video Capture using DirectShow do not provide the preferred buffering mode for capturing raw data from the stream. Some of them were tested only on WinXP and Windows 2000, pretty outdated, the latest one goes back to 1999 or captures samples from video files and so on.

I have Vista on my computer and installed Windows SDK for Vista and DirectX SDK 2007. Now Direct Show is a part of Windows SDK and if you want to use ISampleGrabber for raw data capture, you need to install DirectX SDK too. I developed a simple MFC application supporting enumeration of existing video devices, preview of the video stream, capture raw image data in buffered mode with the desired frame rate, display it to the window with GDI+, taking snap shots to JPEG files. With that application as a skeleton, you can easily start with image/video processing tasks (video codecs development, motion estimation, edge detection). I use that one in my face detection program I'm going to post later.

Background

You need to have an understanding in Direct Show programming. Have a look at the above mentioned articles or Windows Vista SDK help. Understanding of COM technology is desired - have a look at the article, Introduction to COM - What It Is and How to Use It.

Using the Code

Click the enum button that will enumerate available video devices and select one. Specify the desired raw image data capture interval in milliseconds (default one is 1000, capture data every second), click the Run button. The left top small static will preview the video output and the center large one will show captured raw images. To take a snapshot, left mouse double click the center capture static window and the captured image will be saved on disk to the same directory with Snapshot X.jpg name, where X is the number of the image.

For code reuse, I provided Sample Grabber and Video Capture implementation in separate files. I used the Video Capture routines from the SDK example (SDK\Samples\Multimedia\DirectShow\Capture\PlayCap). I wrapped them in functions easy to use with an external application, and also added video devices enumeration:

  • C++
    void vcGetCaptureDevices(CComboBox& adaptersBox);
  • C++
    HRESULT vcCaptureVideo(HWND msgWindow, HWND prvWindow, unsigned int devIndex = 1);
  • C++
    void vcStopCaptureVideo();

vcGetCaptureDevices() enumerate video devices and add them to ComboBox. vcCaptureVideo() starts video preview to prvWindow with msgWindow application that will handle Filter Graph notification messages. devIndex is the index of the video device to capture data from. vcStopCaptureVideo stops capturing video data.
The Sample Grabber added to Filter Graph in vcCaptureVideo functions this way:

C++
//...

hr = sgAddSampleGrabber(g_pGraph);
if (FAILED(hr)) {
        Msg(TEXT("Couldn't add the SampleGrabber filter to the graph!  hr=0x%x"), hr);
        return hr;
}
hr = sgSetSampleGrabberMediaType();
if (FAILED(hr)) {
        Msg(TEXT("Couldn't set the SampleGrabber media type!  hr=0x%x"), hr);
        return hr;
}
IBaseFilter* pGrabber = sgGetSampleGrabber();

// Render the preview pin on the video capture filter
// Use this instead of g_pGraph->RenderFile
hr = g_pCapture->RenderStream(&PIN_CATEGORY_PREVIEW, &MEDIATYPE_Video,
                              pSrcFilter, pGrabber/*NULL*/, NULL);
if (FAILED(hr)) {
        Msg(TEXT("Couldn't render the video capture stream.  hr=0x%x\r\n")
            TEXT("The capture device may already be in use by another application.
                  \r\n\r\n")
            TEXT("The sample will now close."), hr);
        pSrcFilter->Release();
        return hr;
}

hr = sgGetSampleGrabberMediaType();

//...

The corresponding sg* functions are located in samplegrab.cpp:

C++
HRESULT sgAddSampleGrabber(IGraphBuilder *pGraph)
{
        // Create the Sample Grabber.
        HRESULT hr = CoCreateInstance(CLSID_SampleGrabber, NULL, CLSCTX_INPROC_SERVER,
                                      IID_IBaseFilter, (void**) & pGrabberFilter);
        if (FAILED(hr)) {
                return hr;
        }
        hr = pGraph->AddFilter(pGrabberFilter, L"Sample Grabber");
        if (FAILED(hr)) {
                return hr;
        }

        pGrabberFilter->QueryInterface(IID_ISampleGrabber, (void**)&pGrabber);
        return hr;
}

HRESULT sgSetSampleGrabberMediaType()
{
        AM_MEDIA_TYPE mt;
        ZeroMemory(&mt, sizeof(AM_MEDIA_TYPE));
        mt.majortype = MEDIATYPE_Video;
        mt.subtype = MEDIASUBTYPE_RGB24;
        HRESULT hr = pGrabber->SetMediaType(&mt);
        if (FAILED(hr)) {
                return hr;
        }
        //Do not stop the graph after one shot
        hr = pGrabber->SetOneShot(FALSE);
        //Use buffered mode
        hr = pGrabber->SetBufferSamples(TRUE);
        return hr;
}

IBaseFilter* sgGetSampleGrabber()
{
        return pGrabberFilter;
}

HRESULT sgGetSampleGrabberMediaType()
{
        AM_MEDIA_TYPE mt;
        HRESULT hr = pGrabber->GetConnectedMediaType(&mt);
        if (FAILED(hr)) {
                return hr;
        }

        VIDEOINFOHEADER *pVih = (VIDEOINFOHEADER *)mt.pbFormat;
        gChannels = pVih->bmiHeader.biBitCount / 8;
        gWidth = pVih->bmiHeader.biWidth;
        gHeight = pVih->bmiHeader.biHeight;

        sgFreeMediaType(mt);
        return hr;
}

The initialization of video preview and capture is implemented in the Run button click routine CVidCapDlg::OnBnClickedRunButton(), where the timer is created to query captured raw image data by Sample Grabber filter which is the preferred way of using it as described in SDK help under the topic "Using the Sample Grabber".

To grab the raw image data, use the following functions from samplegrab.h:

  • C++
    unsigned char* sgGrabData();
  • C++
    Gdiplus::Bitmap* sgGetBitmap();
  • C++
    long sgGetBufferSize();

The first one returns a pointer to the buffer with raw image data, or NULL if the Sample Grabber is not ready yet. Note that Sample Grabber filter fills that buffer with the image turned upside down, as a bottom up bitmap. sgGetBitmap() returns GDI+ bitmap filled with raw image data after sgGrabData() call or NULL if the data was not captured. I use sgFlipUpDown() function to flip the raw image before copying it to the Bitmap. You should use those functions in that order, first call sgGrabData() to get the raw image and if you want Bitmap object to draw it into the window, call sgGetBitmap() later. To get the length of the buffer with raw image, call sgGetBufferSize().

The code for the grabbing is shown below:

C++
unsigned char* sgGrabData()
{
        HRESULT hr;

        if (pGrabber == 0)
                return 0;

        long Size = 0;
        hr = pGrabber->GetCurrentBuffer(&Size, NULL);
        if (FAILED(hr))
                return 0;
        else if (Size != pBufferSize) {
                pBufferSize = Size;
                if (pBuffer != 0)
                        delete[] pBuffer;
                pBuffer = new unsigned char[pBufferSize];
        }

        hr = pGrabber->GetCurrentBuffer(&pBufferSize, (long*)pBuffer);
        if (FAILED(hr))
                return 0;
        else {
                sgFlipUpDown(pBuffer);
                return pBuffer;
        }
}

To determine the image size, call these functions:

  • C++
    unsigned int sgGetDataWidth();
  • C++
    unsigned int sgGetDataHeight();
  • C++
    unsigned int sgGetDataChannels();

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)


Written By
Engineer
Russian Federation Russian Federation
Highly skilled Engineer with 14 years of experience in academia, R&D and commercial product development supporting full software life-cycle from idea to implementation and further support. During my academic career I was able to succeed in MIT Computers in Cardiology 2006 international challenge, as a R&D and SW engineer gain CodeProject MVP, find algorithmic solutions to quickly resolve tough customer problems to pass product requirements in tight deadlines. My key areas of expertise involve Object-Oriented
Analysis and Design OOAD, OOP, machine learning, natural language processing, face recognition, computer vision and image processing, wavelet analysis, digital signal processing in cardiology.

Comments and Discussions

 
AnswerRe: Grabbing data without preview Pin
Chesnokov Yuriy26-May-08 21:49
professionalChesnokov Yuriy26-May-08 21:49 
GeneralReduce the capture buffer size Pin
irvind21-Apr-08 16:37
irvind21-Apr-08 16:37 
GeneralRe: Reduce the capture buffer size Pin
Chesnokov Yuriy26-May-08 21:48
professionalChesnokov Yuriy26-May-08 21:48 
Generali wanna ask about this.. Pin
arik078-Mar-08 2:20
arik078-Mar-08 2:20 
AnswerRe: i wanna ask about this.. Pin
Chesnokov Yuriy18-Mar-08 20:30
professionalChesnokov Yuriy18-Mar-08 20:30 
GeneralRe: i wanna ask about this.. Pin
arikdp31-Mar-08 2:11
arikdp31-Mar-08 2:11 
Generalcouldn`t run the graph Pin
pesokes6-Feb-08 13:37
pesokes6-Feb-08 13:37 
AnswerRe: couldn`t run the graph Pin
Chesnokov Yuriy18-Mar-08 20:29
professionalChesnokov Yuriy18-Mar-08 20:29 
GeneralRe: couldn`t run the graph Pin
pesokes25-Mar-08 0:12
pesokes25-Mar-08 0:12 
GeneralRequires a 64 bit dll ? [modified] Pin
sanidhyakhilnani26-Jan-08 11:36
sanidhyakhilnani26-Jan-08 11:36 
AnswerRe: Requires a 64 bit dll ? Pin
Chesnokov Yuriy28-Jan-08 21:17
professionalChesnokov Yuriy28-Jan-08 21:17 
GeneralChange the capture default size Pin
kit200018-Dec-07 21:53
kit200018-Dec-07 21:53 
GeneralRe: Change the capture default size Pin
captainc/c++13-Jan-08 7:17
captainc/c++13-Jan-08 7:17 
QuestionHow to Change the Capture Size? Pin
kit200018-Dec-07 21:51
kit200018-Dec-07 21:51 
AnswerRe: How to Change the Capture Size? Pin
Chesnokov Yuriy14-Jan-08 4:56
professionalChesnokov Yuriy14-Jan-08 4:56 
GeneralSending Data over the Network Pin
captainc/c++10-Dec-07 3:52
captainc/c++10-Dec-07 3:52 
GeneralRe: Sending Data over the Network Pin
captainc/c++13-Jan-08 7:20
captainc/c++13-Jan-08 7:20 
AnswerRe: Sending Data over the Network Pin
Chesnokov Yuriy14-Jan-08 5:02
professionalChesnokov Yuriy14-Jan-08 5:02 
QuestionProject Environment Pin
yklein126-Nov-07 9:11
yklein126-Nov-07 9:11 
GeneralRe: Project Environment Pin
Chesnokov Yuriy14-Jan-08 4:59
professionalChesnokov Yuriy14-Jan-08 4:59 
GeneralCodec Pin
captainc/c++26-Nov-07 2:41
captainc/c++26-Nov-07 2:41 
GeneralVisual Studio/c++ V6 Pin
worksopbenny19-Nov-07 18:31
worksopbenny19-Nov-07 18:31 
GeneralRe: Visual Studio/c++ V6 Pin
Chesnokov Yuriy19-Nov-07 19:57
professionalChesnokov Yuriy19-Nov-07 19:57 
GeneralRe: Visual Studio/c++ V6 Pin
uvik11-Oct-10 0:05
uvik11-Oct-10 0:05 
AnswerRe: Visual Studio/c++ V6 Pin
Chesnokov Yuriy11-Oct-10 2:36
professionalChesnokov Yuriy11-Oct-10 2:36 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.