Click here to Skip to main content
15,867,488 members
Articles / Multimedia / DirectX
Article

Extract Frames from Video Files

Rate me:
Please Sign up or sign in to vote.
4.72/5 (40 votes)
26 Sep 2007CPOL3 min read 541.1K   40.2K   129   99
Class to extract frames from most video file formats using IMediaDet
Sample Image - extractvideoframes.jpg

Introduction

This class contains methods to use the IMediaDet interface that can be found in Microsoft DirectShow. The Media Detector object, among other things, can be used to extract still pictures from several file formats including *.avi, *.wmv and some *.mpeg files.

This class exposes the GetFrameFromVideo, GetVideoSize and SaveFrameFromVideo methods that can be used from any .NET application. The class also takes care of translating HRESULTs returned from the functions to meaningful .NET exceptions.

Using the Code

Just add a reference to JockerSoft.Media.dll in your project (or include the source code). Remember also to distribute Interop.DexterLib.dll.
All the methods are static, so to use them just do something like this:

C#
try
{
    this.pictureBox1.Image = FrameGrabber.GetFrameFromVideo(strVideoFile, 0.2d);
}
catch (InvalidVideoFileException ex)
{
    MessageBox.Show(ex.Message, "Extraction failed");
}
catch (StackOverflowException)
{
    MessageBox.Show("The target image size is too big", "Extraction failed");
}

or

C#
try
{
    FrameGrabber.SaveFrameFromVideo(strVideoFile, 0.2d, strBitmapFile);
}
catch (InvalidVideoFileException ex)
{
    MessageBox.Show(ex.Message, "Extraction failed");
}

Here, we used the simplest of the three overloads of GetFrameFromVideo and SaveFrameFromVideo methods, presented in this article.

Points of Interest

The IMediaDet and linked interfaces/classes are exposed in qedit.dll, that can be found in System32 directory. Fortunately this DLL can be imported automatically using tlbimp, so no code is needed to wrap it.

To extract images, there are two methods: extract them in memory (using GetBitmapBits - here GetFrameFromVideo) or extract them and save to a bitmap file (using WriteBitmapBits - here SaveFrameFromVideo).

WriteBitmapBits is really simple to be used: we just need to find the video stream on the file, open it and specify an output file name for the bitmap image.

C#
public static void SaveFrameFromVideo(string videoFile,
         double percentagePosition, string outputBitmapFile,
         out double streamLength, Size target)
{
    if (percentagePosition > 1 || percentagePosition < 0)
        throw new ArgumentOutOfRangeException("percentagePosition", 
                percentagePosition, "Valid range is 0.0 .. 1.0");

    try
    {
        MediaDetClass mediaDet;
        _AMMediaType mediaType;
        if (openVideoStream(videoFile, out mediaDet, out mediaType))
        {
            streamLength = mediaDet.StreamLength;
            
            //calculates the REAL target size of our frame
            if (target == Size.Empty)
                target = getVideoSize(mediaType);
            else
                target = scaleToFit(target, getVideoSize(mediaType));

            mediaDet.WriteBitmapBits(streamLength * percentagePosition, 
                        target.Width, target.Height, outputBitmapFile);

            return;
        }
    }
    catch (COMException ex)
    {
        throw new InvalidVideoFileException(getErrorMsg((uint)ex.ErrorCode), ex);
    }

    throw new InvalidVideoFileException("No video stream was found");
}

You'll notice that two private methods are used here. They are openVideoStream and getVideoSize. Their implementation is straight forward:

C#
private static bool openVideoStream(string videoFile, 
            out MediaDetClass mediaDetClass, out _AMMediaType aMMediaType)
{
    MediaDetClass mediaDet = new MediaDetClass();
    
    //loads file
    mediaDet.Filename = videoFile;

    //gets # of streams
    int streamsNumber = mediaDet.OutputStreams;

    //finds a video stream and grabs a frame
    for (int i = 0; i < streamsNumber; i++)
    {
        mediaDet.CurrentStream = i;
        _AMMediaType mediaType = mediaDet.StreamMediaType;

        if (mediaType.majortype == JockerSoft.Media.MayorTypes.MEDIATYPE_Video)
        {
            mediaDetClass = mediaDet;
            aMMediaType = mediaType;
            return true;
        }
    }

    mediaDetClass = null;
    aMMediaType = new _AMMediaType();
    return false;
}

(where MEDIATYPE_Video is the GUID used for video files).

C#
private static Size getVideoSize(_AMMediaType mediaType)
{
    WinStructs.VIDEOINFOHEADER videoInfo = 
        (WinStructs.VIDEOINFOHEADER)Marshal.PtrToStructure(mediaType.pbFormat, 
            typeof(WinStructs.VIDEOINFOHEADER));
    
    return new Size(videoInfo.bmiHeader.biWidth, videoInfo.bmiHeader.biHeight);
}

Using GetBitmapBits to avoid saving the image on disk is a bit trickier, since we need to deal with direct access to memory.

The first part is identical to SaveFrameFromVideo, then we have to call GetBitmapBits with the pBuffer parameter set to null to get the size for the buffer of bytes that will contain the 24bpp image (GetBitmapBits always returns 24bpp images).

Once we have the size of the buffer, we allocate memory on the heap to receive the image (in the first version of this code, memory was allocated on the stack which is fine if the target image is small sized, but if it is big we may get a nice StackOverflowException because stack memory is rather limited).

After this, we call GetBitmapBits again, but this time the buffer will be filled with image bytes. Now we create a bitmap from these bytes (remembering that they start with BITMAPINFOHEADER structure, the size of which is 40 bytes).

C#
public static Bitmap GetFrameFromVideo(string videoFile, 
            double percentagePosition, out double streamLength, Size target)
{
    if (percentagePosition > 1 || percentagePosition < 0)
        throw new ArgumentOutOfRangeException("percentagePosition", 
                percentagePosition, "Valid range is 0.0 .. 1.0");

    try 
    {
        MediaDetClass mediaDet;
        _AMMediaType mediaType;
        if (openVideoStream(videoFile, out mediaDet, out mediaType))
        {
            streamLength = mediaDet.StreamLength;

            //calculates the REAL target size of our frame
            if (target == Size.Empty)
                target = getVideoSize(mediaType);
            else
                target = scaleToFit(target, getVideoSize(mediaType));

            unsafe 
            {
                Size s= GetVideoSize(videoFile);
                //equal to sizeof(CommonClasses.BITMAPINFOHEADER);
                int bmpinfoheaderSize = 40;                 

                //get size for buffer
                int bufferSize = (((s.Width * s.Height) * 24) / 8 ) + bmpinfoheaderSize;
                //equal to mediaDet.GetBitmapBits
                //    (0d, ref bufferSize, ref *buffer, target.Width, target.Height);    

                //allocates enough memory to store the frame
                IntPtr frameBuffer = 
                    System.Runtime.InteropServices.Marshal.AllocHGlobal(bufferSize);
                byte* frameBuffer2 = (byte*)frameBuffer.ToPointer();

                //gets bitmap, save in frameBuffer2
                mediaDet.GetBitmapBits(streamLength * percentagePosition, 
                    ref bufferSize, ref *frameBuffer2, target.Width, target.Height);

                //now in buffer2 we have a BITMAPINFOHEADER structure 
                //followed by the DIB bits
                Bitmap bmp = new Bitmap(target.Width, target.Height, target.Width * 3, 
                    System.Drawing.Imaging.PixelFormat.Format24bppRgb, 
                    new IntPtr(frameBuffer2 + bmpinfoheaderSize));

                bmp.RotateFlip(RotateFlipType.Rotate180FlipX);
                System.Runtime.InteropServices.Marshal.FreeHGlobal(frameBuffer);
                return bmp;
            }
        }
    }
    catch (COMException ex)
    {
        throw new InvalidVideoFileException(getErrorMsg((uint)ex.ErrorCode), ex);
    }

    throw new InvalidVideoFileException("No video stream was found");
}

Known Limitations

  • The biggest one is the StackOverflowException using stackalloc. There must be a way to pass to GetBitmapBits a buffer created on the heap, but I'm not yet very good with this unmanaged stuff.
    • This problem has been resolved. The solution is provided by _coder_ in the comments below.
  • On my machine, I got random errors when passing certain target sizes to GetBitmapBits: when the target size is 125x125 for example, the Bitmap constructor fails.
    • Only sizes multiples of 4 are allowed. Thanks to ujr (see comments).
  • "The IMediaDet interface does not support VIDEOINFOHEADER2 formats": This means it cannot open some *.mpeg video files.

History

  • 27th February, 2006: Initial release
  • 17th March, 2006: Replaced memory allocation on the stack
  • 27th September, 2007: Some fixes

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Italy Italy

Comments and Discussions

 
GeneralRe: Extract frame by index Pin
AlabamaKonsta29-Nov-07 4:15
AlabamaKonsta29-Nov-07 4:15 
GeneralRe: Extract frame by index Pin
SpaceWalker19-Feb-08 11:53
SpaceWalker19-Feb-08 11:53 
GeneralRe: Extract frame by index Pin
SpaceWalker20-Feb-08 2:50
SpaceWalker20-Feb-08 2:50 
AnswerRe: Extract frame by index Pin
SpaceWalker20-Feb-08 5:19
SpaceWalker20-Feb-08 5:19 
GeneralVery Nice Pin
merlin98127-Sep-07 3:35
professionalmerlin98127-Sep-07 3:35 
Generalfile not closed after opening Pin
phillkalksjf23-Sep-07 10:56
phillkalksjf23-Sep-07 10:56 
GeneralRe: file not closed after opening Pin
JockerSoft27-Sep-07 0:04
JockerSoft27-Sep-07 0:04 
QuestionRe: file not closed after opening Pin
K. Zimny1-Dec-07 3:31
K. Zimny1-Dec-07 3:31 
Hello,
I have also the same problem. You new release 1.0.0.1 didn't help, the file is still locked and cannot be deleted. Any idea why?
QuestionIs Extracting frames from Live video possible? Pin
kazim bhai17-Sep-07 23:28
kazim bhai17-Sep-07 23:28 
AnswerRe: Is Extracting frames from Live video possible? Pin
JockerSoft26-Sep-07 23:33
JockerSoft26-Sep-07 23:33 
Questionwhat am I doing wrong ? Pin
hedon13-Aug-07 2:21
hedon13-Aug-07 2:21 
GeneralGetVideoSize(videoFile) memory leak Pin
chack76-Jul-07 5:03
chack76-Jul-07 5:03 
GeneralRe: GetVideoSize(videoFile) memory leak Pin
JockerSoft26-Sep-07 23:36
JockerSoft26-Sep-07 23:36 
GeneralException from HRESULT: 0x8004027F Pin
sehajpal28-Jun-07 21:07
sehajpal28-Jun-07 21:07 
GeneralRe: Exception from HRESULT: 0x8004027F Pin
JockerSoft28-Jun-07 21:59
JockerSoft28-Jun-07 21:59 
GeneralRe: Exception from HRESULT: 0x8004027F Pin
sehajpal28-Jun-07 22:03
sehajpal28-Jun-07 22:03 
GeneralRe: Exception from HRESULT: 0x8004027F Pin
Nagendrababu23-Oct-07 1:11
Nagendrababu23-Oct-07 1:11 
GeneralExtract frames video files is written by C++ 6.0 Pin
cua16-May-07 23:00
cua16-May-07 23:00 
GeneralReally gr8 :rolleyes: Pin
alambix3-Apr-07 1:48
alambix3-Apr-07 1:48 
QuestionGetting the xth frame and/or the total number of frames Pin
mary ann19-Mar-07 1:19
mary ann19-Mar-07 1:19 
GeneralPartial trust problem Pin
P.Sravan1-Feb-07 23:53
P.Sravan1-Feb-07 23:53 
GeneralRe: Partial trust problem [modified] Pin
JockerSoft1-Apr-07 7:14
JockerSoft1-Apr-07 7:14 
QuestionRe: Partial trust problem Pin
A1x2-Oct-07 20:20
A1x2-Oct-07 20:20 
Generaltarget size Pin
ujr16-Jan-07 23:39
ujr16-Jan-07 23:39 
GeneralExtract quicktime 7 movie frame Pin
Phoen255-Aug-06 9:09
Phoen255-Aug-06 9:09 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.