Click here to Skip to main content
15,868,005 members
Articles / Multimedia / DirectX
Article

Extract Frames from Video Files

Rate me:
Please Sign up or sign in to vote.
4.72/5 (40 votes)
26 Sep 2007CPOL3 min read 541.1K   40.2K   129   99
Class to extract frames from most video file formats using IMediaDet
Sample Image - extractvideoframes.jpg

Introduction

This class contains methods to use the IMediaDet interface that can be found in Microsoft DirectShow. The Media Detector object, among other things, can be used to extract still pictures from several file formats including *.avi, *.wmv and some *.mpeg files.

This class exposes the GetFrameFromVideo, GetVideoSize and SaveFrameFromVideo methods that can be used from any .NET application. The class also takes care of translating HRESULTs returned from the functions to meaningful .NET exceptions.

Using the Code

Just add a reference to JockerSoft.Media.dll in your project (or include the source code). Remember also to distribute Interop.DexterLib.dll.
All the methods are static, so to use them just do something like this:

C#
try
{
    this.pictureBox1.Image = FrameGrabber.GetFrameFromVideo(strVideoFile, 0.2d);
}
catch (InvalidVideoFileException ex)
{
    MessageBox.Show(ex.Message, "Extraction failed");
}
catch (StackOverflowException)
{
    MessageBox.Show("The target image size is too big", "Extraction failed");
}

or

C#
try
{
    FrameGrabber.SaveFrameFromVideo(strVideoFile, 0.2d, strBitmapFile);
}
catch (InvalidVideoFileException ex)
{
    MessageBox.Show(ex.Message, "Extraction failed");
}

Here, we used the simplest of the three overloads of GetFrameFromVideo and SaveFrameFromVideo methods, presented in this article.

Points of Interest

The IMediaDet and linked interfaces/classes are exposed in qedit.dll, that can be found in System32 directory. Fortunately this DLL can be imported automatically using tlbimp, so no code is needed to wrap it.

To extract images, there are two methods: extract them in memory (using GetBitmapBits - here GetFrameFromVideo) or extract them and save to a bitmap file (using WriteBitmapBits - here SaveFrameFromVideo).

WriteBitmapBits is really simple to be used: we just need to find the video stream on the file, open it and specify an output file name for the bitmap image.

C#
public static void SaveFrameFromVideo(string videoFile,
         double percentagePosition, string outputBitmapFile,
         out double streamLength, Size target)
{
    if (percentagePosition > 1 || percentagePosition < 0)
        throw new ArgumentOutOfRangeException("percentagePosition", 
                percentagePosition, "Valid range is 0.0 .. 1.0");

    try
    {
        MediaDetClass mediaDet;
        _AMMediaType mediaType;
        if (openVideoStream(videoFile, out mediaDet, out mediaType))
        {
            streamLength = mediaDet.StreamLength;
            
            //calculates the REAL target size of our frame
            if (target == Size.Empty)
                target = getVideoSize(mediaType);
            else
                target = scaleToFit(target, getVideoSize(mediaType));

            mediaDet.WriteBitmapBits(streamLength * percentagePosition, 
                        target.Width, target.Height, outputBitmapFile);

            return;
        }
    }
    catch (COMException ex)
    {
        throw new InvalidVideoFileException(getErrorMsg((uint)ex.ErrorCode), ex);
    }

    throw new InvalidVideoFileException("No video stream was found");
}

You'll notice that two private methods are used here. They are openVideoStream and getVideoSize. Their implementation is straight forward:

C#
private static bool openVideoStream(string videoFile, 
            out MediaDetClass mediaDetClass, out _AMMediaType aMMediaType)
{
    MediaDetClass mediaDet = new MediaDetClass();
    
    //loads file
    mediaDet.Filename = videoFile;

    //gets # of streams
    int streamsNumber = mediaDet.OutputStreams;

    //finds a video stream and grabs a frame
    for (int i = 0; i < streamsNumber; i++)
    {
        mediaDet.CurrentStream = i;
        _AMMediaType mediaType = mediaDet.StreamMediaType;

        if (mediaType.majortype == JockerSoft.Media.MayorTypes.MEDIATYPE_Video)
        {
            mediaDetClass = mediaDet;
            aMMediaType = mediaType;
            return true;
        }
    }

    mediaDetClass = null;
    aMMediaType = new _AMMediaType();
    return false;
}

(where MEDIATYPE_Video is the GUID used for video files).

C#
private static Size getVideoSize(_AMMediaType mediaType)
{
    WinStructs.VIDEOINFOHEADER videoInfo = 
        (WinStructs.VIDEOINFOHEADER)Marshal.PtrToStructure(mediaType.pbFormat, 
            typeof(WinStructs.VIDEOINFOHEADER));
    
    return new Size(videoInfo.bmiHeader.biWidth, videoInfo.bmiHeader.biHeight);
}

Using GetBitmapBits to avoid saving the image on disk is a bit trickier, since we need to deal with direct access to memory.

The first part is identical to SaveFrameFromVideo, then we have to call GetBitmapBits with the pBuffer parameter set to null to get the size for the buffer of bytes that will contain the 24bpp image (GetBitmapBits always returns 24bpp images).

Once we have the size of the buffer, we allocate memory on the heap to receive the image (in the first version of this code, memory was allocated on the stack which is fine if the target image is small sized, but if it is big we may get a nice StackOverflowException because stack memory is rather limited).

After this, we call GetBitmapBits again, but this time the buffer will be filled with image bytes. Now we create a bitmap from these bytes (remembering that they start with BITMAPINFOHEADER structure, the size of which is 40 bytes).

C#
public static Bitmap GetFrameFromVideo(string videoFile, 
            double percentagePosition, out double streamLength, Size target)
{
    if (percentagePosition > 1 || percentagePosition < 0)
        throw new ArgumentOutOfRangeException("percentagePosition", 
                percentagePosition, "Valid range is 0.0 .. 1.0");

    try 
    {
        MediaDetClass mediaDet;
        _AMMediaType mediaType;
        if (openVideoStream(videoFile, out mediaDet, out mediaType))
        {
            streamLength = mediaDet.StreamLength;

            //calculates the REAL target size of our frame
            if (target == Size.Empty)
                target = getVideoSize(mediaType);
            else
                target = scaleToFit(target, getVideoSize(mediaType));

            unsafe 
            {
                Size s= GetVideoSize(videoFile);
                //equal to sizeof(CommonClasses.BITMAPINFOHEADER);
                int bmpinfoheaderSize = 40;                 

                //get size for buffer
                int bufferSize = (((s.Width * s.Height) * 24) / 8 ) + bmpinfoheaderSize;
                //equal to mediaDet.GetBitmapBits
                //    (0d, ref bufferSize, ref *buffer, target.Width, target.Height);    

                //allocates enough memory to store the frame
                IntPtr frameBuffer = 
                    System.Runtime.InteropServices.Marshal.AllocHGlobal(bufferSize);
                byte* frameBuffer2 = (byte*)frameBuffer.ToPointer();

                //gets bitmap, save in frameBuffer2
                mediaDet.GetBitmapBits(streamLength * percentagePosition, 
                    ref bufferSize, ref *frameBuffer2, target.Width, target.Height);

                //now in buffer2 we have a BITMAPINFOHEADER structure 
                //followed by the DIB bits
                Bitmap bmp = new Bitmap(target.Width, target.Height, target.Width * 3, 
                    System.Drawing.Imaging.PixelFormat.Format24bppRgb, 
                    new IntPtr(frameBuffer2 + bmpinfoheaderSize));

                bmp.RotateFlip(RotateFlipType.Rotate180FlipX);
                System.Runtime.InteropServices.Marshal.FreeHGlobal(frameBuffer);
                return bmp;
            }
        }
    }
    catch (COMException ex)
    {
        throw new InvalidVideoFileException(getErrorMsg((uint)ex.ErrorCode), ex);
    }

    throw new InvalidVideoFileException("No video stream was found");
}

Known Limitations

  • The biggest one is the StackOverflowException using stackalloc. There must be a way to pass to GetBitmapBits a buffer created on the heap, but I'm not yet very good with this unmanaged stuff.
    • This problem has been resolved. The solution is provided by _coder_ in the comments below.
  • On my machine, I got random errors when passing certain target sizes to GetBitmapBits: when the target size is 125x125 for example, the Bitmap constructor fails.
    • Only sizes multiples of 4 are allowed. Thanks to ujr (see comments).
  • "The IMediaDet interface does not support VIDEOINFOHEADER2 formats": This means it cannot open some *.mpeg video files.

History

  • 27th February, 2006: Initial release
  • 17th March, 2006: Replaced memory allocation on the stack
  • 27th September, 2007: Some fixes

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Italy Italy

Comments and Discussions

 
QuestionThanks and updates Pin
Maurício Gouveia3-Jun-20 4:19
Maurício Gouveia3-Jun-20 4:19 
GeneralMy vote of 1 Pin
hamid norouzi11-May-20 4:58
hamid norouzi11-May-20 4:58 
BugVIDEOINFOHEADER2 won't initialize correctly in x64 Pin
Member 997965110-Aug-15 12:11
Member 997965110-Aug-15 12:11 
Questioncan we extract key frame using this? Pin
Member 110823658-Nov-14 21:34
Member 110823658-Nov-14 21:34 
Questiondoes not works if Pin
AbdEllah Gogop20-Oct-14 23:30
AbdEllah Gogop20-Oct-14 23:30 
QuestionMP4 Pin
vincent6419-Jun-14 23:05
vincent6419-Jun-14 23:05 
GeneralMy vote of 1 Pin
GMBarak1-Apr-14 11:04
GMBarak1-Apr-14 11:04 
doesn't work
QuestionCrash Pin
jeff parnau12-Nov-13 1:59
jeff parnau12-Nov-13 1:59 
QuestionMP4 format file not working Pin
Sabyasachi Maity15-May-13 21:57
Sabyasachi Maity15-May-13 21:57 
QuestionMaximum number of accesses before it returns error Pin
hariprasad1020-Nov-12 16:08
hariprasad1020-Nov-12 16:08 
GeneralMy vote of 1 Pin
Mohamed Hamouda8-Oct-12 7:37
Mohamed Hamouda8-Oct-12 7:37 
Questionasking for flv video image extract in c# Pin
Member 927580727-Sep-12 18:24
Member 927580727-Sep-12 18:24 
Questionmp4 files Pin
igorchirok9-Jul-12 8:50
igorchirok9-Jul-12 8:50 
QuestionGetting an error when using example Pin
Munken_0621-Jun-12 23:07
Munken_0621-Jun-12 23:07 
GeneralMy vote of 5 Pin
Joe Sonderegger15-May-12 7:49
Joe Sonderegger15-May-12 7:49 
QuestionPossibe to extract from MKV file? Pin
babzog25-Apr-12 6:39
babzog25-Apr-12 6:39 
QuestionExtracting all frames but only first frame is extracted from .avi file Pin
farid vanzara121-Apr-12 1:32
farid vanzara121-Apr-12 1:32 
QuestionWhy does Target size must be a multiple of 4 ? Pin
rolbackse22-Jan-12 17:39
rolbackse22-Jan-12 17:39 
GeneralDo not use this Pin
drweb8621-Dec-10 8:21
drweb8621-Dec-10 8:21 
GeneralRe: Do not use this Pin
drweb8621-Dec-10 9:11
drweb8621-Dec-10 9:11 
GeneralDexterlib (distribution or source location) Pin
Member 39968537-Dec-10 14:05
Member 39968537-Dec-10 14:05 
GeneralSoftware is not supporting to mpg Pin
vijayr18-Nov-10 19:17
vijayr18-Nov-10 19:17 
GeneralCapture frame at given time within video Pin
akula7702-Jun-10 5:39
akula7702-Jun-10 5:39 
GeneralJockerSoft.Media.dll Pin
Saira138-Apr-10 5:18
Saira138-Apr-10 5:18 
QuestionHow to get it working on 64-bit CPU Pin
Dmitri Nеstеruk25-Aug-09 3:45
Dmitri Nеstеruk25-Aug-09 3:45 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.