Click here to Skip to main content
15,867,771 members
Articles / Multimedia / DirectX
Article

Extract Frames from Video Files

Rate me:
Please Sign up or sign in to vote.
4.72/5 (40 votes)
26 Sep 2007CPOL3 min read 541.1K   40.2K   129   99
Class to extract frames from most video file formats using IMediaDet
Sample Image - extractvideoframes.jpg

Introduction

This class contains methods to use the IMediaDet interface that can be found in Microsoft DirectShow. The Media Detector object, among other things, can be used to extract still pictures from several file formats including *.avi, *.wmv and some *.mpeg files.

This class exposes the GetFrameFromVideo, GetVideoSize and SaveFrameFromVideo methods that can be used from any .NET application. The class also takes care of translating HRESULTs returned from the functions to meaningful .NET exceptions.

Using the Code

Just add a reference to JockerSoft.Media.dll in your project (or include the source code). Remember also to distribute Interop.DexterLib.dll.
All the methods are static, so to use them just do something like this:

C#
try
{
    this.pictureBox1.Image = FrameGrabber.GetFrameFromVideo(strVideoFile, 0.2d);
}
catch (InvalidVideoFileException ex)
{
    MessageBox.Show(ex.Message, "Extraction failed");
}
catch (StackOverflowException)
{
    MessageBox.Show("The target image size is too big", "Extraction failed");
}

or

C#
try
{
    FrameGrabber.SaveFrameFromVideo(strVideoFile, 0.2d, strBitmapFile);
}
catch (InvalidVideoFileException ex)
{
    MessageBox.Show(ex.Message, "Extraction failed");
}

Here, we used the simplest of the three overloads of GetFrameFromVideo and SaveFrameFromVideo methods, presented in this article.

Points of Interest

The IMediaDet and linked interfaces/classes are exposed in qedit.dll, that can be found in System32 directory. Fortunately this DLL can be imported automatically using tlbimp, so no code is needed to wrap it.

To extract images, there are two methods: extract them in memory (using GetBitmapBits - here GetFrameFromVideo) or extract them and save to a bitmap file (using WriteBitmapBits - here SaveFrameFromVideo).

WriteBitmapBits is really simple to be used: we just need to find the video stream on the file, open it and specify an output file name for the bitmap image.

C#
public static void SaveFrameFromVideo(string videoFile,
         double percentagePosition, string outputBitmapFile,
         out double streamLength, Size target)
{
    if (percentagePosition > 1 || percentagePosition < 0)
        throw new ArgumentOutOfRangeException("percentagePosition", 
                percentagePosition, "Valid range is 0.0 .. 1.0");

    try
    {
        MediaDetClass mediaDet;
        _AMMediaType mediaType;
        if (openVideoStream(videoFile, out mediaDet, out mediaType))
        {
            streamLength = mediaDet.StreamLength;
            
            //calculates the REAL target size of our frame
            if (target == Size.Empty)
                target = getVideoSize(mediaType);
            else
                target = scaleToFit(target, getVideoSize(mediaType));

            mediaDet.WriteBitmapBits(streamLength * percentagePosition, 
                        target.Width, target.Height, outputBitmapFile);

            return;
        }
    }
    catch (COMException ex)
    {
        throw new InvalidVideoFileException(getErrorMsg((uint)ex.ErrorCode), ex);
    }

    throw new InvalidVideoFileException("No video stream was found");
}

You'll notice that two private methods are used here. They are openVideoStream and getVideoSize. Their implementation is straight forward:

C#
private static bool openVideoStream(string videoFile, 
            out MediaDetClass mediaDetClass, out _AMMediaType aMMediaType)
{
    MediaDetClass mediaDet = new MediaDetClass();
    
    //loads file
    mediaDet.Filename = videoFile;

    //gets # of streams
    int streamsNumber = mediaDet.OutputStreams;

    //finds a video stream and grabs a frame
    for (int i = 0; i < streamsNumber; i++)
    {
        mediaDet.CurrentStream = i;
        _AMMediaType mediaType = mediaDet.StreamMediaType;

        if (mediaType.majortype == JockerSoft.Media.MayorTypes.MEDIATYPE_Video)
        {
            mediaDetClass = mediaDet;
            aMMediaType = mediaType;
            return true;
        }
    }

    mediaDetClass = null;
    aMMediaType = new _AMMediaType();
    return false;
}

(where MEDIATYPE_Video is the GUID used for video files).

C#
private static Size getVideoSize(_AMMediaType mediaType)
{
    WinStructs.VIDEOINFOHEADER videoInfo = 
        (WinStructs.VIDEOINFOHEADER)Marshal.PtrToStructure(mediaType.pbFormat, 
            typeof(WinStructs.VIDEOINFOHEADER));
    
    return new Size(videoInfo.bmiHeader.biWidth, videoInfo.bmiHeader.biHeight);
}

Using GetBitmapBits to avoid saving the image on disk is a bit trickier, since we need to deal with direct access to memory.

The first part is identical to SaveFrameFromVideo, then we have to call GetBitmapBits with the pBuffer parameter set to null to get the size for the buffer of bytes that will contain the 24bpp image (GetBitmapBits always returns 24bpp images).

Once we have the size of the buffer, we allocate memory on the heap to receive the image (in the first version of this code, memory was allocated on the stack which is fine if the target image is small sized, but if it is big we may get a nice StackOverflowException because stack memory is rather limited).

After this, we call GetBitmapBits again, but this time the buffer will be filled with image bytes. Now we create a bitmap from these bytes (remembering that they start with BITMAPINFOHEADER structure, the size of which is 40 bytes).

C#
public static Bitmap GetFrameFromVideo(string videoFile, 
            double percentagePosition, out double streamLength, Size target)
{
    if (percentagePosition > 1 || percentagePosition < 0)
        throw new ArgumentOutOfRangeException("percentagePosition", 
                percentagePosition, "Valid range is 0.0 .. 1.0");

    try 
    {
        MediaDetClass mediaDet;
        _AMMediaType mediaType;
        if (openVideoStream(videoFile, out mediaDet, out mediaType))
        {
            streamLength = mediaDet.StreamLength;

            //calculates the REAL target size of our frame
            if (target == Size.Empty)
                target = getVideoSize(mediaType);
            else
                target = scaleToFit(target, getVideoSize(mediaType));

            unsafe 
            {
                Size s= GetVideoSize(videoFile);
                //equal to sizeof(CommonClasses.BITMAPINFOHEADER);
                int bmpinfoheaderSize = 40;                 

                //get size for buffer
                int bufferSize = (((s.Width * s.Height) * 24) / 8 ) + bmpinfoheaderSize;
                //equal to mediaDet.GetBitmapBits
                //    (0d, ref bufferSize, ref *buffer, target.Width, target.Height);    

                //allocates enough memory to store the frame
                IntPtr frameBuffer = 
                    System.Runtime.InteropServices.Marshal.AllocHGlobal(bufferSize);
                byte* frameBuffer2 = (byte*)frameBuffer.ToPointer();

                //gets bitmap, save in frameBuffer2
                mediaDet.GetBitmapBits(streamLength * percentagePosition, 
                    ref bufferSize, ref *frameBuffer2, target.Width, target.Height);

                //now in buffer2 we have a BITMAPINFOHEADER structure 
                //followed by the DIB bits
                Bitmap bmp = new Bitmap(target.Width, target.Height, target.Width * 3, 
                    System.Drawing.Imaging.PixelFormat.Format24bppRgb, 
                    new IntPtr(frameBuffer2 + bmpinfoheaderSize));

                bmp.RotateFlip(RotateFlipType.Rotate180FlipX);
                System.Runtime.InteropServices.Marshal.FreeHGlobal(frameBuffer);
                return bmp;
            }
        }
    }
    catch (COMException ex)
    {
        throw new InvalidVideoFileException(getErrorMsg((uint)ex.ErrorCode), ex);
    }

    throw new InvalidVideoFileException("No video stream was found");
}

Known Limitations

  • The biggest one is the StackOverflowException using stackalloc. There must be a way to pass to GetBitmapBits a buffer created on the heap, but I'm not yet very good with this unmanaged stuff.
    • This problem has been resolved. The solution is provided by _coder_ in the comments below.
  • On my machine, I got random errors when passing certain target sizes to GetBitmapBits: when the target size is 125x125 for example, the Bitmap constructor fails.
    • Only sizes multiples of 4 are allowed. Thanks to ujr (see comments).
  • "The IMediaDet interface does not support VIDEOINFOHEADER2 formats": This means it cannot open some *.mpeg video files.

History

  • 27th February, 2006: Initial release
  • 17th March, 2006: Replaced memory allocation on the stack
  • 27th September, 2007: Some fixes

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Italy Italy

Comments and Discussions

 
Questionusage of DLL in commercial product -- any patent issues? Pin
rtswguru4-May-09 7:35
rtswguru4-May-09 7:35 
AnswerRe: usage of DLL in commercial product -- any patent issues? Pin
akula7703-Jun-10 8:01
akula7703-Jun-10 8:01 
GeneralShow only the first frame Pin
paradoxshark3-Apr-09 11:23
paradoxshark3-Apr-09 11:23 
GeneralRe: Show only the first frame Pin
JockerSoft4-Apr-09 2:27
JockerSoft4-Apr-09 2:27 
GeneralRe: Show only the first frame Pin
Lucasussei4-Nov-09 17:05
Lucasussei4-Nov-09 17:05 
GeneralRe: Show only the first frame Pin
Lucasussei25-Nov-09 2:43
Lucasussei25-Nov-09 2:43 
QuestionRe: Show only the first frame Pin
xmu_research9-Dec-09 16:05
xmu_research9-Dec-09 16:05 
NewsOpen Video with browse........... Pin
sushilabhanvar1-Apr-09 0:44
sushilabhanvar1-Apr-09 0:44 
GeneralIssue with track bar in vedio cutter................... Pin
sushilabhanvar1-Apr-09 0:38
sushilabhanvar1-Apr-09 0:38 
GeneralInsert and edit frame. Pin
neglewis11-Mar-09 3:46
neglewis11-Mar-09 3:46 
GeneralRe: Insert and edit frame. Pin
JockerSoft4-Apr-09 2:25
JockerSoft4-Apr-09 2:25 
GeneralRe: Insert and edit frame. Pin
helakhedira4-Jun-11 10:46
helakhedira4-Jun-11 10:46 
GeneralThank you Pin
xeinet14-Jan-09 22:46
xeinet14-Jan-09 22:46 
GeneralOperations on yuv files using C++ Pin
saeed.ullah14-Dec-08 22:36
saeed.ullah14-Dec-08 22:36 
GeneralThe system cannot find the path specified. (Exception from HRESULT: 0x80070003) Pin
Member 304769129-Oct-08 21:02
Member 304769129-Oct-08 21:02 
Questionwhat is name of that movie? Pin
Huisheng Chen25-Sep-08 5:37
Huisheng Chen25-Sep-08 5:37 
AnswerRe: what is name of that movie? Pin
JockerSoft25-Sep-08 8:15
JockerSoft25-Sep-08 8:15 
GeneralRe: what is name of that movie? Pin
Huisheng Chen25-Sep-08 15:57
Huisheng Chen25-Sep-08 15:57 
GeneralMultimedia Pin
sanaritasages30-Aug-08 2:19
sanaritasages30-Aug-08 2:19 
QuestionProblem while running FrameGrabber.SaveFrameFromVideo Pin
minamohebn10-Jul-08 11:51
minamohebn10-Jul-08 11:51 
GeneralErroneous image grabber Pin
Radu Simionescu6-Jul-08 12:53
Radu Simionescu6-Jul-08 12:53 
GeneralRe: Erroneous image grabber Pin
JockerSoft25-Sep-08 8:12
JockerSoft25-Sep-08 8:12 
GeneralException from HRESULT: 0x80040200 Pin
nithydurai27-Jun-08 0:38
nithydurai27-Jun-08 0:38 
GeneralRe: Exception from HRESULT: 0x80040200 Pin
JockerSoft25-Sep-08 8:10
JockerSoft25-Sep-08 8:10 
GeneralProblem with Vista Pin
maaldo6-May-08 13:41
maaldo6-May-08 13:41 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.