Click here to Skip to main content
15,868,016 members
Articles / Programming Languages / C++
Tip/Trick

Custom Media Sink for Use with Media Foundation Topologies and WinRT/WRL MediaCapture

Rate me:
Please Sign up or sign in to vote.
4.50/5 (3 votes)
26 Dec 2015CPOL1 min read 33.4K   408   5   10
Custom Media Sink for use with Media Foundation Topologies and WinRT/WRL MediaCapture

Introduction

WinRT and WRL introduce the requirement that most Media Foundation interfaces can no longer be directly created. If one wants to record audio or video through a sink, the only way to do so is with quite literally hundreds to thousands of lines of code in a custom sink. This prototype will get someone started towards achieving that with greater ease. Native C++ and C++/CX compatible. This is also compatible with non-WinRT/non-WRL uses since custom sinks could also be used prior to Windows 8 but generally shortcut interfaces were used.

Background

Knowledge of WinRT, WRL, C++, C++/CX, MediaCapture, Media Foundation, ATL and COM are recommended.

Using the Code

The block of code here is for all scenarios including non-WinRT/non-WRL cases where it is bound as an output node of a topology.

C++
#ifdef _WINRT_DLL
#ifdef __cplusplus_winrt
Windows::Media::MediaProperties::VideoEncodingProperties^ vidProps;
Windows::Media::MediaProperties::AudioEncodingProperties^ audProps;
Windows::Media::MediaProperties::MediaEncodingProfile^ medEncProf = 
        ref new Windows::Media::MediaProperties::MediaEncodingProfile;
medEncProf->Video = vidProps;
medEncProf->Audio = audProps;
//set any properties to pass to the Custom Sink in a PropertySet as a best practice
Windows::Foundation::Collections::IPropertySet^ pSet = 
        ref new Windows::Foundation::Collections::PropertySet();
Windows::Media::Capture::MediaCapture^ medCap;
HRESULT hr = medCap->StartPreviewToCustomSinkAsync(medEncProf, 
    ref new Platform::String(RuntimeClass_TimeStamp_WebCamSecExtensions_MediaSink), pSet);
//when finished
hr = medCap->StopPreviewAsync();
#else
ABI::Windows::Media::Capture::IMediaCapture* medCap;
ABI::Windows::Media::MediaProperties::IVideoEncodingProperties* vidProps;
ABI::Windows::Media::MediaProperties::IAudioEncodingProperties* audProps;
ABI::Windows::Foundation::IAsyncAction** pAction;
HRESULT hr;
Microsoft::WRL::ComPtr<ABI::Windows::Media::Capture::IMediaCaptureVideoPreview> imedPrevCap;
if (pAction == nullptr) return E_POINTER;
hr = medCap->QueryInterface<ABI::Windows::Media::Capture::IMediaCaptureVideoPreview>(&imedPrevCap);
if (FAILED(hr)) return hr;
Microsoft::WRL::ComPtr<IActivationFactory> objFactory;
hr = Windows::Foundation::GetActivationFactory(Microsoft::WRL::Wrappers::HStringReference
    (RuntimeClass_Windows_Media_MediaProperties_MediaEncodingProfile).Get(), objFactory.ReleaseAndGetAddressOf());
if (FAILED(hr)) return hr;
Microsoft::WRL::ComPtr<IInspectable> medEncProf;
hr = objFactory->ActivateInstance(medEncProf.ReleaseAndGetAddressOf());
if (FAILED(hr)) return hr;
Microsoft::WRL::ComPtr<ABI::Windows::Media::MediaProperties::IMediaEncodingProfile> encProps;
hr = medEncProf.As<ABI::Windows::Media::MediaProperties::IMediaEncodingProfile>(&encProps);
if (FAILED(hr)) return hr;
hr = encProps->put_Video(vidProps);
if (FAILED(hr)) return hr;
//if audio desired must also set an audio sink stream
hr = encProps->put_Audio(audProps);
if (FAILED(hr)) return hr;
hr = Windows::Foundation::GetActivationFactory(Microsoft::WRL::Wrappers::HStringReference
    (RuntimeClass_Windows_Foundation_Collections_PropertySet).Get(), objFactory.ReleaseAndGetAddressOf());
if (FAILED(hr)) return hr;
Microsoft::WRL::ComPtr<IInspectable> pSet;
hr = objFactory->ActivateInstance(pSet.GetAddressOf());
if (FAILED(hr)) return hr;
Microsoft::WRL::ComPtr<ABI::Windows::Foundation::Collections::IPropertySet> pProps;
hr = pSet.As(&pProps);
if (FAILED(hr)) return hr;
Microsoft::WRL::ComPtr<ABI::Windows::Foundation::Collections::IMap<HSTRING, IInspectable *>> spSetting;
hr = pSet.As(&spSetting);
if (FAILED(hr)) return hr;
hr = Windows::Foundation::GetActivationFactory(Microsoft::WRL::Wrappers::HStringReference
    (RuntimeClass_Windows_Foundation_PropertyValue).Get(), objFactory.ReleaseAndGetAddressOf());
if (FAILED(hr)) return hr;
Microsoft::WRL::ComPtr<ABI::Windows::Foundation::IPropertyValueStatics> spPropVal;
hr = objFactory.As(&spPropVal);
if (FAILED(hr)) return hr;
hr = imedPrevCap->StartPreviewToCustomSinkIdAsync(encProps.Get(), Microsoft::WRL::Wrappers::HStringReference
    (RuntimeClass_TimeStamp_WebCamSecExtensions_MediaSink).Get(), pProps.Get(), pAction);
//when finished
hr = imedPrevCap->StopPreviewAsync(pAction);
#endif
#else
CComPtr<IMFTopologyNode> pNode; // Create the node.
HRESULT hr = MFCreateTopologyNode(MF_TOPOLOGY_OUTPUT_NODE, &pNode);
if (SUCCEEDED(hr)) {
CComPtr<IMFMediaSink> pSink;
hr = TimeStamp::WebCamSecExtensions::MediaSink::CreateInstance<IMFMediaSink>(&pSink);
if (FAILED(hr)) return hr;
CComPtr<IMFAttributes> pAttr;
hr = pSink->QueryInterface(&pAttr);
if (FAILED(hr)) return hr;
CComPtr<IMFMediaTypeHandler> pHandler; hr = pSD->GetMediaTypeHandler(&pHandler);
if (FAILED(hr)) return hr;
CComPtr<IMFMediaType> pMediaType;
hr = pHandler->GetCurrentMediaType(&pMediaType);
if (FAILED(hr)) return hr;
hr = pAttr->SetUnknown(MF_MEDIASINK_PREFERREDTYPE, pMediaType.Detach());
if (FAILED(hr)) return hr;
hr = pNode->SetObject(pSink);
}
#endif

Points of Interest

Passing parameters is done through a PropertyBag in WinRT/WRL yet through the IMFAttribute interface in non-WinRT/non-WRL code which can be found through IUnknown QueryInterface on the Custom Sink object.

An object of type ICustomSinkCallback must be passed and the interface implemented to utilize the features including initialization, cleanup, property change notification, preparing for samples and sample processing.

When initializing the custom sink, if planning to reuse the MediaCapture object, then be sure to pass the activation name to StartPreviewToCustomSinkAsync and not an instance of a CustomSink object. There is a bug in the MediaCapture library which causes it to become in a bad state if not using the activation ID which is present in Windows 8 and Windows 8.1.

The code is compatible with Visual Studio 2012/2013/2015 and Windows 8/8.1/10.

History

  • Initial version

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Software Developer (Senior)
United States United States
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
QuestionMy Previous Post Pin
Eitan Barazani31-Oct-14 6:11
Eitan Barazani31-Oct-14 6:11 
QuestionReading RAW video stream from a Windows Phone/Tablet camera (in WinRT) Pin
Eitan Barazani29-Oct-14 9:25
Eitan Barazani29-Oct-14 9:25 
AnswerRe: Reading RAW video stream from a Windows Phone/Tablet camera (in WinRT) Pin
Gregory Morse29-Oct-14 10:34
Gregory Morse29-Oct-14 10:34 
GeneralRe: Reading RAW video stream from a Windows Phone/Tablet camera (in WinRT) Pin
Eitan Barazani29-Oct-14 11:25
Eitan Barazani29-Oct-14 11:25 
QuestionIs FTM a must have? Pin
zxli6-Jun-14 19:12
zxli6-Jun-14 19:12 
AnswerRe: Is FTM a must have? Pin
Gregory Morse8-Jun-14 0:18
Gregory Morse8-Jun-14 0:18 
QuestionIs this something I would need given my requirement? Pin
Steven123456789026-May-14 8:56
Steven123456789026-May-14 8:56 
AnswerRe: Is this something I would need given my requirement? Pin
Gregory Morse26-May-14 12:47
Gregory Morse26-May-14 12:47 
1) You could use this code to do what you want. But you could also see my other article [^] on getting audio interfaces as I do not recommend this approach for audio. It is only recommended for video and the audio is there for demonstration or to supplement the video as generally a substitute for the SampleGrabberSink interface which is lacking. You can in few lines of code and in pure C# grab an IAudioClient2 interface and be processing audio samples without saving them in Windows 8/8.1 in no time.

2) It can be implemented in C# as a COM class but the amount of translation to get that to work is quite significant (thousands of lines of code). Since this is a COM object already, you do not need a wrapper, just export the interface in IDL and import the typelib in your C# project and you can use it.

3) I have yet to investigate WP 8.1 but one of the next updates in all of my projects will be to add support for it given that VS 2013 Update 2 has added universal app support which is something I would like to implement in my own apps. Right now I have no idea if there are special considerations.
QuestionThis is really helpful... Pin
zxli20-May-14 13:13
zxli20-May-14 13:13 
AnswerRe: This is really helpful... Pin
Gregory Morse20-May-14 16:07
Gregory Morse20-May-14 16:07 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.