Integration of Blackmagic Video Frames into DirectX Scene

Introduction

I am going to share with you some tips on how to display the images captured by Blackmagic in a DirectX environment.

Creating the Texture

First thing is trying to use IDeckLinkDX9ScreenPreviewHelper interface which does everything for you. The problem with this class is that you don't have a texture in your hand so you can place it wherever you want. Anyway if you want to use this class, you will notice that it will mess up the whole scene. The trick is to have it surrounded by Sprite::Being Sprite::End. Example:

m_device->SetTransform(D3DTS_WORLD, &matrix); 
m_sprite->Begin(D3DXSPRITE_OBJECTSPACE | D3DXSPRITE_ALPHABLEND);
helper->Render(NULL);
m_sprite->End();

In my case, I wanted to have a full control of the texture, and have the smallest latency possible, which I guess all developers want.

The basic idea is to use the format D3DFMT_UYVY which is supported by DirectX, at least in my case, where I have an NVIDIA QUADO 600. That means that I only process the format bmdFormat8BitYUV.

If you want to tell the driver to use this, you can specify it in a parameter IDeckLinkInput::EnableVideoInput function. Example:

HRESULT MFDeckLinkImpl::VideoInputFormatChanged 
(/* in */ BMDVideoInputFormatChangedEvents notificationEvents, /* in 
*/ IDeckLinkDisplayMode *newMode, /* in */ BMDDetectedVideoInputFormatFlags detectedSignalFlags)
    {    
        Error* error = NULL;
            //if (detectedSignalFlags & bmdDetectedVideoInputRGB444)
            //    pixelFormat = bmdFormat10BitRGB;

            HRC(m_deckLinkInput->StopStreams());
            HRC(m_deckLinkInput->EnableVideoInput
               (newMode->GetDisplayMode(), bmdFormat8BitYUV, bmdVideoInputEnableFormatDetection));
            HRC(m_deckLinkInput->StartStreams());
        }

done:
        if (error)
            return error->Hr;

        return S_OK;
    } 

This way, you will force all the frames received to be in 8 bit YUV. For SDI, you will notice that it doesn't work, but changing one settings will do the work. Go to "Blackmagic Desktop Video Setup" conversion, and choose SD->HD and Anamorphic transformation. This way, the frames will be well formatted with 8 bit YUV data.

Next, we need to have the 8-bit YUV image transformed into a texture. The way I did it is to create an offline surface:

m_device->CreateOffscreenPlainSurface(_width, _height, D3DFMT_UYVY, D3DPOOL_DEFAULT, &m_surface, NULL);

and whenever you received a frame in DrawFrame callback method, just fill in the surface with data from "theFrame".

m_surface->LockRect(&rect, 0, D3DLOCK_DISCARD);
memcpy(rect.pBits, _buffer, _bufferSize);
m_surface->UnlockRect();

_buffer is coming from theFrame->GetBytes(); _bufferSize is width * height * 2 in case of 8-bit YUV;

You need to stretch the surface into a texture with RGB format, so you create the texture:

m_device->CreateTexture(_width, _height, 1, D3DUSAGE_RENDERTARGET, 
D3DFMT_X8R8G8B8, D3DPOOL_DEFAULT, &m_texture, NULL);

then copy the surface:

IDirect3DSurface9* surface;

m_texture->GetSurfaceLevel(0, &surface);
HRESULT hr = m_device->StretchRect(m_surface, NULL, surface,  NULL, D3DTEXF_NONE);
surface->Release();

Now you have the texture and you can do everything that you want with it :).

Consideration

You should be careful how you design the app, usually the DrawFrame runs on a different thread than your DirectX scene, and if you use the transformation above in the DrawFrame thread, LockRect and StrechRect will cause delay. What I did instead, I have made a copy of the bytes received from "theFrame", that is on the DrawFrame thread, then I use the byte array and stretch the image and texture before drawing it on the scene thread.

Hope this helps you to get the best performance when using DirectX and Blackmagic card.