Creating a Subtitle/Closed Captions Plugin for SMF (Silverlight Media Framework)

• Download source - 3.56 KB

Introduction

Silverlight Media Framework (or SMF), now a part of Microsoft Media Platform: Player Framework, is a feature-rich framework for developing media players in Silverlight. It contains a fairly comprehensive and skinnable UI, and supports a reasonable amount of video formats and delivery methods out-of-the-box. But currently, it's really lacking in the area of subtitles/closed-captions. This article will walk you through the creation of custom caption plugins to support any format you like.

The Code: Preamble

• At the time of writing, MMPPF version 2.5 is the latest 'stable'. All development has been done against this version.
• As this is extracted from a large scale closed-source commercial project, I cannot just provide the whole source, due to company rules, etc... Sorry about that!
• I will start by creating a base class, which isn't very useful if you only want to support one format, but I swear this is not due to over-engineering! Once again, this is an excerpt from a much larger codebase, which features more than one supported format.
• The file downloading part will be left out of the scope of this article. In my case, I have a complex utility class that handles both downloading and local IsolatedStorage caching. You can choose to handle the downloading in the plugin or outside, or use local files, whatever suits you best. A simplistic download solution is provided here.
• This has been moderately tested only, as the code is quite 'fresh', so some bugs can still be present. Please report them in the comments!

SMF Plugin Basics

Before writing SMF plugins, you can read some official documentation:

• The 'Plugin Developer Guide' is useful, it gives an overview and some details about the process of creating plugins, but it is not quite complete.
• The API documentation (CHM) comes in handy quite often, but is sometimes outdated (see the 'Architecture' page for an example: no IMarkerProvider!)
• The source code is sometimes the only real help, especially since it features some interesting samples, see 'Samples.MarkerProvider' in this case.

As SMF plugins make use of MEF (Managed Extensibility Framework), it could be useful to get acquainted with this beast. This is not mandatory if you just want to get going, but it could help a lot when debugging plugin loading, concurrency or performance issues.

The Implementation

The Common Base Class: SubTitlePlugin

The IPlugin interfaces contain quite a lot of members (see picture). Fortunately, you're probably using Visual Studio 2008 or 2010 to work with Silverlight, so you'll be able to save a lot of typing using the awesome 'implement interface' feature.

So let's get started writing code! First, I'll be creating a new empty class, named SubTitlePlugin, abstract and which implements Microsoft.SilverlightMediaFramework.Plugins.IMarkerProvider. This class will provide all common features for subtitle plugins, reducing code 'noise' when working with actual plugins. This will be very much 'required boilerplate'. Anyway, if you're not copy/pasting my code, you will want to right-click IMarkerProvider and select 'Implement interface' => 'Implement interface' in Visual Studio. This will generate stubs for all interface members, properties, methods and events. Now, let's fill our first public method.

/// <summary>
/// Start the retrieval of markers.
/// This is asynchronous.
/// </summary>
public void BeginRetrievingMarkers()
{
    if (Source == null)
        throw new ArgumentNullException("This requires a valid Source to download from");

    _cancelRequested = false;

#if USE_DOWNLOAD_MANAGER
    // Initiate file download.
    var localFileName = GetFileName(Source);
    DownloadManager.DownloadFile(url = Source, localFileName = localFileName, 
	source = this, maxAge = null, completed = ReadSubtitles, error = DownloadError);
#elif USE_WEBCLIENT
    // Basic WebClient version:
    var client = new WebClient();
    client.DownloadStringCompleted += client_DownloadStringCompleted;
    client.DownloadStringAsync(Source);
#endif
}

I've provided here a basic WebClient-based retrieval that you might use, but in the next steps, I will use my custom DownloadManager-based code. The DownloadManager method parameters should be self-explanatory, anyway, the interesting point here is that once the file is successfully downloaded, it will call ReadSubtitles, which will be our next method below, along with some private members and the other callback method DownloadError. If you were using the WebClient method, the main difference is that you get a string result instead of a byte[].

private bool _cancelRequested;
private MediaMarkerCollection<MediaMarker> _subtitles;

private void ReadSubtitles(object source, byte[] data)
{
    if (_cancelRequested)
    {
        _cancelRequested = false;
        return;
    }

    try
    {
        _subtitles = LoadMarkers(data);
    }
    catch (Exception ex)
    {
        if (RetrieveMarkersFailed != null)
            RetrieveMarkersFailed(this, ex);
        return;
    }

    if (NewMarkers != null)
        NewMarkers(this, _subtitles);
}

private void DownloadError(object source, Exception error)
{
    if (RetrieveMarkersFailed != null)
        RetrieveMarkersFailed(this, error);
}

This is a very important method for our plugins. It calls the abstract protected LoadMarkers method, which will do the actual parsing of our chosen format. It also handles some mandatory event handling, especially the NewMarkers call, where the plugin is notifying the SMFPlayer that it has some content to provide, remember the interface name IMarkerProvider? If you were to forget this call, nothing would happen at all for the end user! Your plugin could properly parse the plugins and all, but they would never show up in the player!

Note: If you don't like the endless if (x != null) x(...); event firing constructs, the SMF source contains an extension method IfNotNull, that you might want to check out.

Now we have the core of the code for SubTitlePlugin! Most of the rest is automatic properties and events, except for the below two methods: Load and Unload. As their names imply, they contain initialization and cleanup code respectively.

/// <summary>
/// Loads the plug-in.
/// </summary>
public void Load()
{
    try
    {
        DoLoadPlugin();
        IsLoaded = true;
        if (PluginLoaded != null)
            PluginLoaded(this);
    }
    catch (Exception ex)
    {
        if (PluginLoadFailed != null)
            PluginLoadFailed(this, ex);
    }
}

/// <summary>
/// Unloads the plug-in.
/// </summary>
public void Unload()
{
    try
    {
        DoUnloadPlugin();
        _subtitles = null;

        IsLoaded = false;
        if (PluginUnloaded != null)
            PluginUnloaded(this);
    }
    catch (Exception ex)
    {
        if (PluginUnloadFailed != null)
            PluginUnloadFailed(this, ex);
    }
}

These methods are pretty straightforward: they handle event firing and provide two protected virtual (hence optional) methods to allow derived class to perform specific loading/unloading tasks.

That's all for the base class. You can find it in the source package here.

A Concrete Sub-class: SamiProvider

Up to this point, we didn't have to use anything related to MEF. The base class isn't instanciable, so it doesn't require MEF decorators. Our concrete format-specific classes on the other hand require some decorator 'magic' to be found by the SMFPlayer when it looks for plugins. The 'Plugin Developer Guide' provides good information here.

So, let's create a second class named SamiProvider which will be derived from SubTitlePlugin. Here is the whole code for this class:

/// <summary>
/// This class provides SAMI subtitle support as a SMF player plugin.
/// </summary>
[ExportMarkerProvider(
    PluginName = PluginName,
    PluginDescription = PluginDescription,
    PluginVersion = PluginVersion,
    SupportsPolling = false,
    SupportedFormat = "SAMI")]
[PartCreationPolicy(CreationPolicy.NonShared)]
public class SamiProvider : SubTitlePlugin
{
    #region MetaData
    private const string PluginName = "SAMICaptionsPlugin";
    private const string TypeAttribute = "type";
    private const string SubTypeAttribute = "subtype";
    private const string PluginDescription = 
	"Provides SMF with the ability to parse and display SAMI captions.";
    private const string PluginVersion = "1.0";
    #endregion

    /// <summary>
    /// Load the markers from the data.
    /// </summary>
    /// <param name="data">A SAMI file as a byte array</param>
    /// <returns>A collection of markers suitable for use with SMF</returns>
    protected override MediaMarkerCollection<MediaMarker> LoadMarkers(byte[] data)
    {
        SamiLoader loader = new SamiLoader();
        return loader.ParseSAMI(data);
    }
}

Note: The whole MEF part is heavily based on the official sample mentioned earlier ('Samples.MarkerProvider' in the sources for MMPPF).

The first decorator is ExportMarkerProvider. Its three first parameters come from IPlugin, they are just a textual identity for the plugin. The other parameters are specific to IMarkerProvider.

• SupportsPolling is false for this family of plugins, it's disabled in the SubTitlePlugin (see PollingInterval), it is not relevant for file-based, downloaded subtitles. It would be useful if using for instance a stream of markers.
• The last parameter SupportedFormat is actually the most important of all. It's this parameter that will allow selection of this plugin to handle SAMI subtitles.

The second decorator, PartCreationPolicy, configures the instantiation of the plugin. I will direct you to the almighty MSDN for details. Here, we're saying we want one instance of the plugin per subtitle file. This is important as the retrieval and parsing are asynchronous, so we don't want multiple requests to mess up our plugin.

The SamiLoader class is where I parse the downloaded data into a collection of MediaMarkers. The real gotcha there is the use of CaptionRegion. You should create one for each continuous block of text that you wish to display.

Using the Plugin

Now we have the structure to create plugins, but they won't be activated yet. In my project, I'm using a CustomPlaylistItem, but for the sake of simplicity, I will just explain the method used. If you're interested by the custom playlist item, you can find it in the sources.

void SetupMarkers(PlaylistItem playListItem, string subtitleUrl)
{
    string format = "SAMI";
    var markers = new MarkerResource { Source = new Uri(subtitleUrl), Format = format };
    if (playListItem.MarkerResources != null)
    {
        playListItem.MarkerResources.Clear();
    }
    else
    {
        playListItem.MarkerResources = new List<MarkerResource>();
    }
    playListItem.MarkerResources.Add(markers);
}

The key point here is the format specification. This will allow the right plugin to be selected. If you have multiple plugins, you can pick the one that matches your subtitle format by passing the corresponding string here.

History

• 20 Sept 2011 - First version