HTTP Live Streaming (HLS-VOD)

• Download source - 109.1 KB

We are going to implement the technique by using Apple HTTP Live Streaming (HLS) with Video on Demand (VOD) using a desktop application and IIS.

Introduction

What Is Adaptive Bit-Rate Streaming?

“Adaptive bit-rate streaming is a technique used in streaming multimedia over computer networks” - Wikipedia.

The key point is adapting the streaming according to the client machine status like bandwidth up down. By using HLS, we are going to implement the thought.

What is HLS?

Apple HTTP Live Streaming known as HLS, which is a video streaming protocol, based on HTTP.

HLS Supports:

• Live streaming
• Video on demand (VOD)
• Multiple bit rates (MBR) Streaming
• Smart switching streams based on client environment
• Media encryption
• User authentication

Learn more: https://developer.apple.com/documentation/http_live_streaming

We are going to separate the whole process in different steps with the main thought of:

• Transcoding Tools
• Server Configuration & Testing

Transcoding Tools

In this step, we need to encode the input video file to different segment with VOD playlist. Using ffmpeg, we are going to transcode our input file by command.

We are going to use Windows Form application to transcode the video file. Let’s create a new windows form application and prepare the UI.

Open Visual Studio Go to > File > New > Project, then choose Windows Form Application by clicking menu > Windows Desktop.

Go to App.config to add the output folder path.

Manage Encoding Process

Add a class library which will process the task while we are going to transcode the video file. This is a custom solution based on the original source https://github.com/vladjerca/FFMpegSharp.

Which has some modification for multi-bit rate segment command.

FFMpeg

Let’s download the open source ffmpeg libraries from the below download link:

• https://www.videohelp.com/download/ffmpeg-3.1.4-win32-static.zip

Version Used- ffmpeg-3.1.4. Unzip downloaded file, there will be several folders like the below folder image.

Open bin folder to copy the below three files to application directory, in this example, the files copied to debug folder to access.

Let’s get started with the coding section.

Transcoder

First, we are going to browse the video file (.mp4) with the below code section to display the browsed path in textbox, which is our actual input path.

txtFile.Text = string.Empty;
OpenFileDialog ofdFile = new OpenFileDialog();
ofdFile.Multiselect = false;
ofdFile.Filter = "Video files (*.mp4)|*.mp4";
ofdFile.Title = "Select File.";
if (ofdFile.ShowDialog() == DialogResult.OK)
{
    var file = ofdFile.FileNames;
    txtFile.Text = file[0].ToString();
}

Next, we are going to process the video file into common video resolution with ffmpeg command:

• 1080p
• 720p
• 480p
• 360p
• 240p

Here’s the different command for each resolution:

1080p

" -i \"{0}\" -vf scale=w=1920:h=1080:force_original_aspect_ratio=decrease 
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 
-keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 5000k -maxrate 5350k 
-bufsize 7500k -b:a 192k -hls_segment_filename " + fileOutput + "1080p_%d.ts " 
+ fileOutput + "1080p.m3u8"

720p

" -i \"{0}\" -vf scale=w=1280:h=720:force_original_aspect_ratio=decrease 
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 
-hls_time 4 -hls_playlist_type vod -b:v 2800k -maxrate 2996k -bufsize 4200k -b:a 128k 
-hls_segment_filename " + fileOutput + "720p_%d.ts " 
+ fileOutput + "720p.m3u8"

480p

" -i \"{0}\" -vf scale=w=842:h=480:force_original_aspect_ratio=decrease 
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 
-keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 1400k -maxrate 1498k 
-bufsize 2100k -b:a 128k -hls_segment_filename " + fileOutput + "480p_%d.ts " 
+ fileOutput + "480p.m3u8"

360p

" -i \"{0}\" -vf scale=w=640:h=360:force_original_aspect_ratio=decrease 
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 
-keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 800k -maxrate 856k 
-bufsize 1200k -b:a 96k -hls_segment_filename " + fileOutput + "360p_%d.ts " 
+ fileOutput + "360p.m3u8"

240p

" -i \"{0}\" -vf scale=w=426:h=240:force_original_aspect_ratio=decrease 
-c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 
-keyint_min 48 -hls_time 4 -hls_playlist_type vod -b:v 240k -maxrate 240k 
-bufsize 480k -b:a 64k -hls_segment_filename " + fileOutput + "240p_%d.ts " 
+ fileOutput + "240p.m3u8"

For multiple bitrate file, we need to create a master playlist. The below code section will create a master playlist with all those command information.

//Create index as master playlist
string path = fileOutput + "index.m3u8";
File.Create(path).Dispose();
string[] line ={
    "#EXTM3U",
    "#EXT-X-VERSION:3",
    "#EXT-X-STREAM-INF:BANDWIDTH=10000,RESOLUTION=426x240",
    "240p.m3u8",
    "#EXT-X-STREAM-INF:BANDWIDTH=420000,RESOLUTION=640x360",
    "360p.m3u8",
    "#EXT-X-STREAM-INF:BANDWIDTH=680000,RESOLUTION=842x480",
    "480p.m3u8",
    "#EXT-X-STREAM-INF:BANDWIDTH=1256000,RESOLUTION=1280x720",
    "720p.m3u8",
    "#EXT-X-STREAM-INF:BANDWIDTH=2000000,RESOLUTION=1920x1080",
    "1080p.m3u8"
};

File.WriteAllLines(path, line);

Then, with the below code section, start the process by passing the command string with input source.

FFMpeg encoder = new FFMpeg();
encoder.OnProgress += encoder_OnProgress;
Task.Run(() => encoder.ToTS(inputFile, conversionArgs));

Finally, the full Transcoding event:

private void btnTranscode_Click(object sender, EventArgs e)
{
    try
    {
        bool isProcessing = IsProcessRunning("ffmpeg");
        if (!isProcessing)
        {
            if (txtFile.Text != string.Empty)
            {
                string _rootPath = Environment.CurrentDirectory;
                string ffmpegOutput = ConfigurationManager.AppSettings["ffmpegOutput"];
                this.Cursor = Cursors.WaitCursor;
                this.Text = "Transcoding...";
                btnTranscode.Text = "Transcoding..";

                string inputFile = txtFile.Text.ToString();
                string fileOutput = ffmpegOutput + 
                       toUnderscore(Path.GetFileNameWithoutExtension(inputFile)) + "\\";
                if (!Directory.Exists(fileOutput))
                {
                    SetFolderPermission(fileOutput);
                    DirectoryInfo di = Directory.CreateDirectory(fileOutput);
                }

                //Create index as master playlist
                string path = fileOutput + "index.m3u8";
                File.Create(path).Dispose();
                string[] line ={
                    "#EXTM3U",
                    "#EXT-X-VERSION:3",
                    "#EXT-X-STREAM-INF:BANDWIDTH=10000,RESOLUTION=426x240",
                    "240p.m3u8",
                    "#EXT-X-STREAM-INF:BANDWIDTH=420000,RESOLUTION=640x360",
                    "360p.m3u8",
                    "#EXT-X-STREAM-INF:BANDWIDTH=680000,RESOLUTION=842x480",
                    "480p.m3u8",
                    "#EXT-X-STREAM-INF:BANDWIDTH=1256000,RESOLUTION=1280x720",
                    "720p.m3u8",
                    "#EXT-X-STREAM-INF:BANDWIDTH=2000000,RESOLUTION=1920x1080",
                    "1080p.m3u8"
                };
                File.WriteAllLines(path, line);

                //Command
                string conversionArgs = string.Format("-hide_banner -y" +
                                        " -i \"{0}\" -vf scale=w=426:h=240:
                                          force_original_aspect_ratio=decrease -c:a 
                                          aac -ar 48000 -c:v h264 -profile:v main 
                                          -crf 20 -sc_threshold 0 -g 48 -keyint_min 48 
                                          -hls_time 4 -hls_playlist_type vod -b:v 240k 
                                          -maxrate 240k -bufsize 480k -b:a 64k 
                                          -hls_segment_filename " + fileOutput + "240p_%d.ts " 
                                          + fileOutput + "240p.m3u8" +
                                        " -i \"{0}\" -vf scale=w=640:h=360:
                                          force_original_aspect_ratio=decrease 
                                          -c:a aac -ar 48000 -c:v h264 -profile:v main -crf 20 
                                          -sc_threshold 0 -g 48 -keyint_min 48 -hls_time 4 
                                          -hls_playlist_type vod -b:v 800k -maxrate 856k 
                                          -bufsize 1200k 
                                          -b:a 96k -hls_segment_filename " + fileOutput 
                                          + "360p_%d.ts " + fileOutput + "360p.m3u8" +
                                        " -i \"{0}\" -vf scale=w=842:h=480:
                                        force_original_aspect_ratio=decrease -c:a aac -ar 48000 
                                        -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 
                                        -keyint_min 48 -hls_time 4 -hls_playlist_type vod 
                                        -b:v 1400k 
                                        -maxrate 1498k -bufsize 2100k -b:a 128k 
                                        -hls_segment_filename " 
                                        + fileOutput + "480p_%d.ts " + fileOutput 
                                        + "480p.m3u8" +
                                        " -i \"{0}\" -vf scale=w=1280:h=720:
                                        force_original_aspect_ratio=decrease -c:a aac -ar 48000 
                                        -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 
                                        -keyint_min 48 -hls_time 4 -hls_playlist_type vod 
                                        -b:v 2800k 
                                        -maxrate 2996k -bufsize 4200k -b:a 128k 
                                        -hls_segment_filename " 
                                        + fileOutput + "720p_%d.ts " + fileOutput 
                                        + "720p.m3u8" +
                                        " -i \"{0}\" -vf scale=w=1920:h=1080:
                                        force_original_aspect_ratio=decrease -c:a aac -ar 48000 
                                        -c:v h264 -profile:v main -crf 20 -sc_threshold 0 -g 48 
                                        -keyint_min 48 -hls_time 4 -hls_playlist_type vod 
                                        -b:v 5000k 
                                        -maxrate 5350k -bufsize 7500k -b:a 192k 
                                        -hls_segment_filename " 
                                        + fileOutput + "1080p_%d.ts " + fileOutput 
                                        + "1080p.m3u8", inputFile, fileOutput);

                //Process
                FFMpeg encoder = new FFMpeg();
                encoder.OnProgress += encoder_OnProgress;
                Task.Run(() => encoder.ToTS(inputFile, conversionArgs));
            }
        }
    }
    catch (Exception ex)
    {
        ex.ToString();
    }
}

The below code section will show the progress in progress bar.

void encoder_OnProgress(int percentage)
{
    try
    {
        //Update UI
        Invoke(new Action(() =>
        {
            progressBar1.Value = percentage;
            this.Text = "Transcoding..." + percentage + "%";
        }));

        if (percentage == 100)
        {
            Invoke(new Action(() =>
            {
                this.btnTranscode.Text = "Transcode";
                this.Cursor = Cursors.Default;
            }));
        }
    }
    catch (Exception ex)
    {
        ex.ToString();
    }
}

Now run the application & browse .mp4 file, then click on transcode button to start the transcoding process.

Output: Go to > C:\ProgramData\transcode

There’s a single index file with .m3u8 extension with multi-resolution information of sub playlist which is the master playlist.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-STREAM-INF:BANDWIDTH=10000,RESOLUTION=426x240
240p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=420000,RESOLUTION=640x360
360p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=680000,RESOLUTION=842x480
480p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1256000,RESOLUTION=1280x720
720p.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2000000,RESOLUTION=1920x1080
1080p.m3u8

Sub-Playlist: As you can see from master playlist 240p.m3u8 is listed which has another list of segmented file with .ts extension.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:5
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:4.004000,
240p_0.ts
#EXTINF:4.004000,
240p_1.ts
#EXT-X-ENDLIST

Learn more at this link.

Streaming Server (IIS) Configuration & Testing

Create a new website in IIS to serve the HLS live stream. We need to add MIME (Multipurpose Internet Mail Extensions) types to our website to play .m3u8 extension.

Double click on MIME Types to add new extension.

As you can see in Web.Config, configuration is added.

<configuration>
  <system.webserver>
    <staticcontent>
      <mimemap fileextension=".m3u8" mimetype="application/x-mpegURL">
    </mimemap></staticcontent>
  </system.webserver>
</configuration>

Transfer transcoded file in videos folder, then add player in index.html file to play the video from live streaming server.

Next, we are going to add video.js player for live streaming in Index.htm.

Video.Js

<link href="https://vjs.zencdn.net/7.4.1/video-js.css" rel="stylesheet" />
<script src="https://vjs.zencdn.net/ie8/1.1.2/videojs-ie8.min.js"></script>
<script src='https://vjs.zencdn.net/7.4.1/video.js'></script>
<script src="https://unpkg.com/videojs-flash/dist/videojs-flash.js"></script>
<script src="https://unpkg.com/videojs-contrib-hls/dist/videojs-contrib-hls.js"></script>

Here videojs-contrib-hls.js is HLS library for cross browser HLS support.

Player: As you can see, our source path is set to index.m3u8 file from the below source tag.

<video class="video-js vjs-default-skin vjs-big-play-centered" 
data-setup='{"controls": true, "autoplay": true, 
"aspectRatio":"16:9", "fluid": true}'>
    <source src="http://localhost:8081/videos/Sunset_at_Bhawal_Resort/index.m3u8" 
     type="application/x-mpegURL" />
</video>

Learn more: http://videojs.github.io/videojs-contrib-hls

Finally, Index.htm:

<!DOCTYPE html>
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
    <title>IIS HLS-VOD</title>
    <link href="https://vjs.zencdn.net/7.4.1/video-js.css" rel="stylesheet" />
    <script src="https://vjs.zencdn.net/ie8/1.1.2/videojs-ie8.min.js"></script>
    <script src='https://vjs.zencdn.net/7.4.1/video.js'></script>
    <script src="https://unpkg.com/videojs-flash/dist/videojs-flash.js"></script>
    <script src="https://unpkg.com/videojs-contrib-hls/dist/videojs-contrib-hls.js"></script>
</head>
<body style="margin:0;padding:0;">
    <div>
        <video class="video-js vjs-default-skin vjs-big-play-centered" 
         data-setup='{"controls": true, "autoplay": true, 
         "aspectRatio":"16:9", "fluid": true}'>
            <source src="http://localhost:8081/videos/Sunset_at_Bhawal_Resort/index.m3u8" 
             type="application/x-mpegURL" />
        </video>
    </div>
</body>
</html>

Open browser to test the streaming. Go to URL > http://localhost:8081

Client Test

I am going to use Chrome browser for testing. As you can see from the below screen, the video is playing in browser.

Change the screen size to see the adaptive live streaming like the above image. Hope this post is going to clarify the thought behind adaptive video playing also creating a video streaming server in IIS.

References

• https://developer.apple.com/documentation/http_live_streaming
• https://developer.apple.com/documentation/http_live_streaming/understanding_the_http_live_streaming_architecture
• https://developer.apple.com/documentation/http_live_streaming/example_playlists_for_http_live_streaming/event_playlist_construction
• https://developer.apple.com/documentation/http_live_streaming/example_playlists_for_http_live_streaming/video_on_demand_playlist_construction
• https://bitmovin.com/adaptive-streaming/
• https://docs.peer5.com/guides/production-ready-hls-vod/
• http://www.tothenew.com/blog/adaptive-video-streaming-hls/
• http://www.tothenew.com/blog/apple-http-live-streaming-hls/
• http://www.tothenew.com/blog/kick-start-with-video-streaming/
• https://github.com/vladjerca/FFMpegSharp