Video Manipulation with the Canvas Element

Gil Fink

Rate me:

5.00/5 (2 votes)

17 Aug 2011CPOL2 min read

19.9K

This post will explain how to combine the video and canvas elements in order to create transparency inside a video during runtime using JavaScript.

Video Manipulation with the Canvas Element

One of the designers in my company asked me to help him with the creation of a transparent video which runs on top of an image in HTML. So I did some digging and found a very good resource to do that - Manipulating video using canvas. This post will explain how to combine the video and canvas elements in order to create transparency inside a video during runtime using JavaScript. I’ve based the code on the previous link so they deserve all the credit. You can download the demo code from here.

The Demands

The designer has created a simple movie file with a red square that is floating on top of a black background. He wanted that during runtime, the black background will be transparent so the image that he wants to show underneath will be visible. Here is a figure that shows how the movie looks like:

The HTML File

XML

<!DOCTYPE HTML>
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <title>Working with Canvas</title>
    <script type="text/javascript" src="main.js"></script>
    <link href="style.css" rel="stylesheet" />
</head>
<body onload="handlePageLoad()">
    <video id="video" controls="controls" width="997px" height="587px" autoplay>
        <source src="vid.mp4" type="video/mp4" />
    </video>
    <div>
        <canvas id="c1" width="997px" height="587px" />
        <canvas id="c2" width="997px" height="587px" />
    </div>
</body>
</html>

The HTML includes two canvas elements and the video element. I use two canvas elements because one will hold the buffer that I’m working on (and make the black background transparent) and the other will show the movie after the adjustments are finished. I also use the following style.css file:

CSS

#c1
{
    background-image<span class="code-none">: url(Bg.png)<span class="code-none">;
    background-repeat<span class="code-none">: no-repeat<span class="code-none">;
<span class="code-none">}
#c2
<span class="code-none">{
    display<span class="code-none">: none<span class="code-none">;
<span class="code-none">}
#video
<span class="code-none">{
    display<span class="code-none">: none<span class="code-none">;
<span class="code-none">}</span></span></span></span></span></span></span></span></span></span></span></span></span>

In the CSS, I’m making sure not to display the buffer canvas and the video element and to show the background image in the first canvas element.

The Real Thing – The JavaScript Code

Here is the implementation of the JavaScript file that will help you do the job:

JavaScript

var video;
var ctx1;
var ctx2;
var width;
var height;

function timerCallback() {
    if (video.paused || video.ended) {
        return;
    }
    generateFrame();
    setTimeout(function () {
        timerCallback();
    }, 0);
}

function handlePageLoad() {
    video = document.getElementById("video");
    var c1 = document.getElementById("c1");
    ctx1 = this.c1.getContext("2d");
    var c2 = document.getElementById("c2");
    ctx2 = this.c2.getContext("2d");
    video.addEventListener("play", function () {
        width = this.videoWidth;
        height = this.videoHeight;
        timerCallback();
    }, false);
}

function generateFrame() {
    ctx2.drawImage(video, 0, 0, width, height);
    var frame = ctx2.getImageData(0, 0, width, height);
    var len = frame.data.length / 4;

    for (var i = 0; i < len; i++) {
        var r = frame.data[i * 4 + 0];
        var g = frame.data[i * 4 + 1];
        var b = frame.data[i * 4 + 2];
        if (g < 10 && r < 10 && b < 10) {
            frame.data[i * 4 + 3] = 0;
        }
    }
    ctx1.putImageData(frame, 0, 0);
    return;
}

Let's deep dive into the things that the functions are doing:

The handlePageLoad function is responsible to initialize all the canvas contexts and to add a listener to the play event of the video. When it occurs, it will start the timerCallback function.
The timerCallback function is responsible to stop the generation of frames when the video stops and to generate the current frame using the generateFrame function.
The generateFrame function is the function that does the heavy lifting of drawing the transparent video on the canvas elements. It draws the video frame to the buffer canvas and then extracts that frame. After extracting the frame, it passes on every pixel and checks whether it is black. If so, it sets its opacity to transparent.

The effect will look like:

TheFullExample

Summary

In the post, I showed you how to use a canvas in order to make changes to a running video on the fly. This is only an experiment that I made to show that it is possible to do that. The performance of this implementation isn’t so good and I would have thought about other solutions such as a video that includes the background image and the movie.

This article was originally posted at http://feeds.feedburner.com/GilFinkBlog

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Written By

Gil Fink

Technical Lead sparXys

Israel

Gil Fink is a web development expert and ASP.Net/IIS Microsoft MVP. He is the founder and owner of sparXys. He is currently consulting for various enterprises and companies, where he helps to develop Web and RIA-based solutions. He conducts lectures and workshops for individuals and enterprises who want to specialize in infrastructure and web development. He is also co-author of several Microsoft Official Courses (MOCs) and training kits, co-author of "Pro Single Page Application Development" book (Apress) and the founder of Front-End.IL Meetup. You can read his publications at his website: http://www.gilfink.net

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Video Manipulation with the Canvas Element

Video Manipulation with the Canvas Element

The Demands

The HTML File

The Real Thing – The JavaScript Code

Summary

License

Comments and Discussions