Click here to Skip to main content
15,860,943 members
Articles / Multimedia / Video

Dynamic Cropping for VirtualDub

Rate me:
Please Sign up or sign in to vote.
4.73/5 (9 votes)
14 Aug 2011GPL36 min read 55.6K   1.6K   20   21
This article describes a project that enables the VirtualDub software with a new dynamic cropping feature.

Introduction

One day, I came across a video that was a dub of an old 8mm film we used to make as kids. The dub was made unprofessionally with no special equipment or even a tripod, so it suffered from a number of problems. The worst was sporadic non-compensated motion of the dubbed screen area. I tried to find software that would allow me to crop the moving areas dynamically but had no luck. There was no free software with such a feature, and all commercial software was way too expensive. Then I came to an idea of writing such software by myself. As a base, I chose VirtualDub because it already provided some static cropping functionality. All I needed to do was to make it dynamic, e.g., with placement of the cropped area defined as a function of time. I decided to make it as flexible as possible, so one could edit, save & restore the dynamic cropping settings, and to design it as an extensible feature.

Design

The "work horse" of the solution is a new IVDDynClippingStorage interface defined in ClippingControl.h:

C++
class VDINTERFACE IVDDynClippingStorage : public vdrefcounted<IVDRefCount> {
public:
	// Initialize clipping map
	virtual void Init(int sourceW, int sourceH) = 0;
	// Get source width
	virtual int GetWidth() = 0;
	// Get source height
	virtual int GetHeight() = 0;
	// Load clipping map from a file
	virtual VDStringW Load(wchar_t *filename, HWND parent) = 0;
	// Save clipping map to a file
	virtual VDStringW Dump(wchar_t *filename, HWND parent) = 0;
	// Display configuration interface
	virtual void Configure(HWND parent) = 0;
	// Set clipping bounds for position
	virtual int SetClipBounds(sint64 pos, const vdrect32& r) = 0;
	// Get clipping bounds for position
	virtual int GetClipBounds(sint64 pos, vdrect32& r) = 0;
	// Get position for clipping map index
	virtual sint64 GetClipPos(int idx) = 0;
	// Get clipping map size
	virtual int GetTotalClipPos() = 0;
	// Get the resulted crop size
	virtual void GetCrop(vdrect32& r) = 0;
	// Delete clipping map entry by index
	virtual void DelClipBounds(int idx) = 0;
	// Crop pixmap for given position
	virtual bool DoCrop(VDPixmap& dst, sint64 pos, vdrect32& crop_area) = 0;
	// Add/drop range to/from clipping map
	virtual void ChangeTimeline(sint64 pos, sint64 nframes, bool add) = 0;
};

It defines a set of methods to manage the clipping map and to perform the actual dynamic cropping. The VDDynClippingStorage class defined in ClippingControl.cpp is implementing the interface. An instance of the class can be created per FilterInstance (however, a need in having more than one filter with dynamic cropping in the chain of filters is highly doubtful).

The following diagram shows relations between the components of the solution:

+--------------+                                                +--------------+
|              |  VDGetDynClippingStorage()                     |              |
|  Script.cpp  +-----------+                                    |    Job.cpp   |
|              |           |                                    |              |
|              |        +--|---------------------------+        |              |
|              |        |  |                           |        |              |
|              |        |  | ClippingControl.cpp       |        |              |
|              |        |  V                           |        |              |
|              | Load() | +--------------------------+ | Dump() |              |
|              +--------->|  IVDDynClippingStorage   |<---------|              |
|              |        | | +----------------------+ | |        |              |
+------------+-+        | | | VDDynClippingStorage | | |        +--+-----------+
             |          | | |                      | | |           |
SetClippingStorage()    | | |                      | | |    GetClippingStorage()
             |          | | |                      | | |           |
             |    +------>| +----------------------+ | |           |
             |    |     | +---^-------^----------^---+ |           |
             |    |     |     |       |          |     |           |
             | DoCrop() | Configure() |          |     |           |
             |    |     |     | GetClipBounds()  |     |           |
             |    |     |     |       | SetClipBounds()|           |
             |    |     |     |       |          |     |           |
             |    |     | +---|-------|----------|---+ |           |
             |    |     | |   |IVDClippingControl|   | |           |
             |    |     | | +-+-------+----------+-+ | |           |
             |    |     | | |  VDClippingControl   | | |           |
             |    |     | | |                      | | |           |
             |    |     | | +----------------------+ | |           |
             |    |     | +----^----------------^----+ |           |
             |    |     +------|----------------|------+           |
             |    |            |                |                  |
             |    | GetClippingStorage()  SetClippingStorage()     |
             |    |            |                |                  |
             |    |     +------+----------------+------+           |
             |    |     |                              |           |
             |    |     |          Filtdlg.cpp         |           |
             |    |     |                              |           |
             |    |     +------+----------------+------+           |
             |    |            |                |                  |
             |    | SetClippingStorage()  GetClippingStorage()     |
             |    |            |                |                  |
          +--V----+------------V----------------V------------------V--+
          |                                                           |
          |                    FilterInstance.cpp                     |
          |                                                           |
          +-----------------------------------------------------------+

There are some changes made to the basic functionality of the cropping control as well. They are not strictly related to dynamic cropping, but are very handy:

  • From now on, you will be able not only to stretch the cropped area by dragging its sides and corners with left mouse button, but also to move the complete cropped area using the right mouse button! Try it and you will see how useful it is for dynamic cropping when you have to go frame-by-frame adjusting the cropped area placement.
  • A new "lock aspect ratio" checkbox allows the cropped area to have the aspect ratio of the original video. It is suggested to always check it with dynamic cropping enabled.

The following VirtualDub components are affected with the patch:

  • Cropping control (ClippingControl.h/.cpp, FiltDlg.cpp, resources)
  • Filter instance class (FilterInstance.h/.cpp)
  • Saving (Job.cpp) and loading (Script.cpp) of the processing settings

The following project files are modified:

  • src\virtualdub\h\clippingcontrol.h
  • src\virtualdub\h\filterinstance.h
  • src\virtualdub\res\resource.h
  • src\virtualdub\res\virtualdub.rc
  • src\virtualdub\source\clippingcontrol.cpp
  • src\virtualdub\source\filtdlg.cpp
  • src\virtualdub\source\filterinstance.cpp
  • src\virtualdub\source\job.cpp
  • src\virtualdub\source\script.cpp

Building

You should patch the source tree of VirtualDub and re-build it.

There are two ways of patching the source tree. You can either:

  • unzip the archive into the root of VirtualDub 1.9.9 source tree overwriting its files, or
  • use the provided dyncropping.patch file that contains diff info produced with GNU diff. Run it with the Win32 version of patch from the root of VirtualDub source tree as follows:
    C:\VirtualDub-1.9.9-src> patch -p1 -i dyncropping.patch

This may work well with versions of VirtualDub other than 1.9.9 too.

After that, run the building process as usual.

Running

Load a video, add any video filter, select "Cropping..." and check the "Dynamic cropping" checkbox in the "Filter input cropping" control. Check also "Lock aspect ratio" for better results.

Cropping.png

Walk the video frame-by-frame inside of the cropping control and adjust the sizes and position of the cropped area accordingly to your needs. All position and size changes will be saved in the map for rendering. If you rewind the video back to the beginning and go again frame-by-frame, you will notice how the cropped area is moving to follow your adjustments! You can click "Options..." and select the desired settings.

CroppingSettings.png

Once done, click "OK" to close "Filter input cropping" and once again "OK" to close "Filters". Run "Preview filtered..." and watch what is happening on the left and right UI panels. Use "Save as AVI..." to save the final rendering.

Here is a result of processing that old video with the software.

Conclusion

In general, this feature can be useful for manual "motion post-compensation" and stabilization in the following cases:

  • Old videos made with camcorders with no hardware motion compensation available
  • Videos made in complex motion situations (in car, boat, on the run)
  • Videos made by kids or impaired people
  • "Noisy", dark, low-contrast videos where automatic algorithms usually fail or produce bad results

Also, it can be handy for some special video effects (digital zoom, stretching deformation, etc.).

It is very basic, but it is doing what it is supposed to do. Here are a couple of more things I wanted to share.

Currently only linear interpolation between vertices of the clipping map is implemented. One can try to implement a cubic spline-based interpolation to see if it improves the rendering results and reduces shaking. Also, one can try to implement a real motion-compensation algorithm inside of
IVDDynClippingStorage::DoCrop that will cache and analyze frames.

There is also a "no interpolation" mode available that can be used for videos that contain sets of scenes in which static cropping is required - this avoids cutting / cropping / gluing them piece-by-piece.

With linear interpolation, you usually achieve the best results with two cropping adjustment passes.

The resampling selection has three settings: original frame size, maximal cropping size and custom. In any case, the result will be resampled to the rounded sizes divisible by 16 for general compatibility with compression algorithms. An attempt to force custom sizes to values non-divisible by 16 or bigger than those of the source will result in an error.

There are two resampling algorithms built into VirtualDub - "bilinear" and "nearest". The "bicubic" is not implemented in the core of VirtualDub but instead as a plugin and therefore is not supported.

It appears to be useful to have a line grid optionally appearing on the cropped area when you slide it with the right mouse button (maybe it can appear when you press and hold the right mouse button and then the left one). It would allow to position the cropped area more precisely. I did not implement it, but left it as an idea for others.

The IVDDynClippingStorage::ChangeTimeline method is defined and implemented in VDDynClippingStorage but it is not being used for anything - it can be a future extension, just in case.

All clipping map vertex indexes are 1-based. That is, what index you see on the interface is what you have in the map.

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)


Written By
Software Developer (Senior)
Canada Canada
I am a Software Engineer with 20+ years of experience in different fields of programming. Presently I live in the Toronto area.

Comments and Discussions

 
BugChanges ratio when it shouldn't be Pin
Member 1460581027-Sep-19 3:27
Member 1460581027-Sep-19 3:27 
GeneralRe: Changes ratio when it shouldn't be Pin
Sergey Strakhov19-Oct-20 9:59
Sergey Strakhov19-Oct-20 9:59 
QuestionFilter Pin
Otofoto18-Jan-18 21:22
Otofoto18-Jan-18 21:22 
AnswerRe: Filter Pin
Sergey Strakhov19-Oct-20 10:01
Sergey Strakhov19-Oct-20 10:01 
GeneralRe: Filter Pin
Member 1601837930-May-23 20:28
Member 1601837930-May-23 20:28 
QuestionVery well described Pin
Harry W223-Jul-14 6:59
Harry W223-Jul-14 6:59 
QuestionProblem Pin
Member 101005769-Jun-13 11:52
Member 101005769-Jun-13 11:52 
AnswerRe: Problem Pin
Sergey Strakhov9-Jun-13 12:55
Sergey Strakhov9-Jun-13 12:55 
GeneralRe: Problem Pin
Member 101005769-Jun-13 12:58
Member 101005769-Jun-13 12:58 
GeneralRe: Problem Pin
Sergey Strakhov9-Jun-13 13:03
Sergey Strakhov9-Jun-13 13:03 
GeneralRe: Problem Pin
Sergey Strakhov9-Jun-13 13:12
Sergey Strakhov9-Jun-13 13:12 
GeneralRe: Problem Pin
Member 101005769-Jun-13 13:52
Member 101005769-Jun-13 13:52 
GeneralRe: Problem Pin
Sergey Strakhov9-Jun-13 14:18
Sergey Strakhov9-Jun-13 14:18 
GeneralRe: Problem Pin
Member 149561265-Oct-20 8:39
Member 149561265-Oct-20 8:39 
QuestionAmazing! Any plugin version (or compiled version)? Pin
DavidGPeters2-Mar-12 3:57
DavidGPeters2-Mar-12 3:57 
AnswerThe compied version is right there! Pin
Sergey Strakhov2-Mar-12 4:10
Sergey Strakhov2-Mar-12 4:10 
GeneralAwsome but buggy :'( Pin
Astyan13-Jun-11 7:13
Astyan13-Jun-11 7:13 
GeneralRe: Awsome but buggy :'( [modified] Pin
Sergey Strakhov13-Jun-11 7:50
Sergey Strakhov13-Jun-11 7:50 
GeneralThe major issue is fixed now Pin
Sergey Strakhov14-Aug-11 5:36
Sergey Strakhov14-Aug-11 5:36 
GeneralMy vote of 5 Pin
A.Oliva28-Jul-10 13:05
A.Oliva28-Jul-10 13:05 
GeneralMy vote of 5 Pin
Sebastian Br.25-Jul-10 20:35
Sebastian Br.25-Jul-10 20:35 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.