FFmpeg is a powerful utility for converting multimedia files but its real power is in the filters functionality. Learn about FFmpeg filters and save on apps or services that provide only a selection of the same functionality.
You may have come across the FFmpeg utility before. It is that open-source command-line utility that can perform simple multimedia format conversions such as these:
ffmpeg -i video.avi video.mp4
ffmpeg -i audio.wav audio.mp3
FFmpeg is smart enough to pre-empt your intentions. For example, if you want to extract the audio from a video file, you do not have to strain yourself with details about codecs and stuff. FFmpeg will automatically choose the best settings:
ffmpeg -i video.mp4 audio.mp3
A lot more FFmpeg functionality is in the form of filters. The syntax for FFmpeg filters is obscure and the documentation is a bit cryptic. So, most people avoid them. Not anymore 'cos I'll learn ya.
For several years, I did not bother with FFmpeg filters. The only filter I used was the volume filter, which I used (blindly) to amplify low-volume audio files. I had even written a FFmpeg tips-and-tricks article in a magazine without knowing any better. Last year, in the midst of the lockdown, I decided to write a follow-up for the article — sort of 'More FFmpeg tips and tricks'. When I went in depth about FFmpeg, I decided to write a book instead. It is now a 152-page full-colour paperback on Amazon - FFMPEG Quick Hacks. In the spirit of open source, I have made its ebook version free. The book has a full tutorial and is not just a collection of hacks. In fact, the tutorial forms much of this book. The hacks part is just one big chapter. I did not realize my mistake until it was too late.
For the book and this article, I used Linux. If you are on Windows, you need to replace the slashes (\) in multiline commands with carets.
FFmpeg Filter Syntax
FFmpeg has lots of filters - more than you can imagine. They are listed and described in detail at this link. Each one of these filters represents a processing phase that can be used individually or daisy-chained with other such filters. One or more multimedia streams (audio, video, subtitles) can be provided as input for the filter. Depending on the kind of processing done by the filter, it might have one or more output multimedia streams. The output from one filter can be used as the input for a subsequent filter.
Here is a simple example of a filter. It increases the volume of the audio stream by a third.
ffmpeg -i low.mp3 -af "volume=1.33" high.mp3
When I wanted to slightly rotate a video, I used this.
ffmpeg -y -i barbara.mp4 \
-vf rotate=angle=-20*PI/180:fillcolor=brown \
The part in bold type in the above command is the filter definition. Most filter definitions in production may seem undecipherable and confusing but it is not rocket science. FFmpeg filters use a standard syntax:
[input label1][input label2]...[input_labelN]filter=key=value pair1:key=value
pair2...key=value pairN[output label1][output label2]...[output labelN]
Use the command switch
-filter_complex before defining your filter. It handles all types of filters and eliminates confusing errors.
Multiple filters need to be separated by a semicolon (;). At the end of the string of filters though, there should be no semicolon. A filter may have one or more inputs and/or outputs. (Some special ones have none.) The inputs can be streams in the input files or any ad-hoc output label you specify in the previous filters. The output labels can be used as input for subsequent filters or for map assignments outside the filter definition. If the input(s) of a filter is/are same as the output(s) of the immediately previous filter, then a comma can be used instead of a semi-colon. Let me explain most of this with an example.
ffmpeg -i Tasmania-Nobrainia.mp4 \
-map '[amped]' \
In this command, two filters are used —
volume. The first filter is for extracting a particular section of its input audio stream. The second is for changing the volume level of its input audio stream. Before
atrim, the input audio stream has been specified as
[0:a:0]. This refers to the first audio stream in the first input file. The
end key-value pairs specify the beginning and ending timestamps between which the extraction has to occur — the second minute. The extracted audio is then packaged as the output stream labelled as
'[trimmed]'. This label is specified as the input for the next filter
volume filter has a key-value pair, which is also named
volume. This filter causes the volume of the input audio stream (labelled
'[trimmed]') to be increased by two-thirds. The filter's output is labelled
'[amped]'. The filter definition ends thereafter. [Note: There is no semicolon at the end of the filter definition.]
It is important that a filter only refers to existing input file streams or previously created output labels. Otherwise, FFmpeg will start making guesses. If it does not succeed, there will be errors. In the above command, the input audio stream specified as
'[0:a:0]' is not really necessary. FFmpeg would have automatically chosen it. Given that the output format is an audio file, the last filter need not have labelled its output for the
-map switch. FFmpeg would have automatically chosen it from the outputs of the last filter. [Note: DO NOT leave any outputs unused. The last outputs should match the streams required by the output file container (format).]
ffmpeg -i Tasmania-Nobrainia.mp4 \
Going further, even the keys are not necessary if the values are in the order specified in the filter documentation.
ffmpeg -i Tasmania-Nobrainia.mp4 \
Most online examples of FFmpeg filters are like this. Without the filter documentation, filter definitions may look cryptic.
More FFmpeg Filter Examples
In some of his movies, Jim Carrey does a live rewind of a scene. Does he sound intelligible if you rewind that footage? Let us use FFmpeg filters and see.
ffmpeg -y -i ace-ventura-reverse.mp4 \
"[0:v:0]reverse[v]; [0:a:0]areverse[a]" \
-map '[v]' -map '[a]' \
#Place them side-by-side
ffmpeg -y -i ace-ventura-reverse.mp4 -i ace-ventura-reverse-reversed.mp4 \
'Original audio on left speaker':fontsize=30:fontfile=Florentia.ttf[lefttext];
'Reversed audio on right speaker':fontsize=30:fontfile=Florentia.ttf[v];
-map '[v]' -map '[a]' ace-ventura-reversal-truth.mp4
The filters in the first FFmpeg command reverses the video and audio streams. [Note: These filters consume a lot of memory. Use short clips.] The filters in the second
FFmpeg places the two videos side-by-side to see if they match up. Specifically, the audio stream from the original video is played on the left speaker while the audio from the reversed video is played on the second speaker. You may have to slide the audio balance of your volume control utility to either side to listen to only one of the audio streams.
Some filters do not need any input. They may use certain built-in filter variables to generate audio or video output. In this command, the
testsrc filter generates a video test pattern out of nothing. The
aevalsrc generates a random white noise using an algebraic equation. [Note: The
-f lavfi switch allows you to use the filter output as a pseudo input file.]
ffmpeg -f lavfi \
-i "testsrc=size=320x260[out0];aevalsrc=random(0)/20[out1]" \
-t 0:0:30 -pix_fmt yuv420p \
In the next example, the built-in filter variables
ih are used to resize a series of numbered images and create a slideshow video — the variables representing the actual dimensions of the current image being processed by the filter. [Note: A comma instead of a semicolon has been used to indicate that all outputs of the filter on the left are consumed by the filter on the right.]
ffmpeg -y -framerate 1/3 -i image%02d.jpg \
-pix_fmt yuv420p -framerate 24 \
You will not crack FFmpeg filters on your first attempt. I certainly did not. After I understood how to use them though, a whole new world opened up to me. There is no need to use expensive software to create various effects. FFmpeg filters can do most of them for free.
If you did not understand something in this article, then read my free ebook. It has hundreds of examples and illustrations. Everything is explained in much more detail. After that, you can use the FFmpeg documentation and go nuts with it.
- 21st June, 2021: Initial version
- 22st June, 2021: Update about the use of comma, instead of a semicolon