Use Processing and FFmpeg to export HD video


I’ve been trying to find a dependable method for exporting HD video from Processing sketches that contain dynamic data and movement as well as complex 3-dimensional shapes rendered using OpenGL. QuickTime’s screen recording option hogs memory and causes frames to skip, as does does ScreenFlow, my usual go-to for screen recording. The best way I have found so far is to export actual resolution PNG files using “save()” and create an HD video using FFmpeg.

FFmpeg is a very fast video and audio converter that can also grab from a live audio/video sources or create and compress video from multiple still images. It is used from the command line and gives one control of bitrates, codecs, formats, resolution, quality, metadata, and many more options for working with video.

To install FFmpeg with Macports
sudo port install ffmpeg

# view help
ffmpeg -h

# list encoders
ffmpeg -encoders

# list codecs
ffmpeg -codecs

# use format
ffmpeg [global_options] {[input_options] -i ‘input_file’} ... {[output_options] ‘output_file’} ...

# make a movie from matching .png files, write over original, codec = libx264 (H.264), framerate = 30, 1 pass, resolution = 1920×1080, video bitrate = 6Mbits, format = mp4
ffmpeg -y -pattern_type glob -i 'p_*.png' -vcodec libx264 -r 30 -q 100 -pass 1 -s 1920x1080 -vb 6M -threads 0 -f mp4 file.mp4

# convert the video file to Apple ProRes codec for use in Final Cut
ffmpeg -y -i file.mp4 -vcodec prores -vb 6M -r 30 -s 1920x1080 -f mov

# here’s another example conversion command. this one has a much higher bitrate
ffmpeg -y -i file_01.mp4 -b:v 40M -vcodec libx264 -pass 1 file_02.mp4

For example, using this Processing sketch, you generate the PNG files, and then run the code above on the command line to make this file:

Also see this video for a more detailed example


7 thoughts on “Use Processing and FFmpeg to export HD video

  1. Nice to see someone else using ffmpeg the same way! I’m curious about a few of your settings:

    1. How did you arrive at the video bitrate of 6Mbits?
    2. Since you’re including them as options (but me not sure what they do), I’m guessing that you intentionally chose ‘threads’ to be 0 and ‘pass’ to be 1 – any particular reason?

    Also, nice filename pattern! Much more flexible than mine which is very finicky about numbering, etc.

  2. Likewise! It was very interesting to discover your sketch after figuring out it could be done. I was actually searching/hoping to stream the images straight from Processing but this method is dependable.

    1. I kept pushing it higher and higher until I the difference between the quality of the original PNG files and the video was acceptable. Many users on other forums pushed FFmpeg higher than 10Mbits.

    2. “Threads tells FFmpeg to guess how many threads to use depending on number of cores” ref, and “pass 2” failed so I just left it at one pass.

    Thanks! There is probably something cleaner in Java for leading zeros but I was in a hurry :-)

    PS. Enjoyed the work on your site as well as your curatorial practice. Wikipedia Loops is very nice. I am looking forward to reading the Leonardo piece. Is it the “All About Data” issue?

  3. Hmm, I’ll have to play with the settings some more. I’ve been really happy using the PNG codec, for what it’s worth – since I write my frames as PNG there is little-to-no loss, though the video files are really big.

    Also: thanks! The Wiki Loops piece will be the data issue – final editing as we speak!

  4. Yeah x264 makes small-ish files even at that high bit rate.

    Cool, we’ll be in the same issue.

  5. Hey, thanks for the snippets!
    Unfortunately I can’t get a 30fps movie (I input 6000 frames and get a 4 minutes file).
    How to fix it?

    k:Words chiara$ ffmpeg -y -pattern_type glob -i ‘*.png’ -vcodec libx264 -r 30 -q 100 -pass 1 -s 1920×1080 -vb 6M -threads 0 -f mp4 file.mp4
    ffmpeg version 1.2.1 Copyright (c) 2000-2013 the FFmpeg developers
    built on Jun 12 2013 13:46:16 with Apple clang version 4.1 (tags/Apple/clang-421.11.66) (based on LLVM 3.1svn)
    configuration: –prefix=/opt/local –enable-swscale –enable-avfilter –enable-libmp3lame –enable-libvorbis –enable-libopus –enable-libtheora –enable-libschroedinger –enable-libopenjpeg –enable-libmodplug –enable-libvpx –enable-libspeex –enable-libass –enable-libbluray –enable-gnutls –enable-libfreetype –mandir=/opt/local/share/man –enable-shared –enable-pthreads –cc=/usr/bin/clang –arch=x86_64 –enable-yasm –enable-gpl –enable-postproc –enable-libx264 –enable-libxvid
    libavutil 52. 18.100 / 52. 18.100
    libavcodec 54. 92.100 / 54. 92.100
    libavformat 54. 63.104 / 54. 63.104
    libavdevice 54. 3.103 / 54. 3.103
    libavfilter 3. 42.103 / 3. 42.103
    libswscale 2. 2.100 / 2. 2.100
    libswresample 0. 17.102 / 0. 17.102
    libpostproc 52. 2.100 / 52. 2.100
    [image2 @ 0x7f809a83d800] Stream #0: not enough frames to estimate rate; consider increasing probesize
    Input #0, image2, from ‘*.png’:
    Duration: 00:04:00.04, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: png, rgba, 1920×1080, 25 tbr, 25 tbn, 25 tbc
    [libx264 @ 0x7f809a841800] using cpu capabilities: none!
    [libx264 @ 0x7f809a841800] profile High 4:4:4 Predictive, level 4.0, 4:4:4 8-bit
    [libx264 @ 0x7f809a841800] 264 – core 129 – H.264/MPEG-4 AVC codec – Copyleft 2003-2013 – – options: cabac=1 ref=1 deblock=1:0:0 analyse=0x1:0 me=dia subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=6 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=6000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
    Output #0, mp4, to ‘file.mp4’:
    encoder : Lavf54.63.104
    Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv444p, 1920×1080, q=-1–1, pass 1, 6000 kb/s, 15360 tbn, 30 tbc
    Stream mapping:
    Stream #0:0 -> #0:0 (png -> libx264)
    Press [q] to stop, [?] for help
    frame= 7201 fps= 15 q=32766.0 Lsize= 82334kB time=00:03:59.96 bitrate=2810.7kbits/s dup=1200 drop=0
    video:82263kB audio:0kB subtitle:0 global headers:0kB muxing overhead 0.086623%
    [libx264 @ 0x7f809a841800] frame I:82 Avg QP:24.02 size: 83566
    [libx264 @ 0x7f809a841800] frame P:3741 Avg QP:22.32 size: 19945
    [libx264 @ 0x7f809a841800] frame B:3378 Avg QP:14.30 size: 820
    [libx264 @ 0x7f809a841800] consecutive B-frames: 35.2% 5.9% 2.4% 56.4%
    [libx264 @ 0x7f809a841800] mb I I16..4: 65.2% 0.0% 34.8%
    [libx264 @ 0x7f809a841800] mb P I16..4: 14.3% 0.0% 0.0% P16..4: 11.7% 0.0% 0.0% 0.0% 0.0% skip:74.0%
    [libx264 @ 0x7f809a841800] mb B I16..4: 0.4% 0.0% 0.0% B16..8: 1.0% 0.0% 0.0% direct: 0.5% skip:98.2% L0:39.6% L1:53.1% BI: 7.3%
    [libx264 @ 0x7f809a841800] final ratefactor: 20.01
    [libx264 @ 0x7f809a841800] coded y,u,v intra: 33.8% 7.5% 2.2% inter: 4.1% 0.5% 0.2%
    [libx264 @ 0x7f809a841800] i16 v,h,dc,p: 34% 42% 20% 3%
    [libx264 @ 0x7f809a841800] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 47% 27% 12% 1% 4% 4% 1% 1% 2%
    [libx264 @ 0x7f809a841800] Weighted P-Frames: Y:6.7% UV:1.8%
    [libx264 @ 0x7f809a841800] kb/s:2807.50

Comments are closed.