Open Source Video Production

I'm always looking for new ways to educate my audience and have recently started setting up my workstation for recording video to post on Youtube. While I could have gone down the easy road and used my Mac laptop for all of this, I wanted to see what it would take to make it all work on my Linux workstation.

There are lots of application choices for capturing and recording video on Linux. Unfortunately, most of the user interfaces I tried were buggy or totally inflexible. Being a long-time Unix user, I started looking around for more powerful tools on the command line. I was not disappointed.

There's VLC and cvlc for command-line work. In fact, VLC is probably the easiest choice for any kind of media and generally works out of the box. I made a couple of test recordings with VLC and it works fine. Since figuring out what to do with VLC took less than five minutes, I figured I'd look for something more challenging and, hopefully, a little more flexible.

Gstreamer is a framework and libraries for working with media on lots of platforms, but it really shines on Linux. It's meant to be used as a library, but it also comes with a useful pair of CLI tools in the form of gst-inspect and gst-launch. Working with these is really confusing at first since the intent behind the syntax of available examples is not parseable without in-depth knowledge of how gstreamer works. Initial experiments showed that I could do what I want and more. For example, here's how to record your webcam to a file:

	#!/bin/bash
	set -e -x
	gst-launch-1.0 \
	v4l2src device=/dev/video0 ! \
	image/jpeg, width=1280, height=720 ! \
	jpegparse ! \
	jpegdec ! \
	autovideoconvert ! \
	autocluttersink

view raw showcam.sh hosted with ❤ by GitHub

I had this idea of recording an overhead camera simultaneously with a forward camera, so I decided to invest some time into learning gstreamer well enough to use it. I was having a hard time figuring out how to do complex flows in gst-launch because it requires an understanding of how the gstreamer API works. I decided to try the Python API to see if it was any easier. It's not Python's fault but that way lies madness. There are currently two major versions of the gstreamer API available, 0.10 and 1.0. This creates problems for gst-launch and python alike. The vast majority of examples are for 0.10. Packages are available for both. I couldn't find a usable combination so I gave up and Googled "golang gstreamer".

Even though the Go bindings for gstreamer are minimal, I did manage to get them working with gstreamer 1.2 and got an example program going.

	package main
	import (
	"fmt"
	"github.com/tobert/glib"
	"github.com/tobert/gst"
	)
	func main() {
	fmt.Printf("Version %s\n", gst.VersionString())
	vsrc := gst.ElementFactoryMake("v4l2src", "Logitech_C920_video")
	vqueue := gst.ElementFactoryMake("queue", "video_queue")
	asrc := gst.ElementFactoryMake("pulsesrc", "pulsesrc")
	sink := gst.ElementFactoryMake("autocluttersink", "display")
	pl := gst.NewPipeline("slicker")
	pl.Add(vsrc, vqueue, asrc, sink)
	vsrc.Link(vqueue, sink)
	pl.SetState(gst.STATE_PLAYING)
	glib.NewMainLoop(nil).Run()
	}
	// vim: ts=4 sw=4 noet tw=120 softtabstop=4

view raw main.go hosted with ❤ by GitHub

My modified bindings for Go/gstreamer are available on Github at https://github.com/tobert/gst.

While I love programming in Go, this was another dead end. The bindings don't have all of the API endpoints I need and, more importantly, 'go build' with cgo involved was really slowing me down so I decided to try gst-launch again using what I've learned.

	#!/bin/bash
	set -e
	timestamp=$(date -Im)
	queue_bytes=$((2**24)) # 16mb queues between threads
	camfile0="$HOME/Video/gst-video0-${timestamp}.mov"
	camfile1="$HOME/Video/gst-video1-${timestamp}.mov"
	camera_resolution="width=1280, height=720"
	camera0="/dev/video0" # Logitech C920
	camera1="/dev/video1" # Microsoft Lifecam
	set -x
	gst-launch-1.0 --eos-on-shutdown -vvv \
	v4l2src device=$camera0 do-timestamp=true ! \
	image/jpeg, $camera_resolution ! \
	queue max-size-bytes=$queue_bytes ! \
	avmux_mov ! \
	filesink location=$camfile0 \
	v4l2src device=$camera1 do-timestamp=true ! \
	image/jpeg, $camera_resolution ! \
	queue max-size-bytes=$queue_bytes ! \
	avmux_mov ! \
	filesink location=$camfile1

view raw 2cameras-2files.sh hosted with ❤ by GitHub

That works, but what I really want is to do a whole lot more. How about capturing both webcams, audio, and a terminal window all in one go? Yup, it can do that.

	#!/bin/bash
	# pulsesrc ⇨ audioconvert ⇨ lamemp3enc
	# ⇩
	# queue
	# ⇩
	# queue ⇨ x264enc ⇨ queue ⇨ mpegtsmux ⇨ filesink
	# ⇧
	# ⇧ queue ⇦ jpegdec ⇦ jpegparse ⇦ v4l2src
	# ⇧ ⇙
	# videomixer ⇦ queue ⇦ jpegdec ⇦ jpegparse ⇦ v4l2src
	# ⇖
	# queue ⇦ videoconvert ⇦ Terminal
	set -e
	queue_bytes=$((2**24)) # 16mb queues between threads
	timestamp=$(date -Im)
	# write the final video to this file
	outfile="$HOME/Video/gst-video0-${timestamp}.mpts"
	# both cameras will capture in MJPG mode even though the C920 has support for H.264/1080p
	# since it looks like either h264 is lagging or USB is. In any case running the same mode
	# on both matches things up nicely
	camera_resolution="width=1280, height=720"
	camera0="/dev/video0" # Logitech C920
	camera1="/dev/video1" # Microsoft Lifecam
	# pactl list short
	microphone="alsa_input.usb-BLUE_MICROPHONE_Blue_Snowball_201202-00-Snowball.analog-stereo"
	# use xwininfo to get the window ID and update this
	# I manually size the terminal window to 2560x1440 before running this script.
	term_xid=0x4e0001d
	set -x
	gst-launch-1.0 --eos-on-shutdown -vvv \
	mpegtsmux name=mux ! \
	filesink location=$outfile \
	videomixer name=mix sink_1::xpos=1280 sink_2::ypos=720 ! \
	video/x-raw, width=2560, height=1440 ! \
	queue max-size-bytes=$queue_bytes ! \
	x264enc ! \
	queue max-size-bytes=$queue_bytes ! \
	mux. \
	v4l2src device=$camera0 do-timestamp=true ! \
	image/jpeg, $camera_resolution ! \
	jpegparse ! jpegdec ! \
	queue max-size-bytes=$queue_bytes ! \
	mix. \
	v4l2src device=$camera1 do-timestamp=true ! \
	image/jpeg, $camera_resolution ! \
	jpegparse ! jpegdec ! \
	queue max-size-bytes=$queue_bytes ! \
	mix. \
	ximagesrc xid=$term_xid ! \
	timeoverlay ! \
	videoconvert ! \
	queue max-size-bytes=$queue_bytes ! \
	mix. \
	pulsesrc device=$microphone name=microphone ! \
	audioconvert ! \
	lamemp3enc target=bitrate bitrate=64 cbr=true ! \
	queue max-size-bytes=$queue_bytes ! \
	mux.

view raw record.sh hosted with ❤ by GitHub

And finally, here's the result. I'll switch to a larger font in the terminal in the future, but you get the idea.