The actual video acquisition can be done with a few simple command line statements. Alternatively, a fancier Python script can be used to help trigger and automate the video acquisition.

The video format used for the experiment was 1920x1080 at 30fps. This matches the settings used in the source paper.

Make sure you have enabled camera hardware in raspi-config.

Simple Acquisition

Video can be acquired very simply via the command line with use of the raspivid command. Simply SSH into the Pi and run the following. Replace the ### in the filename with a suitable run number to keep track of your video files.

Download: file
raspivid -t 10000 -w 1920 -h 1080 -fps 30 -o run_###.h264
The -t parameter sets the video length in millseconds.

This will leave you with a "raw" H264 video stream. To allow for playback in media player software, an additional step is required to add the suitable "wrapper" data. We did that using the recommended process:

Download: file
MP4Box -add run_###.h264 run_###.mp4

The MP4Box command can be installed with:

Download: file
sudo apt install -y gpac

With this approach, the action happens as soon as you press the <Enter> key. You may find coordinating this with the test subject's readiness tricky. If so, see the next section for a way to help automate and coordinate things better.

Python Acquisition

This is a fancier approach that requires more setup. However, it can really help with synchronizing the video capture with the test subject's speaking.

To provide an audio cue, the following USB sound card was used:

USB Audio Adapter - Works with Raspberry Pi

PRODUCT ID: 1475
The Raspberry Pi has an on-board audio jack, which is super handy for all kinds of sound effects and speech, just plug and go! However, for when you want better audio for music...
OUT OF STOCK

USB OTG Host Cable - MicroB OTG male to A female

PRODUCT ID: 1099
This cable looks like a USB micro cable but it isn't! Instead of a USB A Plug, it has a USB A Socket on the end. This cable is designed for use with OTG (On the Go) host devices...
$2.50
IN STOCK

Alternatively, you could use one of these (for headphone output):

Adafruit I2S Audio Bonnet for Raspberry Pi

PRODUCT ID: 4037
Add some easy-listenin' tunes to your Raspberry Pi using this basic audio bonnet. It'll give you stereo line out from a digital I2S converter for a good price, and sounds nice...
$9.95
IN STOCK

Or one of these (for amplified output):

Adafruit I2S 3W Stereo Speaker Bonnet for Raspberry Pi

PRODUCT ID: 3346
Hey Mr. DJ! Turn up that Raspberry Pi mix to the max with this cute 3W Stereo Amplifier Bonnet for Raspberry Pi. (It's not big enough to be an...
OUT OF STOCK

To trigger video capture, a basic normally-open button was wired to the Pi's GPIO header. We used one from a junk drawer, but really, any normally open button will do:

Our setup connected the button between GPIO4 and GND, but any available GPIO pin will work.

Here is the source code for a Python script you can use to automate testing.

import time
import math
import os
import RPi.GPIO as GPIO
import simpleaudio as sa
import picamera

camera = picamera.PiCamera()
camera.resolution = (1920, 1080)
VIDEO_LENGTH = 10

BUTTON = 4
GPIO.setmode(GPIO.BCM)
GPIO.setup(BUTTON, GPIO.IN, pull_up_down=GPIO.PUD_UP)

SIN_LENGTH = 500
SIN_AMPLITUDE = 127
SIN_OFFSET = 128
DELTA_PI = 2 * math.pi / SIN_LENGTH
sine_wave = bytes([
     int(SIN_OFFSET + SIN_AMPLITUDE * math.sin(DELTA_PI * i)) for i in range(SIN_LENGTH)
])

def play_tone(length):
    play_back = sa.play_buffer(sine_wave*length, 2, 2, 44100)
    play_back.wait_done()

run_number = int(input("Enter run number:"))

print("Press button when ready.")
while GPIO.input(BUTTON):
    pass

play_tone(100)
camera.start_recording("run_{:03d}.h264".format(run_number))
camera.wait_recording(VIDEO_LENGTH)
camera.stop_recording()
play_tone(100)

err = os.system("MP4Box -add run_{0:03d}.h264 run_{0:03d}.mp4".format(run_number))
This guide was first published on Oct 14, 2020. It was last updated on 2020-10-14 11:57:31 -0400.
This page (Acquiring Data) was last updated on Oct 24, 2020.