The actual video acquisition can be done with a few simple command line statements. Alternatively, a fancier Python script can be used to help trigger and automate the video acquisition.
The video format used for the experiment was 1920x1080 at 30fps. This matches the settings used in the source paper.
Simple Acquisition
Video can be acquired very simply via the command line with use of the raspivid command. Simply SSH into the Pi and run the following. Replace the ### in the filename with a suitable run number to keep track of your video files.
raspivid -t 10000 -w 1920 -h 1080 -fps 30 -o run_###.h264
This will leave you with a "raw" H264 video stream. To allow for playback in media player software, an additional step is required to add the suitable "wrapper" data. We did that using the recommended process:
MP4Box -add run_###.h264 run_###.mp4
The MP4Box command can be installed with:
sudo apt install -y gpac
With this approach, the action happens as soon as you press the <Enter> key. You may find coordinating this with the test subject's readiness tricky. If so, see the next section for a way to help automate and coordinate things better.
Python Acquisition
This is a fancier approach that requires more setup. However, it can really help with synchronizing the video capture with the test subject's speaking.
To provide an audio cue, the following USB sound card was used:
Alternatively, you could use one of these (for headphone output):
Or one of these (for amplified output):
To trigger video capture, a basic normally-open button was wired to the Pi's GPIO header. We used one from a junk drawer, but really, any normally open button will do:
Our setup connected the button between GPIO4 and GND, but any available GPIO pin will work.
Here is the source code for a Python script you can use to automate testing.
# SPDX-FileCopyrightText: 2020 Carter Nelson for Adafruit Industries
#
# SPDX-License-Identifier: MIT
import time
import math
import os
import RPi.GPIO as GPIO
import simpleaudio as sa
import picamera
camera = picamera.PiCamera()
camera.resolution = (1920, 1080)
VIDEO_LENGTH = 10
BUTTON = 4
GPIO.setmode(GPIO.BCM)
GPIO.setup(BUTTON, GPIO.IN, pull_up_down=GPIO.PUD_UP)
SIN_LENGTH = 500
SIN_AMPLITUDE = 127
SIN_OFFSET = 128
DELTA_PI = 2 * math.pi / SIN_LENGTH
sine_wave = bytes([
int(SIN_OFFSET + SIN_AMPLITUDE * math.sin(DELTA_PI * i)) for i in range(SIN_LENGTH)
])
def play_tone(length):
play_back = sa.play_buffer(sine_wave*length, 2, 2, 44100)
play_back.wait_done()
run_number = int(input("Enter run number:"))
print("Press button when ready.")
while GPIO.input(BUTTON):
pass
play_tone(100)
camera.start_recording("run_{:03d}.h264".format(run_number))
camera.wait_recording(VIDEO_LENGTH)
camera.stop_recording()
play_tone(100)
err = os.system("MP4Box -add run_{0:03d}.h264 run_{0:03d}.mp4".format(run_number))
