You'll need some software for this:
- Rhubarb Lip Sync (version 1.13.0 was used while creating this guide)
- The modified g722.1 encoder, available from pypi and github for Windows, Linux and Mac with
pip3 install g722-1-mod - The Python scripts for creating a new .bin file, currently at https://github.com/adafruit/snxrom
- A program for re-sampling audio files. If you don't already have a program for doing this, one open source option is Audacity. There's a dedicated guide for sample rate conversion in Audacity, but for this project you will be converting to 16kHz or 32kHz.
Grab the file Intro.bin from your Teddy Ruxpin's original files and save it as orig_Intro.bin. It is used as the basis for the modified (patched) story file.
Record your new story in the audio format of your choice. Then, create a downsampled version in wav format at 16kHz or 32kHz sample rate and mono (one channel) audio. If you don't already have a program for doing this, one open source option is Audacity. There's a dedicated guide for sample rate conversion in Audacity, just remember that the supported sample rates for this project are 16kHz (16000 Hz) or 32kHz (32000 Hz)
The text below will refer to the original audio file as story.wav and the downsampled version as story32k.wav.
Create the mouth position json file with rhubarb using a commandline like:
rhubarb -f json -o mouth.json story.wav
Now, you have all the files needed to run earpatch from snxrom.git.
python3 earpatch.py --wav story32k.wav --rhubarb-json mouth.json orig_Intro.bin new_Intro.bin
Copy new_Intro.bin back to the Books folder as Intro.bin (the name must match exactly), eject/safely remove the USB device, then turn Teddy's power off and back on. Teddy will begin with your story, the mouth movements detected by Rhubarb, and eye animations occurring on average once every 30 seconds.
{
"metadata": {
"soundFile": "story.wav",
"duration": 23.64
},
"mouthCues": [
{ "start": 0.00, "end": 0.34, "value": "A" },
{ "start": 0.34, "end": 1.12, "value": "B" },
{ "start": 1.12, "end": 1.68, "value": "D" },
{ "start": 1.68, "end": 1.82, "value": "C" },
{ "start": 1.82, "end": 2.17, "value": "B" },
Within the mouthCues section, you can add or delete lines, adjust start or end times, or change the mouth shape, called value.
While doing so, you need to make sure that:
- json syntax is respected
- the
starttimes are in ascending order - each
endtime is no later than the next "start" time - the
valueis one of the following 9 uppercase letters: A, B, C, D, E, F, G, H, X
The letters A, F, and X correspond to a closed mouth; B, C, G, and H to a partially open mouth, and D and E to a fully open mouth.
Text editor powered by tinymce.
