Install requirements

There are a few Python packages that TensorFlow requires that need to be installed:

sudo apt install -y python3-numpy python3-pillow python3-pygame

sudo apt install -y python3-numpy python3-pillow python3-pygame

Install the Speech Output package

Copy Text

sudo apt install -y festival

sudo apt install -y festival

Install rpi-vision

Now to install an Adafruit fork of a program originally written by Leigh Johnson that uses the MobileNet V2 model to detect objects. While there is a MobileNet V3, it isn't as accurate as V2. This part will take a few minutes to complete.

Copy Text

cd ~
source env/bin/activate
git clone --depth 1 https://github.com/adafruit/rpi-vision.git
cd rpi-vision
pip3 install -e .

cd ~
source env/bin/activate
git clone --depth 1 https://github.com/adafruit/rpi-vision.git
cd rpi-vision
pip3 install -e .

Install TensorFlow 2.x

Next you can download and install Tensorflow 2. There are a nice set of releases over on PINTO0309's GitHub Releases Page. To install TensorFlow 2.15.0, run the following commands. This cryptic bit of shell code figures out the current version of CPython running on your Pi, substitutes that in, and then downloads and installs the appropriate release file.

Copy Text

RELEASE=https://github.com/PINTO0309/Tensorflow-bin/releases/download/v2.15.0.post1/tensorflow-2.15.0.post1-cp311-none-linux_aarch64.whl
CPVER=$(python --version | grep -Eo '3\.[0-9]{1,2}' | tr -d '.')
pip install $(echo "$RELEASE" | sed -e "s/cp[0-9]\{3\}/CP$CPVER/g")

RELEASE=https://github.com/PINTO0309/Tensorflow-bin/releases/download/v2.15.0.post1/tensorflow-2.15.0.post1-cp311-none-linux_aarch64.whl
CPVER=$(python --version | grep -Eo '3\.[0-9]{1,2}' | tr -d '.')
pip install $(echo "$RELEASE" | sed -e "s/cp[0-9]\{3\}/CP$CPVER/g")

After this, go ahead and reboot the Pi.

Copy Text

sudo reboot

sudo reboot

If you get a warning with "No module named 'rpi_vision'", this means Python couldn't load the module. Either the module or the Python binding likely wasn't installed correctly.

Running the Graphic Labeling Demo

Finally you are ready to run the detection software. To run a program that will display the object it sees on screen, type in the following:

Copy Text

cd rpi-vision
python3 tests/pitft_labeled_output.py --tflite

cd rpi-vision
python3 tests/pitft_labeled_output.py --tflite

You should see a bunch of text scrolling in your SSH window.

Now start holding up various items in front of the camera and it should display what it thinks it sees, which isn't actually what the item may be. Some items that it's pretty good about identifying are coffee mugs and animals.

Speech Output

As an added bonus, you can hook up a pair of headphones or a speaker to the Raspberry Pi and it will actually tell you what it is detecting. Make sure you don't have any HDMI cords plugged in though or the audio will go through the monitor.

If you don't hear anything, make sure the sound source you are expecting is selected. With the Raspberry Pi Desktop, you have a couple of ways to do this. If you have a mouse connected, you can right-click the speaker icon in the upper right-hand corner and choose your source.

If you prefer the command line, you can also run raspi-config, and go to System Options → Audio to select it.

If you still don't hear anything, make sure your volume is turned up by using alsamixer. For the BrainCraft HAT, this is described in more detail on the Audio Setup page.

Finally, you can test the text to speech from the command line by running the following command:

Copy Text