Using Google Assistant on the BrainCraft HAT or Voice Bonnet

Adafruit Voice Bonnet for Raspberry Pi -Two Speakers + Two Mics

Or you can use the Voice Bonnet.

Top down view of a Adafruit Voice Bonnet for Raspberry Pi connected to a Pi/

Your Raspberry Pi computer is like an electronic brain - and with the Adafruit Voice Bonnet you can give it a mouth and ears as well! Featuring two microphones and two...

view guides using product

$24.95

In Stock

Raspberry Pi 4 Model B - 2 GB RAM

The rest of the parts are for either of the above options.

The Raspberry Pi 4 Model B is the newest Raspberry Pi computer made, and the Pi Foundation knows you can always make a good thing

$45.00

In Stock

Official Raspberry Pi Power Supply 5.1V 3A with USB C

The official Raspberry Pi USB-C power supply is here! And of course, we have 'em in classic Adafruit black! Superfast with just the right amount of cable length to get your Pi 4...

$7.95

In Stock

Mono Enclosed Speaker - 3W 4 Ohm

Listen up! This 2.8" x 1.2" speaker is a great addition to any audio project where you need 4 ohm impedance and 3W or less of power. We particularly like...

$3.95

In Stock

SD/MicroSD Memory Card (8 GB SDHC)

Hand removing/installing micro SD card from SD adapter

Add mega-storage in a jiffy using this 8 GB class 4 micro-SD card. It comes with a SD adapter so you can use it with any of our shields or adapters. Preformatted to FAT so it works out...

$9.95

In Stock

Cell-phone TRRS Headset - Earbud Headphones w/ Microphone

Cell-phone TRRS Headset - Earbud Headphones with Microphone

These earbud headphones are the perfect accessory for your FONA - they've been tested to work with our modules - but can be used with any iOS or Android device that uses a TRRS...

$3.95

In Stock

Text editor powered by tinymce.

We will need to configure a new Google project and set up everything before we move over the the Raspberry Pi.

Project Creation

First, you'll need to configure a new Actions Console project.

Start by going to https://console.actions.google.com/ and clicking New Project

Enter a Project name such as Google Assistant BrainCraft and click on Create Project

At the bottom of the page, click on the link to go to Device Registration.

Device Registration

If you have accidentally closed the window, you can get to Device Registration by going back to the Actions Console and Selecting your project. After that, click on the Develop tab, the 3-Bar Menu in the upper left, and the Device registration menu item.

At the device registration, click the Register Model button.

Take note of your Model ID in this next step. You will need this in a later step.

Fill in the fields with the requested information. For device type, choose Speaker. Make note of your Model ID and save it in a safe place. Click the Register Model Button.

If you forgot to copy your Model ID, don't worry. This guide will show you how to retrieve it when you need it.

Click Download OAuth 2.0 Credentials and save the JSON file and put it in a safe place such as your Desktop. Save the file as client-secret.json. You will upload this to your Raspberry Pi in a later step. Click Next.

Under Traits, click All 7 traits and then click Save Traits

Google Assistant API Setup

Go to the Google Developers Console to enable the API at https://console.developers.google.com/apis/api/embeddedassistant.googleapis.com/overview

Before clicking the button, we need to select the project we created in the first step. By default, the incorrect project is selected.

Click on the project selector at the top.

This opens a dialog box of projects. If the project you created in the first step is not listed here, click inside the search box at the top.

Start typing the name of the project in the search box and click on the result that comes up. Click Open.

Now that the correct project is selected, we want to enable the Google API for this project. Click on the Enable button.

Once it is enabled, you will be taken to the Overview Screen. Click on the Credentials tab on the left and then click the Configure Consent Screen button on the right.

For User Type, select External and Click Create

Enter an Application Name. This is the name that will appear on the permissions screen.

Select a Support Email. This is the email that will appear when you click on the Application name on the permissions screen.

You may need to add another Email Address under Developer contact information near the bottom of the form.

Scroll to the bottom and Click Save and Continue

Add Test Users

Next you'll need to add yourself as a test user to your app or you will get an Access Denied error when you try to authorize your credentials.

Type in the email addresses of any of your Google Accounts that you will be using to authorize the app. Then click Save.

Enabling Permissions

Go to Activity Controls at https://myaccount.google.com/activitycontrols

Make sure Web & App Activity is on.

Also make sure to select the Include Chrome history and activity from sites, apps, and devices that use Google services checkbox.

An additional dialog may pop up asking you to confirm. Click the Turn on button.

You may want to turn on Location History and YouTube History as well.

Text editor powered by tinymce.

Make sure you are using the 32-bit Lite version of Raspberry Pi OS. The desktop version has had some issues with Google Voice and the audio driver and Google drivers aren't available for the 64-bit version.

Next you'll want to setup all of the packages on the Raspberry Pi. If you haven't done so already, take a look at our Adafruit BrainCraft HAT - Easy Machine Learning for Raspberry Pi guide if you are using the BrainCraft HAT.

If you are using the Voice Bonnet, you'll want to follow the setup steps in our Adafruit Voice Bonnet guide.

These guides will take you through all the steps needed to get the Raspberry Pi updated and the BrainCraft HAT or Voice Bonnet all set up to the point we need to continue. In either case, make sure to install the 32-bit Lite version of Raspberry Pi OS.

Upload the client-secret.json OAuth 2.0 Credentials file that you downloaded in the Google Setup section to your Raspberry Pi and place it in your home directory. We're going to assume this is located at /home/pi. If it differs, please change any commands accordingly.

Next we're going to check that we have the required packages and setup a Virtual Environment.

cd ~
sudo apt-get update
sudo apt-get install -y python3-dev python3-venv pulseaudio
python3 -m venv env
env/bin/python -m pip install --upgrade pip setuptools wheel
source env/bin/activate

cd ~
sudo apt-get update
sudo apt-get install -y python3-dev python3-venv pulseaudio
python3 -m venv env
env/bin/python -m pip install --upgrade pip setuptools wheel
source env/bin/activate

Once you are done with that, your prompt should see (env) to the left to indicate you are in a Virtual Environment.

A recent change to a Google SDK dependency requires rust to be installed and the version of rust included in apt-get is too low of a version to work. To install a more recent version, use the following commands:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

After it finishes running, you'll need to add it to your path:

export PATH=$PATH:/home/pi/.cargo/bin
source "$HOME/.cargo/env"

export PATH=$PATH:/home/pi/.cargo/bin
source "$HOME/.cargo/env"

Install the Authorization library:

pip3 install --upgrade google-auth-oauthlib[tool]

pip3 install --upgrade google-auth-oauthlib[tool]

Install the SDK with:

sudo apt install -y portaudio19-dev libffi-dev libssl-dev
pip3 install --upgrade google-assistant-library
git clone https://github.com/adafruit/assistant-sdk-python.git
cd assistant-sdk-python/google-assistant-sdk/
pip install .
cd

sudo apt install -y portaudio19-dev libffi-dev libssl-dev
pip3 install --upgrade google-assistant-library
git clone https://github.com/adafruit/assistant-sdk-python.git
cd assistant-sdk-python/google-assistant-sdk/
pip install .
cd

Install Additional Libraries:

pip install --upgrade pyaudio sounddevice tenacity google-assistant-grpc google-api-python-client adafruit-circuitpython-dotstar

pip install --upgrade pyaudio sounddevice tenacity google-assistant-grpc google-api-python-client adafruit-circuitpython-dotstar

Generating an OAuth Token

Before we can run the script, we need to generate an OAuth Token. Run the following command:

Be sure you have uploaded the client_secret.json file into your home directory or the OAuth Token generation script will fail.

google-oauthlib-tool --scope https://www.googleapis.com/auth/assistant-sdk-prototype --save --headless --client-secrets ~/client_secret.json

google-oauthlib-tool --scope https://www.googleapis.com/auth/assistant-sdk-prototype --save --headless --client-secrets ~/client_secret.json

The script should provide a URL to visit to generate the token. Go ahead and copy and paste the URL into a web browser.

You may run into an alert that says your app isn't verified. Go ahead and click on the Continue link.

After all that, it will come up with a confirmation dialog with your account name and the permissions that you are granting. Go ahead and click on Continue.

Finally, you will be given an Authorization Code. Click on the Copy icon and it will get copied to your clipboard.

Paste the Authorization Code back into the script and the token will be generated and saved.

Your Pi should now be all ready to go.

Text editor powered by tinymce.

Now that you have everything set up, it's time to use the Google Voice Assistant. Let's put a script on the Pi, run it and look at some of the things it can do.

First make sure you are in the Virtual Environment. If you have exited for some reason such as restarting the Pi, you can get back in by typing:

source env/bin/activate

source env/bin/activate

Running the Google Assistant Script

We wrote a script that will make use of the on-board DotStar LEDs and button on the BrainCraft HAT to make it easier to use the Google Assistant. Below is the full script. You can either save it onto your Pi by copying it below or you can type:

wget https://raw.githubusercontent.com/adafruit/Adafruit_Learning_System_Guides/master/BrainCraft_Google_Assistant/gv_buttontotalk.py

wget https://raw.githubusercontent.com/adafruit/Adafruit_Learning_System_Guides/master/BrainCraft_Google_Assistant/gv_buttontotalk.py

and it will save it directly to the Pi.

Associating your Project ID and Model ID

Google recently made a change that requires you to run a sample script that will associate your Hardware Model ID and Project ID for the main script to work. This step only needs to be done once. To get your Project ID you can open your client_secrets.json file and find it there.

If you saved your Model ID during the Google Setup, you're all set. If not, you can find it by following these steps.

Open the Actions console at https://console.actions.google.com/u/0/. Select your Project.

Click the Develop Tab at the top, then select Device Registration, then click on the product name.

Run the following command, swapping out [Your Project ID] and [Your Model ID] with the appropriate values:

googlesamples-assistant-pushtotalk --project-id [Your Project ID] --device-model-id [Your Model ID]

Press Control-C a couple of times to exit the script. Now you're ready to run the project script.

Running the Project Script

To run the main script, type: