Overview

Save Subscribe

Have you ever thought about setting up your own Google Assistant device that you can customize? With the Raspberry Pi and BrainCraft HAT, you can do just that.

In this tutorial, we will walk you through setting up the Google Assistant API. Once that's all set up, you can install a few library, enable permissions and get the Google Assistant running on the Pi. Now you can ask Google what you want with the simple push of a button.

Parts

There are a few basic parts that are needed for this project. To start with, You can use a BrainCraft HAT which includes a display.

Video of a white hand hovering a coffe mug over a Adafruit BrainCraft HAT thats connected to a Raspberry Pi 4. Display detects that its a coffee mug.
The idea behind the BrainCraft HAT is that you’d be able to “craft brains” for Machine Learning on the EDGE, with Microcontrollers & Microcomputers. On ASK...
$44.95
In Stock
Add to Cart

Or you can use the Voice Bonnet.

Top down view of a Adafruit Voice Bonnet for Raspberry Pi connected to a Pi/
Your Raspberry Pi computer is like an electronic brain - and with the Adafruit Voice Bonnet you can give it a mouth and ears as well! Featuring two microphones and two...
$24.95
In Stock
Add to Cart

The rest of the parts are for either of the above options.

Angled shot of Raspberry Pi 4
NOTE: Due to stock limitations we may only be able to offer refunds or store credit for Pis that are defective, damaged or lost in...
Out of Stock
Out of Stock
Angled shot of Official Raspberry Pi Power Supply 5.1V 3A with USB C with Power plug facing down.
The official Raspberry Pi USB-C power supply is here! And of course, we have 'em in classic Adafruit black! Superfast with just the right amount of cable length to get your Pi 4...
$7.95
In Stock
Add to Cart
Enclosed Speaker with JST cable
Listen up! This 2.8" x 1.2" speaker is a great addition to any audio project where you need 4 ohm impedance and 3W or less of power. We particularly like...
$3.95
In Stock
Add to Cart
Hand removing/installing micro SD card from SD adapter
Add mega-storage in a jiffy using this 8 GB class 4 micro-SD card. It comes with a SD adapter so you can use it with any of our shields or adapters. Preformatted to FAT so it works out...
$9.95
In Stock
Add to Cart
Cell-phone TRRS Headset - Earbud Headphones with Microphone
These earbud headphones are the perfect accessory for your FONA - they've been tested to work with our modules - but can be used with any iOS or Android device that uses a TRRS...
$3.95
In Stock
Add to Cart

Google Setup

We will need to configure a new Google project and set up everything before we move over the the Raspberry Pi.

Project Creation

First, you'll need to configure a new Actions Console project.

Start by going to https://console.actions.google.com/ and clicking New Project

Enter a Project name such as Google Assistant BrainCraft and click on Create Project

At the bottom of the page, click on the link to go to Device Registration.

Device Registration

If you have accidentally closed the window, you can get to Device Registration by going back to the Actions Console and Selecting your project. After that, click on the Develop tab, the 3-Bar Menu in the upper left, and the Device registration menu item.

At the device registration, click the Register Model button.

Take note of your Model ID in this next step. You will need this in a later step.

Fill in the fields with the requested information. For device type, choose Speaker. Make note of your Model ID and save it in a safe place. Click the Register Model Button.

If you forgot to copy your Model ID, don't worry. This guide will show you how to retrieve it when you need it.

Click Download OAuth 2.0 Credentials and save the JSON file and put it in a safe place such as your Desktop. Save the file as client-secret.json. You will upload this to your Raspberry Pi in a later step. Click Next.

Under Traits, click All 7 traits and then click Save Traits

Google Assistant API Setup

Go to the Google Developers Console to enable the API at https://console.developers.google.com/apis/api/embeddedassistant.googleapis.com/overview

Before clicking the button, we need to select the project we created in the first step. By default, the incorrect project is selected.

Click on the project selector at the top.

This opens a dialog box of projects. If the project you created in the first step is not listed here, click inside the search box at the top.

Start typing the name of the project in the search box and click on the result that comes up. Click Open.

Now that the correct project is selected, we want to enable the Google API for this project. Click on the Enable button.

Once it is enabled, you will be taken to the Overview Screen. Click on the Credentials tab on the left and then click the Configure Consent Screen button on the right.

For User Type, select External and Click Create

Enter an Application Name. This is the name that will appear on the permissions screen.

Select a Support Email. This is the email that will appear when you click on the Application name on the permissions screen.

You may need to add another Email Address under Developer contact information near the bottom of the form.

Scroll to the bottom and Click Save and Continue

Add Test Users

Next you'll need to add yourself as a test user to your app or you will get an Access Denied error when you try to authorize your credentials.

Type in the email addresses of any of your Google Accounts that you will be using to authorize the app. Then click Save.

Enabling Permissions

Go to Activity Controls at https://myaccount.google.com/activitycontrols

Make sure Web & App Activity is on.

Also make sure to select the Include Chrome history and activity from sites, apps, and devices that use Google services checkbox.

An additional dialog may pop up asking you to confirm. Click the Turn on button.

You may want to turn on Location History and YouTube History as well.

Device Setup

Make sure you are using the 32-bit Lite version of Raspberry Pi OS. The desktop version has had some issues with Google Voice and the audio driver and Google drivers aren't available for the 64-bit version.

Next you'll want to setup all of the packages on the Raspberry Pi. If you haven't done so already, take a look at our Adafruit BrainCraft HAT - Easy Machine Learning for Raspberry Pi guide if you are using the BrainCraft HAT.

If you are using the Voice Bonnet, you'll want to follow the setup steps in our Adafruit Voice Bonnet guide.

These guides will take you through all the steps needed to get the Raspberry Pi updated and the BrainCraft HAT or Voice Bonnet all set up to the point we need to continue. In either case, make sure to install the 32-bit Lite version of Raspberry Pi OS.

internet_of_things___iot_braincraft.jpeg

Upload the client-secret.json OAuth 2.0 Credentials file that you downloaded in the Google Setup section to your Raspberry Pi and place it in your home directory. We're going to assume this is located at /home/pi. If it differs, please change any commands accordingly.

Next we're going to check that we have the required packages and setup a Virtual Environment.

Copy Text 
cd ~
sudo apt-get update
sudo apt-get install -y python3-dev python3-venv pulseaudio
python3 -m venv env
env/bin/python -m pip install --upgrade pip setuptools wheel
source env/bin/activate
internet_of_things___iot_Install_Libraries.png

Once you are done with that, your prompt should see (env) to the left to indicate you are in a Virtual Environment.

internet_of_things___iot_Activate_venv.png

A recent change to a Google SDK dependency requires rust to be installed and the version of rust included in apt-get is too low of a version to work. To install a more recent version, use the following commands:

Copy Text 
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

After it finishes running, you'll need to add it to your path:

Copy Text 
export PATH=$PATH:/home/pi/.cargo/bin
source "$HOME/.cargo/env"

Install the Authorization library:

Copy Text 
pip3 install --upgrade google-auth-oauthlib[tool]

Install the SDK with:

Copy Text 
sudo apt install -y portaudio19-dev libffi-dev libssl-dev
pip3 install --upgrade google-assistant-library
git clone https://github.com/adafruit/assistant-sdk-python.git
cd assistant-sdk-python/google-assistant-sdk/
pip install .
cd

Install Additional Libraries:

Copy Text 
pip install --upgrade pyaudio sounddevice tenacity google-assistant-grpc google-api-python-client adafruit-circuitpython-dotstar

Generating an OAuth Token

Before we can run the script, we need to generate an OAuth Token. Run the following command:

Be sure you have uploaded the client_secret.json file into your home directory or the OAuth Token generation script will fail.
Copy Text 
google-oauthlib-tool --scope https://www.googleapis.com/auth/assistant-sdk-prototype --save --headless --client-secrets ~/client_secret.json

The script should provide a URL to visit to generate the token. Go ahead and copy and paste the URL into a web browser.

You may run into an alert that says your app isn't verified. Go ahead and click on the Continue link.

After all that, it will come up with a confirmation dialog with your account name and the permissions that you are granting. Go ahead and click on Continue.

Finally, you will be given an Authorization Code. Click on the Copy icon and it will get copied to your clipboard.

Paste the Authorization Code back into the script and the token will be generated and saved.

Your Pi should now be all ready to go.

Usage

Now that you have everything set up, it's time to use the Google Voice Assistant. Let's put a script on the Pi, run it and look at some of the things it can do.

First make sure you are in the Virtual Environment. If you have exited for some reason such as restarting the Pi, you can get back in by typing:

Copy Text 
source env/bin/activate

Running the Google Assistant Script

We wrote a script that will make use of the on-board DotStar LEDs and button on the BrainCraft HAT to make it easier to use the Google Assistant. Below is the full script. You can either save it onto your Pi by copying it below or you can type:

Copy Text 
wget https://raw.githubusercontent.com/adafruit/Adafruit_Learning_System_Guides/master/BrainCraft_Google_Assistant/gv_buttontotalk.py

and it will save it directly to the Pi.

Associating your Project ID and Model ID

Google recently made a change that requires you to run a sample script that will associate your Hardware Model ID and Project ID for the main script to work. This step only needs to be done once. To get your Project ID you can open your client_secrets.json file and find it there.

If you saved your Model ID during the Google Setup, you're all set. If not, you can find it by following these steps.

Open the Actions console at https://console.actions.google.com/u/0/. Select your Project.

Click the Develop Tab at the top, then select Device Registration, then click on the product name.

Run the following command, swapping out [Your Project ID] and [Your Model ID] with the appropriate values:

googlesamples-assistant-pushtotalk --project-id [Your Project ID] --device-model-id [Your Model ID]

Press Control-C a couple of times to exit the script. Now you're ready to run the project script.

Running the Project Script

To run the main script, type:

Copy Text 
python3 gv_buttontotalk.py

Wait a few seconds until the script tells you to press a button and then press the button on the BrainCraft HAT. The lights should turn green, meaning it is waiting for you to give it a command or ask a question.

Phrases to Try

Try asking some of these phrases:

What is the weather?

What movies are playing?

What is your favorite color?

What are the directions to the nearest grocery store?

This guide was first published on Oct 21, 2020. It was last updated on Oct 21, 2020.