Have you ever thought about setting up your own Google Assistant device that you can customize? With the Raspberry Pi and BrainCraft HAT, you can do just that.

In this tutorial, we will walk you through setting up the Google Assistant API. Once that's all set up, you can install a few library, enable permissions and get the Google Assistant running on the Pi. Now you can ask Google what you want with the simple push of a button.

Parts

There are a few basic parts that are needed for this project. To start with, You can use a BrainCraft HAT which includes a display.

The idea behind the BrainCraft HAT is that you’d be able to “craft brains” for Machine Learning on the EDGE, with Microcontrollers & Microcomputers. On ASK...
$44.95
In Stock

Or you can use the Voice Bonnet.

Your Raspberry Pi computer is like an electronic brain - and with the Adafruit Voice Bonnet you can give it a mouth and ears as well! Featuring two microphones and two...
$19.95
In Stock

The rest of the parts are for either of the above options.

The Raspberry Pi 4 Model B is the newest Raspberry Pi computer made, and the Pi Foundation knows you can always make a good thing better! And what could make the Pi 4 better...
Out of Stock
The official Raspberry Pi USB-C power supply is here! And of course, we have 'em in classic Adafruit black! Superfast with just the right amount of cable length to get your Pi 4...
$7.95
In Stock
Listen up! This 2.8" x 1.2" speaker is a great addition to any audio project where you need 4 ohm impedance and 3W or less of power. We particularly like...
$3.95
In Stock
Add mega-storage in a jiffy using this 8 GB class 4 micro-SD card. It comes with a SD adapter so you can use it with any of our shields or adapters. Preformatted to FAT so it works out...
$9.95
In Stock
Always be ready rock out or record with this 5-watt Guitar Amplifier, Portable Recorder, and USB Audio Interface from Monoprice.This handheld 5-watt amp...
Out of Stock
These earbud headphones are the perfect accessory for your FONA - they've been tested to work with our modules - but can be used with any iOS or Android device that uses a TRRS...
$3.95
In Stock

We will need to configure a new Google project and set up everything before we move over the the Raspberry Pi.

Project Creation

First, you'll need to configure a new Actions Console project.

Start by going to https://console.actions.google.com/ and clicking New Project

Enter a Project name such as Google Assistant BrainCraft and click on Create Project

At the bottom of the page, click on the link to go to Device Registration.

Device Registration

If you have accidentally closed the window, you can get to Device Registration by going back to the Actions Console and Selecting your project. After that, click on the Develop tab, the 3-Bar Menu in the upper left, and the Device registration menu item.

At the device registration, click the Register Model button.

Take note of your Model ID in this next step. You will need this in a later step.

Fill in the fields with the requested information. For device type, choose Speaker. Make note of your Model ID and save it in a safe place. Click the Register Model Button.

Click Download OAuth 2.0 Credentials and save the JSON file and put it in a safe place such as your Desktop. Save the file as client-secret.json. You will upload this to your Raspberry Pi in a later step. Click Next.

Under Traits, click All 7 traits and then click Save Traits

Google Assistant API Setup

Go to the Google Developers Console to enable the API at https://console.developers.google.com/apis/api/embeddedassistant.googleapis.com/overview

Before clicking the button, we need to select the project we created in the first step. By default, the incorrect project is selected.

Click on the project selector at the top.

This opens a dialog box of projects. If the project you created in the first step is not listed here, click inside the search box at the top.

Start typing the name of the project in the search box and click on the result that comes up. Click Open.

Now that the correct project is selected, we want to enable the Google API for this project. Click on the Enable button.

Once it is enabled, you will be taken to the Overview Screen. Click on the Credentials tab on the left and then click the Configure Consent Screen button on the right.

For User Type, select External and Click Create

Enter an Application Name. This is the name that will appear on the permissions screen.

Select a Support Email. This is the email that will appear when you click on the Application name on the permissions screen.

You may need to add another Email Address under Developer contact information near the bottom of the form.

Scroll to the bottom and Click Save and Continue

Publish the App

Next you'll need to publish your app or you will get an Access Denied error when you try to authorize your credentials.

Click the Confirm link to finalize publishing it.

Enabling Permissions

Go to Activity Controls at https://myaccount.google.com/activitycontrols

Make sure Web & App Activity is on.

Also make sure to select the Include Chrome history and activity from sites, apps, and devices that use Google services checkbox.

An additional dialog may pop up asking you to confirm. Click the Turn on button.

You may want to turn on Location History and YouTube History as well.

Make sure you are using the Lite version of Raspberry Pi OS. The desktop version has had some issues with Google Voice and the audio driver.

Next we want to setup all of the packages on the Raspberry Pi. If you haven't done so already, take a look at our Adafruit BrainCraft HAT - Easy Machine Learning for Raspberry Pi guide if you are using the BrainCraft HAT.

If you are using the Voice Bonnet, you'll want to follow the setup steps in our Adafruit Voice Bonnet guide.

This will take you through all the steps needed to get the Raspberry Pi updated and the BrainCraft HAT or Voice Bonnet all set up to the point we need to continue.

Upload the client-secret.json OAuth 2.0 Credentials file that you downloaded in the Google Setup section to your Raspberry Pi and place it in your home directory. We're going to assume this is located at /home/pi. If it differs, please change any commands accordingly.

Next we're going to check that we have the required packages and setup a Virtual Environment.

cd ~
sudo apt-get update
sudo apt-get install -y python3-dev python3-venv pulseaudio
python3 -m venv env
env/bin/python -m pip install --upgrade pip setuptools wheel
source env/bin/activate

Once you are done with that, your prompt should see (env) to the left to indicate you are in a Virtual Environment.

A recent change to a Google SDK dependency requires rust to be installed and the version of rust included in apt-get is too low of a version to work. To install a more recent version, use the following commands:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

After it finishes running, you'll need to add it to your path:

export PATH=$PATH:/home/pi/.cargo/bin

Install the SDK with:

sudo apt-get install -y portaudio19-dev libffi-dev libssl-dev
python -m pip install --upgrade google-assistant-sdk[samples]

Install the Authorization library.

python -m pip install --upgrade google-auth-oauthlib[tool]

Generating an OAuth Token

Before we can run the script, we need to generate an OAuth Token. Run the following command:

Be sure you have uploaded the client_secret.json file into your home directory or the OAuth Token generation script will fail.
google-oauthlib-tool --scope https://www.googleapis.com/auth/assistant-sdk-prototype --save --headless --client-secrets ~/client_secret.json

The script should provide a URL to visit to generate the token. Go ahead and copy and paste the URL into a web browser.

You may run into an alert that says your app isn't verified. Go ahead and click on the Advanced link.

That will expand the dialog. Click Go to Quickstart (unsafe) or whatever you decided on for a project name.

After all that, it will come up with a confirmation dialog with your account name and the permissions that you are granting. Go ahead and click on Allow.

Finally, you will be given an Authorization Code. Click on the Copy icon and it will get copied to your clipboard.

Paste the Authorization Code back into the script and the token will be generated and saved.

Your Pi should now be all ready to go.

Now that you have everything set up, it's time to use the Google Voice Assistant. Let's put a script on the Pi, run it and look at some of the things it can do.

First make sure you are in the Virtual Environment. If you have exited for some reason such as restarting the Pi, you can get back in by typing:

source env/bin/activate

Running the Google Assistant Script

We wrote a script that will make use of the on-board DotStar LEDs and button on the BrainCraft HAT to make it easier to use the Google Assistant. Below is the full script. You can either save it onto your Pi by copying it below or you can type:

wget https://raw.githubusercontent.com/adafruit/Adafruit_Learning_System_Guides/master/BrainCraft_Google_Assistant/gv_buttontotalk.py

and it will save it directly to the Pi.

Associating your Project ID and Model ID

Google recently made a change that requires you to run a sample script that will associate your Hardware Model ID and Project ID for the main script to work. This step only needs to be done once. To get your Project ID you can open your client_secrets.json file and find it there.

If you saved your Model ID during the Google Setup, you're all set. If not, you can find it by following these steps.

Open the Actions console at https://console.actions.google.com/u/0/. Select your Project.

Click the Develop Tab at the top, then select Device Registration, then click on the product name.

Run the following command, swapping out [Your Project ID] and [Your Model ID] with the appropriate values:

googlesamples-assistant-pushtotalk --project-id [Your Project ID] --device-model-id [Your Model ID]

Press Control-C to exit the script. Now you're ready to run the project script.

Running the Project Script

To run the main script, type:

python3 gv_buttontotalk.py

Wait a few seconds until the script tells you to press a button and then press the button on the BrainCraft HAT. The lights should turn green, meaning it is waiting for you to give it a command or ask a question.

Phrases to Try

Try asking some of these phrases:

What is the weather?

What movies are playing?

What is your favorite color?

What are the directions to the nearest grocery store?

This guide was first published on Oct 21, 2020. It was last updated on Oct 21, 2020.