# Using Google Assistant on the BrainCraft HAT or Voice Bonnet
## Overview
https://www.youtube.com/watch?v=oS0FhTQRCM8&feature=youtu.be
Have you ever thought about setting up your own Google Assistant device that you can customize? With the Raspberry Pi and BrainCraft HAT, you can do just that.
In this tutorial, we will walk you through setting up the Google Assistant API. Once that's all set up, you can install a few library, enable permissions and get the Google Assistant running on the Pi. Now you can ask Google what you want with the simple push of a button.
## Parts
There are a few basic parts that are needed for this project. To start with, You can use a BrainCraft HAT which includes a display.
### Adafruit BrainCraft HAT - Machine Learning for Raspberry Pi 4
[Adafruit BrainCraft HAT - Machine Learning for Raspberry Pi 4](https://www.adafruit.com/product/4374)
The idea behind the BrainCraft HAT is that you’d be able to “craft brains” for Machine Learning on the EDGE, with Microcontrollers & Microcomputers. On ASK AN ENGINEER, our founder & engineer chatted with Pete Warden, the technical lead of the mobile,...
In Stock
[Buy Now](https://www.adafruit.com/product/4374)
[Related Guides to the Product](https://learn.adafruit.com/products/4374/guides)
Or you can use the Voice Bonnet.
The rest of the parts are for either of the above options.
### Raspberry Pi 4 Model B - 2 GB RAM
[Raspberry Pi 4 Model B - 2 GB RAM](https://www.adafruit.com/product/4292)
The Raspberry Pi 4 Model B is the newest Raspberry Pi computer made, and the Pi Foundation knows you can always make a good thing _better_! And what could make the Pi 4...
Out of Stock
[Buy Now](https://www.adafruit.com/product/4292)
[Related Guides to the Product](https://learn.adafruit.com/products/4292/guides)
### Official Raspberry Pi Power Supply 5.1V 3A with USB C
[Official Raspberry Pi Power Supply 5.1V 3A with USB C](https://www.adafruit.com/product/4298)
The official Raspberry Pi USB-C power supply is here! And of course, we have 'em in classic Adafruit black! Superfast with just the right amount of cable length to get your Pi 4 projects up and running!
Best for use with Pi 4 series, [Pi...](https://www.adafruit.com/product/5814)
In Stock
[Buy Now](https://www.adafruit.com/product/4298)
[Related Guides to the Product](https://learn.adafruit.com/products/4298/guides)
### Mono Enclosed Speaker - 3W 4 Ohm
[Mono Enclosed Speaker - 3W 4 Ohm](https://www.adafruit.com/product/3351)
Listen up! This 2.8" x 1.2" speaker is a great addition to any audio project where you need 4 ohm impedance and 3W or less of power. We particularly like this speaker as it is small and enclosed for good audio volume and quality. It has a handy JST 2PH...
In Stock
[Buy Now](https://www.adafruit.com/product/3351)
[Related Guides to the Product](https://learn.adafruit.com/products/3351/guides)
### SD/MicroSD Memory Card (8 GB SDHC)
[SD/MicroSD Memory Card (8 GB SDHC)](https://www.adafruit.com/product/1294)
Add mega-storage in a jiffy using this 8 GB class 4 micro-SD card. It comes with a SD adapter so you can use it with any of our shields or adapters. Preformatted to FAT so it works out of the box with our projects. Tested and works great with our
In Stock
[Buy Now](https://www.adafruit.com/product/1294)
[Related Guides to the Product](https://learn.adafruit.com/products/1294/guides)
### Cell-phone TRRS Headset - Earbud Headphones w/ Microphone
[Cell-phone TRRS Headset - Earbud Headphones w/ Microphone](https://www.adafruit.com/product/1966)
These earbud headphones are the perfect accessory for your FONA - they've been tested to work with our modules - but can be used with any iOS or Android device that uses a TRRS pinout. The buds themselves are 16mm in diameter with a nice 40" long. They come with a...
In Stock
[Buy Now](https://www.adafruit.com/product/1966)
[Related Guides to the Product](https://learn.adafruit.com/products/1966/guides)
# Using Google Assistant on the BrainCraft HAT or Voice Bonnet
## Google Setup
We will need to configure a new Google project and set up everything before we move over the the Raspberry Pi.
## Project Creation
First, you'll need to configure a new Actions Console project.
Start by going to [https://console.actions.google.com/](https://console.actions.google.com/) and clicking **New Project**
Enter a Project name such as **Google Assistant BrainCraft** and click on **Create Project**
At the bottom of the page, click on the link to go to Device Registration.
## Device Registration
If you have accidentally closed the window, you can get to Device Registration by going back to the Actions Console and Selecting your project. After that, click on the **Develop** tab, the **3-Bar Menu** in the upper left, and the ** Device registration** menu item.
At the device registration, click the **Register Model** button.
Info:
Fill in the fields with the requested information. For device type, choose **Speaker**. Make note of your **Model ID** and save it in a safe place. Click the **Register Model** Button.
If you forgot to copy your **Model ID** , don't worry. This guide will show you how to retrieve it when you need it.
Click **Download OAuth 2.0 Credentials** and save the JSON file and put it in a safe place such as your Desktop. Save the file as **client-secret.json.** You will upload this to your Raspberry Pi in a later step. Click **Next**.
Under Traits, click **All 7 traits** and then click **Save Traits**
## Google Assistant API Setup
Go to the Google Developers Console to enable the API at [https://console.developers.google.com/apis/api/embeddedassistant.googleapis.com/overview](https://console.developers.google.com/apis/api/embeddedassistant.googleapis.com/overview)
**Before clicking the button, we need to select the project we created in the first step. By default, the incorrect project is selected.**
Click on the project selector at the top.
This opens a dialog box of projects. If the project you created in the first step is not listed here, click inside the search box at the top.
Start typing the name of the project in the search box and click on the result that comes up. Click **Open**.
Now that the correct project is selected, we want to enable the Google API for this project. Click on the **Enable** button.
Once it is enabled, you will be taken to the Overview Screen. Click on the **Credentials** tab on the left and then click the **Configure Consent Screen** button on the right.
For User Type, select **External** and Click **Create**
Enter an **Application Name**. This is the name that will appear on the permissions screen.
Select a **Support Email**. This is the email that will appear when you click on the Application name on the permissions screen.
You may need to add another **Email Address** under **Developer contact information** near the bottom of the form.
Scroll to the bottom and Click **Save and Continue**
## Add Test Users
Next you'll need to add yourself as a test user to your app or you will get an Access Denied error when you try to authorize your credentials.
Type in the email addresses of any of your Google Accounts that you will be using to authorize the app. Then click **Save**.
## Enabling Permissions
Go to Activity Controls at [https://myaccount.google.com/activitycontrols](https://myaccount.google.com/activitycontrols)
Make sure **Web & App Activit** y is on.
Also make sure to select the **Include Chrome history and activity from sites, apps, and devices that use Google services** checkbox.
An additional dialog may pop up asking you to confirm. Click the **Turn on** button.
You may want to turn on **Location History** and **YouTube History** as well.
# Using Google Assistant on the BrainCraft HAT or Voice Bonnet
## Device Setup
Danger:
Next you'll want to setup all of the packages on the Raspberry Pi. If you haven't done so already, take a look at our [Adafruit BrainCraft HAT - Easy Machine Learning for Raspberry Pi](https://learn.adafruit.com/adafruit-braincraft-hat-easy-machine-learning-for-raspberry-pi) guide if you are using the BrainCraft HAT.
If you are using the Voice Bonnet, you'll want to follow the setup steps in our [Adafruit Voice Bonnet](https://learn.adafruit.com/adafruit-voice-bonnet) guide.
These guides will take you through all the steps needed to get the Raspberry Pi updated and the **BrainCraft HAT** or **Voice Bonnet** all set up to the point we need to continue. In either case, make sure to install the 32-bit Lite version of Raspberry Pi OS.
Upload the **client-secret.json** OAuth 2.0 Credentials file that you downloaded in the Google Setup section to your Raspberry Pi and place it in your home directory. We're going to assume this is located at **/home/pi**. If it differs, please change any commands accordingly.
Next we're going to check that we have the required packages and setup a Virtual Environment.
```terminal
cd ~
sudo apt-get update
sudo apt-get install -y python3-dev python3-venv pulseaudio
python3 -m venv env
env/bin/python -m pip install --upgrade pip setuptools wheel
source env/bin/activate
```
Once you are done with that, your prompt should see **(env)** to the left to indicate you are in a Virtual Environment.
A recent change to a Google SDK dependency requires rust to be installed and the version of rust included in apt-get is too low of a version to work. To install a more recent version, use the following commands:
```terminal
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
After it finishes running, you'll need to add it to your path:
```terminal
export PATH=$PATH:/home/pi/.cargo/bin
source "$HOME/.cargo/env"
```
Install the Authorization library:
```terminal
pip3 install --upgrade google-auth-oauthlib[tool]
```
Install the SDK with:
```terminal
sudo apt install -y portaudio19-dev libffi-dev libssl-dev
pip3 install --upgrade google-assistant-library
git clone https://github.com/adafruit/assistant-sdk-python.git
cd assistant-sdk-python/google-assistant-sdk/
pip install .
cd
```
Install Additional Libraries:
```terminal
pip install --upgrade pyaudio sounddevice tenacity google-assistant-grpc google-api-python-client adafruit-circuitpython-dotstar
```
## Generating an OAuth Token
Before we can run the script, we need to generate an OAuth Token. Run the following command:
Warning:
```terminal
google-oauthlib-tool --scope https://www.googleapis.com/auth/assistant-sdk-prototype --save --headless --client-secrets ~/client_secret.json
```
The script should provide a URL to visit to generate the token. Go ahead and copy and paste the URL into a web browser.
You may run into an alert that says your app isn't verified. Go ahead and click on the **Continue** link.
After all that, it will come up with a confirmation dialog with your account name and the permissions that you are granting. Go ahead and click on **Continue**.
Finally, you will be given an Authorization Code. Click on the **Copy** icon and it will get copied to your clipboard.
Paste the Authorization Code back into the script and the token will be generated and saved.
Your Pi should now be all ready to go.
# Using Google Assistant on the BrainCraft HAT or Voice Bonnet
## Usage
Now that you have everything set up, it's time to use the Google Voice Assistant. Let's put a script on the Pi, run it and look at some of the things it can do.
First make sure you are in the Virtual Environment. If you have exited for some reason such as restarting the Pi, you can get back in by typing:
```terminal
source env/bin/activate
```
## Running the Google Assistant Script
We wrote a script that will make use of the on-board DotStar LEDs and button on the BrainCraft HAT to make it easier to use the Google Assistant. Below is the full script. You can either save it onto your Pi by copying it below or you can type:
```terminal
wget https://raw.githubusercontent.com/adafruit/Adafruit_Learning_System_Guides/master/BrainCraft_Google_Assistant/gv_buttontotalk.py
```
and it will save it directly to the Pi.
## Associating your Project ID and Model ID
Google recently made a change that requires you to run a sample script that will associate your Hardware Model ID and Project ID for the main script to work. **This step only needs to be done once.** To get your **Project ID** you can open your **client\_secrets.json** file and find it there.
If you saved your **Model ID** during the Google Setup, you're all set. If not, you can find it by following these steps.
Open the Actions console at[https://console.actions.google.com/u/0/](https://console.actions.google.com/u/0/). Select your Project.
Click the **Develop** Tab at the top, then select **Device Registration** , then click on the product name.
Run the following command, swapping out **[Your Project ID]** and **[Your Model ID]** with the appropriate values:
`googlesamples-assistant-pushtotalk --project-id [Your Project ID] --device-model-id [Your Model ID]`
Press Control-C a couple of times to exit the script. Now you're ready to run the project script.
## Running the Project Script
To run the main script, type:
```terminal
python3 gv_buttontotalk.py
```
Wait a few seconds until the script tells you to press a button and then press the button on the BrainCraft HAT. The lights should turn green, meaning it is waiting for you to give it a command or ask a question.
https://www.youtube.com/watch?v=oS0FhTQRCM8&feature=youtu.be
## Phrases to Try
Try asking some of these phrases:
**What is the weather?**
**What movies are playing?**
**What is your favorite color?**
**What are the directions to the nearest grocery store?**
## Featured Products
### Adafruit BrainCraft HAT - Machine Learning for Raspberry Pi 4
[Adafruit BrainCraft HAT - Machine Learning for Raspberry Pi 4](https://www.adafruit.com/product/4374)
The idea behind the BrainCraft HAT is that you’d be able to “craft brains” for Machine Learning on the EDGE, with Microcontrollers & Microcomputers. On ASK AN ENGINEER, our founder & engineer chatted with Pete Warden, the technical lead of the mobile,...
In Stock
[Buy Now](https://www.adafruit.com/product/4374)
[Related Guides to the Product](https://learn.adafruit.com/products/4374/guides)
### Adafruit Voice Bonnet for Raspberry Pi -Two Speakers + Two Mics
[Adafruit Voice Bonnet for Raspberry Pi -Two Speakers + Two Mics](https://www.adafruit.com/product/4757)
Your Raspberry Pi computer is like an electronic brain - and with the **Adafruit Voice Bonnet** you can give it a mouth and ears as well! Featuring two microphones and two 1 Watt speaker outputs using a high quality I2S codec, this Pi add-on will work with _any Raspberry Pi..._
No Longer Stocked
[Buy Now](https://www.adafruit.com/product/4757)
[Related Guides to the Product](https://learn.adafruit.com/products/4757/guides)
### Raspberry Pi 4 Model B - 2 GB RAM
[Raspberry Pi 4 Model B - 2 GB RAM](https://www.adafruit.com/product/4292)
The Raspberry Pi 4 Model B is the newest Raspberry Pi computer made, and the Pi Foundation knows you can always make a good thing _better_! And what could make the Pi 4...
Out of Stock
[Buy Now](https://www.adafruit.com/product/4292)
[Related Guides to the Product](https://learn.adafruit.com/products/4292/guides)
### Official Raspberry Pi Power Supply 5.1V 3A with USB C
[Official Raspberry Pi Power Supply 5.1V 3A with USB C](https://www.adafruit.com/product/4298)
The official Raspberry Pi USB-C power supply is here! And of course, we have 'em in classic Adafruit black! Superfast with just the right amount of cable length to get your Pi 4 projects up and running!
Best for use with Pi 4 series, [Pi...](https://www.adafruit.com/product/5814)
In Stock
[Buy Now](https://www.adafruit.com/product/4298)
[Related Guides to the Product](https://learn.adafruit.com/products/4298/guides)
### SD/MicroSD Memory Card (8 GB SDHC)
[SD/MicroSD Memory Card (8 GB SDHC)](https://www.adafruit.com/product/1294)
Add mega-storage in a jiffy using this 8 GB class 4 micro-SD card. It comes with a SD adapter so you can use it with any of our shields or adapters. Preformatted to FAT so it works out of the box with our projects. Tested and works great with our
In Stock
[Buy Now](https://www.adafruit.com/product/1294)
[Related Guides to the Product](https://learn.adafruit.com/products/1294/guides)
### Cell-phone TRRS Headset - Earbud Headphones w/ Microphone
[Cell-phone TRRS Headset - Earbud Headphones w/ Microphone](https://www.adafruit.com/product/1966)
These earbud headphones are the perfect accessory for your FONA - they've been tested to work with our modules - but can be used with any iOS or Android device that uses a TRRS pinout. The buds themselves are 16mm in diameter with a nice 40" long. They come with a...
In Stock
[Buy Now](https://www.adafruit.com/product/1966)
[Related Guides to the Product](https://learn.adafruit.com/products/1966/guides)
### Mono Enclosed Speaker - 3W 4 Ohm
[Mono Enclosed Speaker - 3W 4 Ohm](https://www.adafruit.com/product/3351)
Listen up! This 2.8" x 1.2" speaker is a great addition to any audio project where you need 4 ohm impedance and 3W or less of power. We particularly like this speaker as it is small and enclosed for good audio volume and quality. It has a handy JST 2PH...
In Stock
[Buy Now](https://www.adafruit.com/product/3351)
[Related Guides to the Product](https://learn.adafruit.com/products/3351/guides)
### Monoprice 5-Watt Guitar Amplifier, Portable Recorder
[Monoprice 5-Watt Guitar Amplifier, Portable Recorder](https://www.adafruit.com/product/4402)
Always be ready rock out or record with this **5-watt Guitar Amplifier, Portable Recorder, and USB Audio Interface** from Monoprice.
This handheld 5-watt amp delivers plenty of loudness and clarity for ad hoc _al fresco_ performances. It has a built-in guitar...
No Longer Stocked
[Buy Now](https://www.adafruit.com/product/4402)
[Related Guides to the Product](https://learn.adafruit.com/products/4402/guides)
## Related Guides
- [Raspberry Pi Care and Troubleshooting](https://learn.adafruit.com/raspberry-pi-care-and-troubleshooting.md)
- [Adafruit BrainCraft HAT - Easy Machine Learning for Raspberry Pi](https://learn.adafruit.com/adafruit-braincraft-hat-easy-machine-learning-for-raspberry-pi.md)
- [Build an ML Rock Paper Scissors Game with Lobe](https://learn.adafruit.com/lobe-rock-paper-scissors.md)
- [Machine Learning 101 with Lobe and BrainCraft](https://learn.adafruit.com/machine-learning-101-lobe-braincraft.md)
- [Python Edge Speech Recognition with Voice2JSON](https://learn.adafruit.com/edge-speech-recognition-with-voice2json.md)
- [Build an ML Package Detector with Lobe](https://learn.adafruit.com/build-an-ml-package-detector.md)
- [Raspberry Pi YouTube Boombox](https://learn.adafruit.com/youtube-radio.md)
- [BrainCraft Camera Case](https://learn.adafruit.com/braincraft-camera-case.md)
- [diy lofi hip hop raspberry pi radio](https://learn.adafruit.com/lofi-hip-hop-raspberry-pi-radio-braincraft.md)
- [Basic TensorFlow Object Recognition on any Computer or iOS device with Google Colab](https://learn.adafruit.com/basic-tensorflow-object-recognition-in-the-cloud-google-colab.md)
- [Running TensorFlow Lite Object Recognition on the Raspberry Pi 4 or Pi 5](https://learn.adafruit.com/running-tensorflow-lite-on-the-raspberry-pi-4.md)
- [Raspberry Pi + Teachable Machine = Teachable Pi](https://learn.adafruit.com/teachable-machine-raspberry-pi-tensorflow-camera.md)
- [Generating AI “Art” with VQGAN+CLIP](https://learn.adafruit.com/generating-ai-art-with-vqgan-clip.md)
- [Breakout Game on the Metro RP2350 and Fruit Jam](https://learn.adafruit.com/breakout-game-on-metro-rp2350-and-fruit-jam.md)
- [CircuitPython on the Xteink X4 eReader](https://learn.adafruit.com/circuitpython-on-the-xteink-x4-ereader.md)
- [Adafruit PyPortal - IoT for CircuitPython](https://learn.adafruit.com/adafruit-pyportal.md)