# Sampling Keyboard with Fruit Jam
## Overview
https://www.youtube.com/watch?v=K60zSBKCgMk
This project is a USB sampling keyboard inspired by the [Casio SK-1](https://en.wikipedia.org/wiki/Casio_SK-1). You can record samples using the microphone and play them back at different pitches using the keys on the USB MIDI keyboard. No keyboard skills? No problem, use MIDI file playback to automatically play an entire song with the press of a single button. The project uses an I2S microphone, the `audiofilewriter` module, and granular pitch shifting. All of which are recent additions to CircuitPython, available as of development release **10.3.0-alpha.4**.
Samples get saved to the microSD card inserted into the built-in slot on the Fruit Jam. Meaning, unlike the SK-1, your sample persists while the device is turned off. Once you power it back up, the sample will be loaded and ready for you to begin playing with the keyboard.
## Parts
Featured
### Adafruit Fruit Jam - Mini RP2350 Computer
[Adafruit Fruit Jam - Mini RP2350 Computer](https://www.adafruit.com/product/6200)
We were catching up on a recent [hackaday hackchat with eben upton](https://hackaday.io/event/202122-raspberry-pi-hack-chat-with-eben-upton) and learned some fun facts: such as the DVI hack for the RP2040 was inspired by
In Stock
[Buy Now](https://www.adafruit.com/product/6200)
[Related Guides to the Product](https://learn.adafruit.com/products/6200/guides)
Featured
### Adafruit I2S MEMS Microphone Breakout - ICS-43434
[Adafruit I2S MEMS Microphone Breakout - ICS-43434](https://www.adafruit.com/product/6049)
Listen to this good news - we now have a breakout board for a super tiny **ICS43434** **I2S** MEMS microphone. Just like 'classic' electret microphones, MEMS mics can detect sound and convert it to voltage, but they're way smaller and thinner. This...
In Stock
[Buy Now](https://www.adafruit.com/product/6049)
[Related Guides to the Product](https://learn.adafruit.com/products/6049/guides)
### USB Midi Keyboard
I used the [Akai MPK mini 4](https://www.akaipro.com/mpk-mini-4/), but any standard USB midi keyboard should work.
Featured
### USB Powered Speakers
[USB Powered Speakers](https://www.adafruit.com/product/1363)
Add some extra boom to your audio project with these powered loudspeakers. We sampled half a dozen different models to find ones with a good frequency response, so you'll get quality audio output for music playback. Simply connect the standard 3.5mm stereo plug into your Raspberry Pi model...
In Stock
[Buy Now](https://www.adafruit.com/product/1363)
[Related Guides to the Product](https://learn.adafruit.com/products/1363/guides)
Featured
### 512MB micro SD Memory Card
[512MB micro SD Memory Card](https://www.adafruit.com/product/5252)
Add storage in a jiffy using this **512MB** microSD card. Preformatted to FAT32, so it works out of the packaging with our projects. Works great with any device in the Adafruit shop that uses microSD cards. Ideal for use with Feathers, data loggers, or small Linux SBCs (not good...
In Stock
[Buy Now](https://www.adafruit.com/product/5252)
[Related Guides to the Product](https://learn.adafruit.com/products/5252/guides)
Featured
### USB Type A to Type C Cable - approx 1 meter / 3 ft long
[USB Type A to Type C Cable - approx 1 meter / 3 ft long](https://www.adafruit.com/product/4474)
As technology changes and adapts, so does Adafruit. This **USB Type A to Type C** cable will help you with the transition to USB C, even if you're still totin' around a USB Type A hub, computer or laptop.
USB C is the latest industry-standard connector for...
In Stock
[Buy Now](https://www.adafruit.com/product/4474)
[Related Guides to the Product](https://learn.adafruit.com/products/4474/guides)
# Sampling Keyboard with Fruit Jam
## Install CircuitPython
[CircuitPython](https://github.com/adafruit/circuitpython) is a derivative of [MicroPython](https://micropython.org) designed to simplify experimentation and education on low-cost microcontrollers. It makes it easier than ever to get prototyping by requiring no upfront desktop software downloads. Simply copy and edit files on the **CIRCUITPY** drive to iterate.
## CircuitPython Quickstart
Follow this step-by-step to quickly get CircuitPython running on your board.
Warning: The `audioi2sin`, `audiofilewriter`, and `GranularPitchShift` modules used by this project are brand new. Running this project requires using the development release of CircuitPython **10.3.0.alpha-4** or newer until there is a 10.3.0 stable release. On the downloads page scroll down to the latest development release panel.
[Download the latest version of CircuitPython for this board via circuitpython.org](https://circuitpython.org/board/adafruit_fruit_jam/)
**Click the link above to download the latest CircuitPython UF2 file.**
Save it wherever is convenient for you.
To enter the bootloader, hold down the **BOOT/**** BOOTSEL button**(highlighted in red above), and while continuing to hold it (don't let go!), press and release the**reset button**(highlighted in red or blue above). **Continue to hold the BOOT/BOOTSEL button until the RP2350 drive appears!**
If the drive does not appear, release all the buttons, and then repeat the process above.
You can also start with your board unplugged from USB, press and hold the BOOTSEL button (highlighted in red above), continue to hold it while plugging it into USB, and wait for the drive to appear before releasing the button.
A lot of people end up using charge-only USB cables and it is very frustrating! **Make sure you have a USB cable you know is good for data sync.**
You will see a new disk drive appear called **RP2350**.
Drag the **adafruit-circuitpython-_boardname_-_language_-_version_.uf2** file to **RP2350.**
The **RP2350** drive will disappear and a new disk drive called **CIRCUITPY** will appear.
That's it, you're done! :)
## Safe Mode
You want to edit your **code.py** or modify the files on your **CIRCUITPY** drive, but find that you can't. Perhaps your board has gotten into a state where **CIRCUITPY** is read-only. You may have turned off the **CIRCUITPY** drive altogether. Whatever the reason, safe mode can help.
Safe mode in CircuitPython does not run any user code on startup, and disables auto-reload. This means a few things. First, safe mode _bypasses any code in_ **boot.py** (where you can set **CIRCUITPY** read-only or turn it off completely). Second, _it does not run the code in_ **code.py**. And finally, _it does not automatically soft-reload when data is written to the_ **CIRCUITPY** _drive_.
Therefore, whatever you may have done to put your board in a non-interactive state, safe mode gives you the opportunity to correct it without losing all of the data on the **CIRCUITPY** drive.
### Entering Safe Mode
To enter safe mode when using CircuitPython, plug in your board or hit reset (highlighted in red above). Immediately after the board starts up or resets, it waits 1000ms. On some boards, the onboard status LED (highlighted in green above) will blink yellow during that time. If you press reset during that 1000ms, the board will start up in safe mode. It can be difficult to react to the yellow LED, so you may want to think of it simply as a slow double click of the reset button. (Remember, a fast double click of reset enters the bootloader.)
### In Safe Mode
If you successfully enter safe mode on CircuitPython, the LED will intermittently blink yellow three times.
If you connect to the serial console, you'll find the following message.
```terminal
Auto-reload is off.
Running in safe mode! Not running saved code.
CircuitPython is in safe mode because you pressed the reset button during boot. Press again to exit safe mode.
Press any key to enter the REPL. Use CTRL-D to reload.
```
You can now edit the contents of the **CIRCUITPY** drive. Remember, _your code will not run until you press the reset button, or unplug and plug in your board, to get out of safe mode._
## Flash Resetting UF2
If your board ever gets into a really _weird_ state and CIRCUITPY doesn't show up as a disk drive after installing CircuitPython, try loading this 'nuke' UF2 to RP2350. which will do a 'deep clean' on your Flash Memory. **You will lose all the files on the board** , but at least you'll be able to revive it! After loading this UF2, follow the steps above to re-install CircuitPython.
[Download flash erasing "nuke" UF2 for RP2350](https://cdn-learn.adafruit.com/assets/assets/000/132/526/original/rp2350_flash_nuke.uf2)
# Sampling Keyboard with Fruit Jam
## Wiring & Assembly
Make the following connections. Refer to the wiring diagram and photos as needed.
### ICS43434 I2S Mic
- Fruit Jam **3V** to mic breakout **3V**
- Fruit Jam **GND** to mic breakout **GND**
- Fruit Jam **D6** to mic breakout bit clock **BCLK**
- Fruit Jam **D7** to mic breakout word select **LRCL**
- Fruit Jam **D9** to mic breakout data out **DOUT**
### MIDI Keyboard & Speaker
- Speaker **USB power** to the Fruit Jam **right USB port** closest to the 3.5mm jack
- Speaker **3.5mm input** to Fruit Jam **3.5mm DAC output jack**
- USB midi keyboard to Fruit Jam **left USB port**
### Micro SD Card
Insert a FAT formatted micro sd card into the built-in micro sd slot on the Fruit Jam.
# Sampling Keyboard with Fruit Jam
## Code
## Getting the Program's Files
To use the application, you need to obtain **code.py** with the program, and the other project files to place on the Fruit Jam **CIRCUITPY** drive.
Thankfully, this can be done in one go. In the example below, click the **Download Project Bundle** button below to download the necessary libraries, the **code.py** file, and other project files in a zip file.
Connect your board to your computer via a known good data+power USB cable. The board should show up in your File Explorer/Finder (depending on your operating system) as a flash drive named **CIRCUITPY**.
Extract the contents of the zip file, copy the **lib** directory files to **CIRCUITPY/lib**. Copy the **code.py** file, as well as the **anna-magdealena-20a.mid** file to your **CIRCUITPY** drive. The program should self start.
https://github.com/adafruit/Adafruit_Learning_System_Guides/blob/main/Fruit_Jam/Fruit_Jam_Sampling_Keyboard/code.py
## Drive Structure
After copying the files, your drive should look like the listing below. It can contain other files as well, but must contain these at a minimum.
# Sampling Keyboard with Fruit Jam
## Code Explanation
This page contains an overview how the code works, and what different sections are responsible for. It's written in top down order following along with **code.py**. To get the most out of it look at the contents of **code.py** side-by-side with this page.
## Configurable Variables
The first thing underneath the imports is a handful of variables that can be used to configure the behavior of the app.
- `OUTPUT_PATH` - Where the sample wave file will get saved to. **/sd/sample0.wav** by default.
- `SONG_PATH` - The MIDI file to use for automated playback. **anna-magdalena-20a.mid** by default.
- `CC_RECORD` - The CC button value to use for the record button. `77` by default
- `CC_PLAY_SONG` - The CC button value to use for the automatic MIDI file playback. `78` by default.
- `SONG_PLAYBACK_BPM` - The approximate beats per minute to play the MIDI file at. `135` by default.
## CC Debouncing
My keyboard always sends duplicate events when the CC buttons are pressed. The code ignores the duplicates by implementing a debouncing mechanism with cooldown timer. The default time to ignore duplicate events is set by `CC_DEBOUNCE_DEFAULT`. Individual CC actions can override the default by adding entries to the `CC_COOLDOWNS` dictionary. There is a helper function, `cc_debounced()`, that enforces the cooldown timers.
## USB MIDI Keyboard Setup
To find the MIDI keyboard, the code loops over all connected USB host devices and attempts to initialize them using the [adafruit\_usb\_host\_midi](https://github.com/adafruit/Adafruit_CircuitPython_USB_Host_MIDI/) library. Once found, the keyboard is hooked into the [adafruit\_midi](https://github.com/adafruit/Adafruit_CircuitPython_MIDI/) library in order to parse the MIDI events received via USB host.
## DAC Setup
The Fruit Jam's on-board TLV320 DAC is initialized. The DAC is configured to output via the 3.5mm headphone jack. The volume is set here as well, adjust the values as needed if your speakers don't have hardware volume control. The I2SOut instance is stored in the `audio` variable.
## Polyphony Setup
The [audiomixer](https://docs.circuitpython.org/en/latest/shared-bindings/audiomixer/index.html) module is used to handle polyphony. A `Mixer` object is instantiated with `voice_count=4`, meaning it can play up to four things layered at once. The mixer is played on the I2SOut `audio` variable, so all 4 voices will play from I2S DAC headphone output.
## General Vars & Sample Setup
Next are 3 variables used by different parts of the code. `effect_chains` is a list that will hold four copies of the effects chain, one per voice in the mixer. `samples` is a list that will hold the `RawSample` instances of the unmodified audio sample, there will be four copies of it, one per voice in the mixer. `recording_file` will hold the opened file reference to the wave file that is being written during recording.
`DataContext` is a class that holds 4 more variables. They are contained within a class to avoid the need to use the `global` keyword in functions that update the value of these variables.
- `data_start` - Offset to the location within the wave file of the section that contains raw audio data.
- `pcm` - A `memoryview` that will hold the raw audio data extracted from the wave file.
- `cur_voice_index` - The index of the voice within the mixer to use for playing the next sample. It will get automatically incremented whenever a sample is played. As you play more notes it will cycle between the 4 available voices within the mixer.
- `song_player` - A `SamplerMIDIPlayer` instance that will handle parsing and playing the song from the MIDI file.
The `load_samples()` function is responsible for loading the sample from the wave file on the SD card. It will populate the 4 `RawSamples` inside of the `samples` list.
## Effects Chain Setup
4 copies of the effects chain are initialized and stored in the `effects_chains` list. Each copy includes two effects:
- `amp` - An instance of `audiofilters.Distortion` with the `drive` set to `0.0`, meaning it won't add any actual distortion. The `pre_gain` argument on it is used to boost the volume of the sample. This is helpful because the microphone recordings tend to be quiet.
- `gran_pitch_shift` - An instance of `audiodelays.GranularPitchShift`. This is used to apply the change in pitch for each of the notes on the keyboard. When different keys are pressed, the `semitones` property is updated to an offset based on the key's note value.
There are 4 copies of the effects chain so that each voice in the `audiomixer` can have it's own chain to work with independently.
## Play Note & Song Functions
The `trigger_note()` function will play the note specified by the value of `note` argument. It is called when the keys on the keyboard are pressed. It's also used by the automatic MIDI file playback. It will automatically select the correct voice within the audiomixer, and increment `data_context.cur_voice_index` in preparation for the next note.
A `SamplerMIDIPlayer` class is defined that extends `adafruit_midi_parser.MIDIPlayer`. The class overrides the `on_note_on()` function to use the `trigger_note()` function. When paired with the MIDIParser `on_note_on()` will get called automatically as appropriate based on the song in the MIDI file and BPM.
The `start_song()` function initializes the `MIDIParser` and uses it along with the instance of `SamplerMIDIPlayer` to begin playing the MIDI file song. On subsequent calls, after the objects are already initialized, it will instead just stop and rewind the player back to the beginning of the song.
## Main Loop
The main loop of the code boils down to polling the USB MIDI keyboard for new events and handling any that are received. The code uses a basic [state machine](https://learn.adafruit.com/circuitpython-101-state-machines/overview) with a few different states. The current state is kept in the `STATE` variable. In the normal "playing" state, when keyboard key events are recieved it will call `trigger_note()` for the note pressed.
When CC events are recieved it will check if they are the defined `CC_RECORD`, or `CC_PLAY_SONG` values. For the `CC_PLAY_SONG` button it will call the `start_song()` function.
For `CC_RECORD` it changes the state to "recording\_prompt" and sets a NeoPixel to yellow. In the "recording\_prompt" state the standard keyboard keys have different behavior, instead of playing the sample, pressing a key will start the live recording and releasing the key stops the recording. While recording the NeoPixels changes from yellow to red. When recording stops, the NeoPixel is turned off.
# Sampling Keyboard with Fruit Jam
## Use
When the project is first loaded, there is no sample saved. It will be silent when the keys are pressed. Once you record a sample, the file gets saved to the micro SD card. From then on it will automatically load when the Fruit Jam boots up.
## Control Change Values
MIDI control change events are used to control the sample recording and automated MIDI file playback. Near the top of the **code.py** file you will find variables that configure the CC values used by the project.
```python
CC_RECORD = 77 # midi CC control value to initiate recording
CC_PLAY_SONG = 78 # midi CC control value to initiate playing a song
```
On the Akai MPK Mini 4 the values used: `77`, and `78` correspond with the red solid circle button labeled "quantize", and red circle with plus inside button labeled "automation".
Different keyboards may use different CC values for their buttons. The project code will print the `ControlChange` object for any other buttons when they are pressed. Look in the serial console to find the value of your buttons and substitute them into the code for the `CC_RECORD`, and `CC_PLAY_SONG` variables. See example CC prints and updated variables below.
```terminal
ControlChange(control=73, value=127, channel=0)
ControlChange(control=74, value=127, channel=0)
```
```auto
CC_RECORD = 73 # midi CC control value to initiate recording
CC_PLAY_SONG = 74 # midi CC control value to initiate playing a song
```
## Recording
To record first press the button assigned as `CC_RECORD`, on my keyboard its the red solid circle labeled "quantize". The first NeoPixel in the line of 5 built-in pixels will turn yellow to indicate that it is waiting to begin recording. To start recording press and hold any key on the keyboard. The recording will stop when you release the key. The NeoPixel will turn red while recording.
https://www.youtube.com/watch?v=bNttRrgJCjs
Purple: As an alternative to recording your sample live with the mic, you can copy existing wave files to the root of the micro SD card and name them **sample0.wav** to use with the keyboard.
## MIDI File Player
The project bundle comes with an example MIDI file that contains an aria by Bach. There is a variable near the top of the **code.py** file that configures the MIDI file to use for the auto playback feature.
```python
SONG_PATH = "/anna-magdalena-20a.mid"
```
To customize it, copy your desired MIDI file to the **CIRCUITPY** drive and update the filename in this variable.
To play the song press the CC button that is assigned to `CC_PLAY_SONG`, on my keyboard that's the red circle with plus sign in it, labeled "automation".
Enjoy this rendition of Bach's aria by Princess Catty Beans 😺.
https://youtu.be/6Satn0GRERg
## Going Further
If you've built this project and are interested in taking it to the next level, here are some ideas for expanding the capabilities:
- Allow multiple samples. Recorded samples are saved to **/sd/sample0.wav**. The old file is always overwritten by newly recorded samples. Update the project code to allow sample2, 3, or more to get saved alongside the old sample(s). Set up an unused CC knob or button on your keyboard to cycle between sample files.
- Apply more effects. The only things in the effects chain currently are an amp to boost the volume, and the granular pitch shift to move the sample up or down in pitch. Add additional effects before or after the pitch shift in the chain. Different effects are found in these core modules: [audiofilters](https://docs.circuitpython.org/en/latest/shared-bindings/audiofilters/index.html), [audiodelays](https://docs.circuitpython.org/en/latest/shared-bindings/audiodelays/index.html), and [audiofreeverb](https://docs.circuitpython.org/en/latest/shared-bindings/audiofreeverb/index.html). Look for the `# --- Set up effects chains ---` section header comment in the code.
- Expand the controls using additional MIDI CC knobs and buttons on your keyboard. Make a volume knob by hooking it up to change the `pre_gain` value used for the amp. Add more effects to the chain and hook up buttons to enable/disable them, or knobs to modify their `mix` property.
- Add an additional reserved voice track to the `audiomixer` and use it to play a backing track from a different wave file.
- Support multiple MIDI files. Copy more MIDI files to the **CIRCUITPY** drive and hook up different CC buttons to play each one. Or use one button to cycle thru songs and another button to play the currently selected one.
## Guide Products
### Adafruit Fruit Jam - Mini RP2350 Computer
[Adafruit Fruit Jam - Mini RP2350 Computer](https://www.adafruit.com/product/6200)
We were catching up on a recent [hackaday hackchat with eben upton](https://hackaday.io/event/202122-raspberry-pi-hack-chat-with-eben-upton) and learned some fun facts: such as the DVI hack for the RP2040 was inspired by
In Stock
[Buy Now](https://www.adafruit.com/product/6200)
[Related Guides to the Product](https://learn.adafruit.com/products/6200/guides)
### Adafruit I2S MEMS Microphone Breakout - ICS-43434
[Adafruit I2S MEMS Microphone Breakout - ICS-43434](https://www.adafruit.com/product/6049)
Listen to this good news - we now have a breakout board for a super tiny **ICS43434** **I2S** MEMS microphone. Just like 'classic' electret microphones, MEMS mics can detect sound and convert it to voltage, but they're way smaller and thinner. This...
In Stock
[Buy Now](https://www.adafruit.com/product/6049)
[Related Guides to the Product](https://learn.adafruit.com/products/6049/guides)
### USB Powered Speakers
[USB Powered Speakers](https://www.adafruit.com/product/1363)
Add some extra boom to your audio project with these powered loudspeakers. We sampled half a dozen different models to find ones with a good frequency response, so you'll get quality audio output for music playback. Simply connect the standard 3.5mm stereo plug into your Raspberry Pi model...
In Stock
[Buy Now](https://www.adafruit.com/product/1363)
[Related Guides to the Product](https://learn.adafruit.com/products/1363/guides)
### 512MB micro SD Memory Card
[512MB micro SD Memory Card](https://www.adafruit.com/product/5252)
Add storage in a jiffy using this **512MB** microSD card. Preformatted to FAT32, so it works out of the packaging with our projects. Works great with any device in the Adafruit shop that uses microSD cards. Ideal for use with Feathers, data loggers, or small Linux SBCs (not good...
In Stock
[Buy Now](https://www.adafruit.com/product/5252)
[Related Guides to the Product](https://learn.adafruit.com/products/5252/guides)
### USB Type A to Type C Cable - approx 1 meter / 3 ft long
[USB Type A to Type C Cable - approx 1 meter / 3 ft long](https://www.adafruit.com/product/4474)
As technology changes and adapts, so does Adafruit. This **USB Type A to Type C** cable will help you with the transition to USB C, even if you're still totin' around a USB Type A hub, computer or laptop.
USB C is the latest industry-standard connector for...
In Stock
[Buy Now](https://www.adafruit.com/product/4474)
[Related Guides to the Product](https://learn.adafruit.com/products/4474/guides)
## Related Guides
- [Adafruit I2S MEMS Microphone Breakout](https://learn.adafruit.com/adafruit-i2s-mems-microphone-breakout.md)
- [Adafruit Fruit Jam](https://learn.adafruit.com/adafruit-fruit-jam.md)
- [Retroreflective Greenscreen Light Ring QT Py Controller ](https://learn.adafruit.com/retroreflective-green-screen-light-ring-controller.md)
- [No-Code WipperSnapper Water Detection](https://learn.adafruit.com/water-detection-with-wippersnapper.md)
- [NeoPixel Aquarium with Submersible Lights](https://learn.adafruit.com/neopixel-aquarium-with-submersible-lights.md)
- [Cleveland Museum of Art PyPortal Frame](https://learn.adafruit.com/cleveland-museum-of-art-pyportal-frame.md)
- [ESPectre Human Detector for Feather](https://learn.adafruit.com/espectre-human-detector-for-feather.md)
- [Ahsoka Lightsaber Prop-Maker RP2040 retrofit](https://learn.adafruit.com/lightsaber-retrofit.md)
- [Fisher-Price USB Controller](https://learn.adafruit.com/fisher-price-usb-controller.md)
- [Discord and Slack Connected Smart Plant with Adafruit IO Actions](https://learn.adafruit.com/discord-and-slack-connected-smart-plant-with-adafruit-io-triggers.md)
- [MagTag IoT Menorah](https://learn.adafruit.com/magtag-iot-menorah.md)
- [Return to The Matrix with the Metro RP2350 or Fruit Jam](https://learn.adafruit.com/return-to-the-matrix-with-the-metro-rp2350.md)
- [CircuitPython on the Xteink X4 eReader](https://learn.adafruit.com/circuitpython-on-the-xteink-x4-ereader.md)
- [Larsio Paint Music](https://learn.adafruit.com/larsio-paint-music.md)
- [Lightsaber Prop-Maker RP2040](https://learn.adafruit.com/lightsaber-rp2040.md)
- [Flappy Nyan Cat Game on Fruit Jam and Metro RP2350](https://learn.adafruit.com/flappy-nyan-cat-game-on-metro-rp2350.md)
- [Minesweeper on the Fruit Jam and Metro RP2350](https://learn.adafruit.com/minesweeper-on-metro-rp2350.md)