38
Intermediate
Project guide
What you'll need
The PianoGlove uses a TCS34725 to sense colors, a NeoPixel LED to 'play' those colors back visually, a VS1053 Codec board to play them back aurally, and a Flora wearable microcontroller to handle all the conversion & communication between them.
Prerequisite guides:
To build your own PianoGlove, you'll need the following items:
- Flora
- VS1053 Codec Breakout (includes 3.5mm jack)
-
Lithium Ion Polymer Battery
- Flora Color Sensor
-
Flora RGB Smart Neo Pixel
-
Perma-Proto Half-sized Breadboard PCB
- Fabric glove
- Wire
- Needle & thread
- Soldering iron & solder
- Wire cutters
- Computer & USB cable to program Flora
In this tutorial, we'll be using an inexpensive black glove from a costume shop, but do feel free to experiment & improvise with any other types of gloves, wearables, or whatever inspires you.
Once you've gathered all your gear, we can move on to building the electronics …
Page last edited July 23, 2013
Text editor powered by tinymce.
Solder it
Prepare to be wired
First we'll prep the wires which run from the Flora to the Color Sensor, NeoPixel, and Codec Board.
Ribbon cable can be handy for keeping wires grouped, but any common type of hookup wire will work just fine.
Cut 7 pieces of wire a bit longer than the distance between your pointer fingertip and wrist.
Cut an additional 4 pieces of wire to about 4.5" lengths.
Strip the end of each wire and tin it with a bit of solder.
Wire Flora to Color Sensor & NeoPixel
Using the wiring diagram to the left (click to enlarge), solder the connections between the Flora and Color Sensor.Then do the same for connections to the NeoPixel.
Be sure to connect Flora D6 to NeoPixel's input pad which is labelled with an arrow pointing toward the NeoPixel LED.
Create the Codec-Perma-Proto
The Perma-Proto board is designed to mirror the connections of a half-size breadboard. This means you can easily test your Codec board wiring temporarily on a breadboard before committing to a soldered Perma-Proto.The Codec board comes with two strips of male header pins. Trim each of them to a length of 16 pins and place them in the Perma-Proto board as seen to the left.
Mount the Codec board on the header pins and solder them all in place. Then go ahead and solder the pins to the bottom side of the Perma-Proto.
Clip the excess pin lengths off the bottom side in order to avoid significant wrist discomfort later on!
Wire Codec board to Flora and audio jack
Solder the Codec board's included 3.5mm audio jack to the Perma-Proto in the remaining unused area.
While following along with the wiring diagram, use small pieces of solid core wire or breadboard jumpers to make on-board connections between the Codec board to the audio jack.
If you have a V2 version of the VS1053 breakout, connect the middle headphone jack pin to AGND instead of GBUF (all else is the same)
Solder connections between the Flora and Codec board using the shorter set of 4 wires you cut and tinned earlier.
Page last edited July 23, 2013
Text editor powered by tinymce.
Sew it
It's not necessary to wear the glove while the electronics are sewn in place, but doing so can help ensure a comfortable fit. Keep in mind, you'll need a friend with basic sewing skills and a steady hand to help out if choose to go this route.
Attach Flora
Place the Flora on the back of the glove just past the knuckle-line. Sew it in place using the unused solder pads as anchor points.
Stitch finger wiring
Wrap the NeoPixel wiring around the glove index finger so that the NeoPixel can be attached on top of the fingernail area. Secure the wires to the glove using a whip-stitch running their entire length.
Sew the NeoPixel to the back of the fingertip/fingernail area.
Wrap the Color Sensor wiring around the index finger, ensuring the sensor rests below the the face of the fingertip. Stitch the length of the wiring to the glove and sew the sensor in place.
Sew the Codec Proto
Place the Proto board at the glove's wrist area just behind the joint so that your hand can move freely. Sew it place using the any of holes on either side of the board.
Connect the battery
Turn the Flora's power switch to the off position and plug the battery into the connector opposite the USB port.To secure the battery in place during use, it can be tucked between the Flora and glove.
Optionally, you can further secure the battery by sewing its wiring to back of the glove.
Page last edited July 23, 2013
Text editor powered by tinymce.
Program & Play
Download the code
There are three code libraries which you'll need for this project. For modern Arduinos, please use the Manage Libraries feature to select the following libraries. For more on working with Arduino libraries, see this guide.
Once you have those two libraries loaded, you can download the code. From the program listing below, select Download File to get at the files. This will save a Zip file of the files to your computer. Remember the location you saved them to. After the download, open the Zip file and get the following files Flora_Pianoglove.h and Flora_Pianoglove.ino. You can move the Flora_Pianoglove directory to your Arduino Sketch directory and be ready to go. Just be sure both the ino and h files are in the same directory.
// SPDX-FileCopyrightText: 2019 Anne Barela for Adafruit Industries
//
// SPDX-License-Identifier: MIT
#include <Wire.h>
#include "Adafruit_TCS34725.h"
#include <Adafruit_NeoPixel.h>
#include "Flora_Pianoglove.h"
// we only play a note when the clear response is higher than a certain number
#define CLEARTHRESHHOLD 2000
#define LOWTONE 1000
#define HIGHTONE 2000
// high C
#define LOWKEY 64
// double high C
#define HIGHKEY 76
// our RGB -> eye-recognized gamma color
byte gammatable[256];
// color sensor
Adafruit_TCS34725 tcs = Adafruit_TCS34725(TCS34725_INTEGRATIONTIME_50MS, TCS34725_GAIN_4X);
// one pixel on pin 6
Adafruit_NeoPixel strip = Adafruit_NeoPixel(1, 6, NEO_GRB + NEO_KHZ800);
void setup() {
Serial.begin(9600);
Serial.println("Color Piano!");
if (tcs.begin()) {
Serial.println("Found sensor");
} else {
Serial.println("No TCS34725 found ... check your connections");
while (1); // halt!
}
strip.begin();
strip.show(); // Initialize all pixels to 'off'
// thanks PhilB for this gamma table!
// it helps convert RGB colors to what humans see
for (int i=0; i<256; i++) {
float x = i;
x /= 255;
x = pow(x, 2.5);
x *= 255;
gammatable[i] = x;
//Serial.println(gammatable[i]);
}
// tone output on OC1A / D9
pinMode(9, OUTPUT);
// toggle output
TCCR1A = _BV(COM1A1);
// PWM phase and freq correct
TCCR1B = _BV(WGM13) | _BV(CS11); // div by 8
setFreq(880);
}
void setFreq(uint16_t f) {
if (f == 0) {
TCCR1B = 0; // turn off
} else {
TCCR1B = _BV(WGM13) | _BV(CS11); // div by 8
}
uint32_t i = F_CPU / 16; // div by 8 and 2
i /= f;
ICR1 = i;
OCR1A = i/2;
}
void loop() {
uint16_t clear, red, green, blue;
tcs.setInterrupt(false); // turn on LED
delay(60); // takes 50ms to read
tcs.getRawData(&red, &green, &blue, &clear);
tcs.setInterrupt(true); // turn off LED
// not close enough to colorful item
if (clear < CLEARTHRESHHOLD) {
setFreq(0);
strip.setPixelColor(0, strip.Color(0, 0, 0)); // turn off the LED
strip.show();
return;
}
Serial.print("C:\t"); Serial.print(clear);
Serial.print("\tR:\t"); Serial.print(red);
Serial.print("\tG:\t"); Serial.print(green);
Serial.print("\tB:\t"); Serial.print(blue);
// Figure out some basic hex code for visualization
uint32_t sum = red;
sum += green;
sum += blue;
sum = clear;
float r, g, b;
r = red; r /= sum;
g = green; g /= sum;
b = blue; b /= sum;
r *= 256; g *= 256; b *= 256;
if (r > 255) r = 255;
if (g > 255) g = 255;
if (b > 255) b = 255;
Serial.print("\t");
Serial.print((int)r, HEX); Serial.print((int)g, HEX); Serial.print((int)b, HEX);
Serial.println();
// OK we have to find the two primary colors
// check if blue is smallest. MEME: fix for 'white'
float remove, normalize;
if ((b < g) && (b < r)) {
remove = b;
normalize = max(r-b, g-b);
} else if ((g < b) && (g < r)) {
remove = g;
normalize = max(r-g, b-g);
} else {
remove = r;
normalize = max(b-r, g-r);
}
// get rid of minority report
float rednorm = r - remove;
float greennorm = g - remove;
float bluenorm = b - remove;
// now normalize for the highest number
rednorm /= normalize;
greennorm /= normalize;
bluenorm /= normalize;
Serial.println();
strip.setPixelColor(0, strip.Color(gammatable[(int)r], gammatable[(int)g], gammatable[(int)b]));
strip.show();
Serial.print(rednorm); Serial.print(", ");
Serial.print(greennorm); Serial.print(", ");
Serial.print(bluenorm); Serial.print(" ");
Serial.println();
float rainbowtone = 0;
if (bluenorm <= 0.1) {
// between red and green
if (rednorm >= 0.99) {
// between red and yellow
rainbowtone = 0 + 0.2 * greennorm;
} else {
// between yellow and green
rainbowtone = 0.2 + 0.2 * (1.0 - rednorm);
}
} else if (rednorm <= 0.1) {
// between green and blue
if (greennorm >= 0.99) {
// between green and teal
rainbowtone = 0.4 + 0.2 * bluenorm;
} else {
// between teal and blue
rainbowtone = 0.6 + 0.2 * (1.0 - greennorm);
}
} else {
// between blue and violet
if (bluenorm >= 0.99) {
// between blue and violet
rainbowtone = 0.8 + 0.2 * rednorm;
} else {
// between teal and blue
rainbowtone = 0;
}
}
Serial.print("Scalar "); Serial.println(rainbowtone);
float keynum = LOWKEY + (HIGHKEY - LOWKEY) * rainbowtone;
Serial.print("Key #"); Serial.println(keynum);
float freq = pow(2, (keynum - 49) / 12.0) * 440;
Serial.print("Freq = "); Serial.println(freq);
//Serial.print((int)r ); Serial.print(" "); Serial.print((int)g);Serial.print(" "); Serial.println((int)b );
setFreq(freq);
}
RgbColor HsvToRgb(HsvColor hsv)
{
RgbColor rgb;
unsigned char region, remainder, p, q, t;
if (hsv.s == 0)
{
rgb.r = hsv.v;
rgb.g = hsv.v;
rgb.b = hsv.v;
return rgb;
}
region = hsv.h / 43;
remainder = (hsv.h - (region * 43)) * 6;
p = (hsv.v * (255 - hsv.s)) >> 8;
q = (hsv.v * (255 - ((hsv.s * remainder) >> 8))) >> 8;
t = (hsv.v * (255 - ((hsv.s * (255 - remainder)) >> 8))) >> 8;
switch (region)
{
case 0:
rgb.r = hsv.v; rgb.g = t; rgb.b = p;
break;
case 1:
rgb.r = q; rgb.g = hsv.v; rgb.b = p;
break;
case 2:
rgb.r = p; rgb.g = hsv.v; rgb.b = t;
break;
case 3:
rgb.r = p; rgb.g = q; rgb.b = hsv.v;
break;
case 4:
rgb.r = t; rgb.g = p; rgb.b = hsv.v;
break;
default:
rgb.r = hsv.v; rgb.g = p; rgb.b = q;
break;
}
return rgb;
}
HsvColor RgbToHsv(RgbColor rgb)
{
HsvColor hsv;
unsigned char rgbMin, rgbMax;
rgbMin = rgb.r < rgb.g ? (rgb.r < rgb.b ? rgb.r : rgb.b) : (rgb.g < rgb.b ? rgb.g : rgb.b);
rgbMax = rgb.r > rgb.g ? (rgb.r > rgb.b ? rgb.r : rgb.b) : (rgb.g > rgb.b ? rgb.g : rgb.b);
hsv.v = rgbMax;
if (hsv.v == 0)
{
hsv.h = 0;
hsv.s = 0;
return hsv;
}
hsv.s = 255 * long(rgbMax - rgbMin) / hsv.v;
if (hsv.s == 0)
{
hsv.h = 0;
return hsv;
}
if (rgbMax == rgb.r)
hsv.h = 0 + 43 * (rgb.g - rgb.b) / (rgbMax - rgbMin);
else if (rgbMax == rgb.g)
hsv.h = 85 + 43 * (rgb.b - rgb.r) / (rgbMax - rgbMin);
else
hsv.h = 171 + 43 * (rgb.r - rgb.g) / (rgbMax - rgbMin);
return hsv;
}
// SPDX-FileCopyrightText: 2019 Anne Barela for Adafruit Industries
//
// SPDX-License-Identifier: MIT
typedef struct RgbColor
{
unsigned char r;
unsigned char g;
unsigned char b;
} RgbColor;
typedef struct HsvColor
{
unsigned char h;
unsigned char s;
unsigned char v;
} HsvColor;
Upload the sketch
For more information on programming your Flora board including the software you need to do so, head over to the Getting Started with Flora guide.
Download & open the PianoGlove MIDI sketch in the Adafruit-Arduino IDE
Connect Flora via USB to your computer
Set Adafruit-Arduino IDE's board setting to "Flora" & upload the sketch.
Once the sketch has been uploaded to the Flora, wait for the onboard red LED to stop flashing, then disconnect the USB cable.
Use Flora's power switch to turn on battery power. After a short load time, the Flora's red LED should go dark and Color Sensor's white LED should now be on.
Connect a powered speaker or headphones to the 3.5mm audio jack and move your new color-sensing index finger close to something colorful to play a musical note through the audio jack.
Try moving the sensor quickly between different bright/dark colors, or 'gate' a note by quickly pulling the sensor away from an object. Experiment with … well, anything that has color!
Page last edited July 23, 2013
Text editor powered by tinymce.
