Crunching the Numbers

The iPad retina screen capture contains over nine megabytes of data.

There’s 32 kilobytes of flash memory on an ATmega328P-based microcontroller board. Leaving room for playback code and bootloader, it’s more like a maximum of 26K or so for our data. There are other boards with more flash space, or could have used an SD card, but I wanted to keep it simple and use a Metro 328 board…a 40-pixel NeoPixel Shield would neatly fit atop it for a FakeTV-like form-factor…but it can work just as well with NeoPixel strip, if that’s what you’ve got.

To make the data fit, something will have to go. Several orders of magnitude…

Looking closely at the screen capture, you’ll see these aren’t simple frame averages. The app describes their use of “clusters” in the visualization. To change the clusters into averages we’ll do some image editing with Photoshop.

First, a small black band at the bottom of the screen was cropped off.

There are 54 films in the visualization, each with one horizontal band. To produce an average for each one, the image is simply resized to 54 pixels tall (keeping the same width) specifically using bilinear interpolation mode (which will average all the pixels in each band, unlike bicubic which assigns different pixel weights and will “bleed over” from adjacent rows).

Strictly for the sake of illustration here…not done to the source data…the image was re-stretched to the original size in nearest neighbor mode, so you can see the color stripes.

Scaling down the image vertically this way provides a massive 28X reduction. But the iPad screen is huge (2048 pixels across), and this still represents about 330 kilobytes of color data. A few more reductions are needed…

Culling the herd: Some color timelines aren’t as “lively” as others…they might dwell on one color for a very long time, or are just very dark overall…or too bright, or saturated, or desaturated. Three out of four timelines were discarded, keeping the 13 most visually interesting. These were selected manually.

Another 4X reduction. Now about 80K of data.

Representing each film in its entirety didn’t seem necessary…titles and end credits, for example, were usually very plain…so I cropped just a 1,000-pixel-wide section from the larger image. If we assume each film has about a 100-ish minute average running time, this represents a roughly 45-ish minute chunk from each. That’s about 2.7-ish seconds-ish per pixel-ish.

Slightly over 2X reduction, now at 39 kilobytes. Here’s the resulting image:

This is still too large to fit in the Metro’s flash space, but one more reduction is going to take place…

A Python script will process this image into an Arduino header file. Along the way, this will quantize the 24-bit color data down to 16-bits per pixel (5 bits red, 6 bits green, 5 bits blue)…a 2/3 reduction, or 26 kilobytes total.

Though we lose some color fidelity, these are the least-significant bits…and there are plenty of other factors (LED color balance, diffuse interreflection with a room’s walls and other objects) that will tinge the resulting colors anyway. The playback code will do some shenanigans that should recover some intermediary shades.

Here’s the Python script that converts the image (named crop.png) to a .h file. It requires the Python Imaging Library or Pillow:

from PIL import Image
import sys

# Output one hex byte with C formatting & line wrap ------------------------

numBytes = 0  # Total bytes to be output in table
byteNum  = 0  # Current byte number (0 to numBytes-1)
cols     = 12 # Current column number in output (force indent on first one)

def writeByte(n):
        global cols, byteNum, numBytes

        cols += 1                      # Increment column #
        if cols >= 12:                 # If max column exceeded...
                print                  # end current line
                sys.stdout.write("  ") # and start new one
                cols = 0               # Reset counter
        sys.stdout.write("{0:#0{1}X}".format(n, 4))
        byteNum += 1
        if byteNum < numBytes:
                sys.stdout.write(",")
                if cols < 11:
                        sys.stdout.write(" ")

# Mainline code ------------------------------------------------------------

# Output 8-bit gamma-correction table:
sys.stdout.write("const uint8_t PROGMEM gamma8[] = {")
numBytes = 256
for i in range(256):
        base     = 1 + (i / 3)  # LCD, CRT contrast is never pure black
        overhead = 255.0 - base
        writeByte(base + int(pow(i / 255.0, 2.7) * overhead + 0.5))
print " },"


# Output color data (2 bytes per pixel):
sys.stdout.write("colors[] = {")
image        = Image.open("crop.png")
image.pixels = image.load()
numBytes     = image.size[0] * image.size[1] * 2
byteNum      = 0
cols         = 12
for y in range(image.size[1]):
        for x in range(image.size[0]):
                r = image.pixels[x, y][0]
                g = image.pixels[x, y][1]
                b = image.pixels[x, y][2]
                # Convert 8/8/8 (24-bit) RGB to 5/6/5 (16-bit):
                writeByte((r & 0xF8) | (g >> 5))
                writeByte(((g & 0x1C) << 3) | (b >> 3))
print " };"

The output of this program is redirected to a file (e.g. data.h) and can then be #included by the Arduino code on the next page…

Last updated on May 12, 2016 Published on May 12, 2016