We've finally arrived at the portion of the code that controls the Thermal Camera's primary process. Before getting started in the main process, we need to define a couple of things to get the camera ready for looping.

After taking a performance time stamp at the beginning with the time marker variable t1, the default display mode flags and the initial ranges values are established. Next, the image_group display group is activated, a sample of the iron spectrum colors is displayed for 0.75 seconds, and a "ready" tone is sounded.

# ###--- PRIMARY PROCESS SETUP ---###
t1 = time.monotonic()  # Time marker: Primary Process Setup
fm1 = gc.mem_free()  # Monitor free memory
display_image = True  # Image display mode; False for histogram
display_hold = False  # Active display mode; True to hold display
display_focus = False  # Standard display range; True to focus display range
orig_max_range_f = 0  # Establish temporary range variables
orig_min_range_f = 0

# Activate display and play welcome tone
display.show(image_group)
spectrum()
update_image_frame()
flash_status("IRON", 0.75)
play_tone(880, 0.010)  # A5

Primary Process Loop, Part I

Because of its complexity, the primary process loop is divided into two sections to make it easier to understand. The first section fetches the image sensor's data, analyzes and displays the sensor data as an image or histogram, and checks to see if any of the sensor elements have exceeded the alarm threshold. The second section looks at the buttons and joystick to select display modes and to run the Setup helper.

Retrieve Sensor Data, Display Image or Histogram, Check Alarm Threshold

At time t2, the image sensor's 64 data elements are moved into the sensor list when the display_hold flag is false; otherwise a "-HOLD-" status message is displayed. To allow the sensor's temperature data to be used by the ultra fast ulab interpolation helper, the sensor list is copied into a ulab-compatible array, sensor_data.

Starting at time marker t3, the temperature value of each element of the the sensor_data array is constrained to the valid temperature range of the AMG8833 sensor, 0°C to 80°C.

# ###--- PRIMARY PROCESS LOOP ---###
while True:
    t2 = time.monotonic()  # Time marker: Acquire Sensor Data
    if display_hold:
        flash_status("-HOLD-", 0.25)
    else:
        sensor = amg8833.pixels  # Get sensor_data data
    sensor_data = ulab.numpy.array(sensor)  # Copy to narray

    t3 = time.monotonic()  # Time marker: Constrain Sensor Values
    for row in range(0, 8):
        for col in range(0, 8):
            sensor_data[col, row] = min(max(sensor_data[col, row], 0), 80)

Before the temperature data in the sensor_data array is altered for the interpolation process, the minimum, maximum, and average values of the array are captured in the variables v_min, v_max, and v_ave. The display is then updated with the Fahrenheit values of the current alarm setting as well as the converted minimum, maximum, and average Fahrenheit values. This section starts at time marker t4.

# Update and display alarm setting and max, min, and ave stats
t4 = time.monotonic()  # Time marker: Display Statistics
v_max = ulab.numpy.max(sensor_data)
v_min = ulab.numpy.min(sensor_data)
v_ave = ulab.numpy.mean(sensor_data)

alarm_value.text = str(ALARM_F)
max_value.text = str(celsius_to_fahrenheit(v_max))
min_value.text = str(celsius_to_fahrenheit(v_min))
ave_value.text = str(celsius_to_fahrenheit(v_ave))

It's time to use bilinear interpolation to enlarge the sensor's 64 elements (8x8) into a grid of 225 elements (15x 15). The interpolation process commences at time marker t5 and begins by converting each of the 64 temperature values in the sensor_data array to a normalized value that ranges from 0.0 to 1.0 depending on the recorded temperature value as compared to the currently displayed temperature range. For example, a normalized value of 0.0 represents a temperature at the minimum of the currently displayed range, MIN_RANGE_C; a normalized value of 1.0 represents the maximum of the range, MAX_RANGE_C. Normalizing the temperature values makes it easier to display a full range of pseudocolors for any temperature range that the FOCUS mode may invoke.

After normalization, the known values from the sensor_data array are copied into the grid_data array, starting at [0, 0], the upper left corner, and placed into the cells of the even columns. The odd rows in the grid_data array are initially left blank. Once the grid_data array is filled, the missing values are replaced with the results of the interpolation helper, ulab_bilinear_interpolation(). Refer to the 1-2-3s of Bilinear Interpolation section for details of the image enlargement process.

temperature___humidity_interpolation_grid_example.001.jpeg
The sensor_data array elements are copied into the grid_data array to prepare for interpolation.
# Normalize temperature to index values and interpolate
t5 = time.monotonic()  # Time marker: Normalize and Interpolate
sensor_data = (sensor_data - MIN_RANGE_C) / (MAX_RANGE_C - MIN_RANGE_C)
grid_data[::2, ::2] = sensor_data  # Copy sensor data to the grid array
ulab_bilinear_interpolation()  # Interpolate to produce 15x15 result

This section checks the display_image flag to see whether to display a sensor image or histogram. If display_image is True, the update_image_frame() helper is used to display the data contained in the grid_data array as an thermal image. When False, update_histo_frame() displays the data as a histogram distribution.

# Display image or histogram
t6 = time.monotonic()  # Time marker: Display Image
if display_image:
    update_image_frame(selfie=SELFIE)
else:
    update_histo_frame()

The next step in the primary process loop checks the returned maximum value against the current alarm threshold (ALARM_C). If the threshold is met or exceeded, the NeoPixels flash red and a warning tone is played through the speaker.

# If alarm threshold is reached, flash NeoPixels and play alarm tone
if v_max >= ALARM_C:
    pixels.fill(RED)
    play_tone(880, 0.015)  # A5
    pixels.fill(BLACK)

Primary Process Loop, Part II

The second portion of the primary process loop checks to see if any buttons have been pressed and sets the appropriate flags to select camera functions. This section also watches the SET button to activate the setup_mode() helper to permit changing camera parameters. Finally, all the time markers are analyzed and a code performance report is printed.

Watch the Buttons and Change Parameters

First the HOLD button (the PyGamer's BUTTON_A) is checked. If pressed, an acknowledgment tone is sounded and the boolean display_hold parameter is toggled to the opposite state. After display_hold is toggled, the code waits until the button is released.

# See if a panel button is pressed
buttons = panel.get_pressed()
if buttons & BUTTON_A:  # Toggle display hold (shutter)
    play_tone(1319, 0.030)  # E6
    display_hold = not display_hold

    while buttons & BUTTON_A:
        buttons = panel.get_pressed()
        time.sleep(0.1)

If the IMAGE button (BUTTON_B) is pressed, a tone is played and the boolean display_image value is toggled to the opposite state. After waiting until the button is released, the variable display_image is checked. If True (display image), the histogram legend colors are disabled. If False (display histogram), the histogram legend colors are enabled.

if buttons & BUTTON_B:  # Toggle image/histogram mode (display image)
    play_tone(659, 0.030)  # E5
    display_image = not display_image
    while buttons & BUTTON_B:
        buttons = panel.get_pressed()
        time.sleep(0.1)

    if display_image:
        min_histo.color = None
        max_histo.color = None
        range_histo.color = None
    else:
        min_histo.color = CYAN
        max_histo.color = RED
        range_histo.color = BLUE

When the FOCUS button (PyGamer BUTTON_SELECT) is pressed, a tone is played and the boolean display_focus value is toggled to the opposite state.

If display_focus is True, the default display range values MIN_RANGE_F and MAX_RANGE_F are stored in temporary variables orig_min_range_f and orig_max_range_f. The display range is then updated with the current minimum and maximum values v_min and v_max. This change causes the color spectrum of the display to conform to the new range. The status "FOCUS" is then flashed on the display.

If display_focus is False, the previously stored range variables become the current display range. The display range reverts to the original default values and the colors match the original range. The display flashes the "ORIG" status message.

Finally, the code waits until the button is released before moving on.

if buttons & BUTTON_SELECT:  # Toggle focus mode (display focus)
    play_tone(698, 0.030)  # F5
    display_focus = not display_focus
    if display_focus:
        # Set range values to image min/max for focused image display
        orig_min_range_f = MIN_RANGE_F
        orig_max_range_f = MAX_RANGE_F
        MIN_RANGE_F = celsius_to_fahrenheit(v_min)
        MAX_RANGE_F = celsius_to_fahrenheit(v_max)
        # Update range min and max values in Celsius
        MIN_RANGE_C = v_min
        MAX_RANGE_C = v_max
        flash_status("FOCUS", 0.2)
    else:
        # Restore previous (original) range values for image display
        MIN_RANGE_F = orig_min_range_f
        # Update range min and max values in Celsius
        MIN_RANGE_C = fahrenheit_to_celsius(MIN_RANGE_F)
        MAX_RANGE_C = fahrenheit_to_celsius(MAX_RANGE_F)
        flash_status("ORIG", 0.2)

    while buttons & BUTTON_SELECT:
        buttons = panel.get_pressed()
        time.sleep(0.1)

When the SET button (BUTTON_START) is pressed, a tone is played and the code waits for the button to be released. The setup_mode() helper is then executed returning new values for ALARM_F, MAX_RANGE_F, and MIN_RANGE_F that are promptly converted to Celsius. 

if buttons & BUTTON_START:  # Activate setup mode
    play_tone(784, 0.030)  # G5
    while buttons & BUTTON_START:
        buttons = panel.get_pressed()
        time.sleep(0.1)

    # Invoke startup helper; update alarm and range values
    ALARM_F, MAX_RANGE_F, MIN_RANGE_F = setup_mode()
    ALARM_C = fahrenheit_to_celsius(ALARM_F)
    MIN_RANGE_C = fahrenheit_to_celsius(MIN_RANGE_F)
    MAX_RANGE_C = fahrenheit_to_celsius(MAX_RANGE_F)

Before looping to the start of the primary process loop to display the next image, the time marker t7 is used to record the time at the end of the code loop. The code performance time markers are analyzed and performance results printed to the REPL's serial output.

t7 = time.monotonic()  # Time marker: End of Primary Process
    gc.collect()
    fm7 = gc.mem_free()
    print("*** PyBadge/Gamer Performance Stats ***")
    print(f"    define displayio:      {(t1 - t0):6.3f} sec")
    print(f"    startup free memory: {fm1/1000:6.3} Kb")
    print("")
    print(
        f" 1) data acquisition: {(t4 - t2):6.3f}      rate:  {(1 / (t4 - t2)):5.1f} /sec"
    )
    print(f" 2) display stats:    {(t5 - t4):6.3f}")
    print(f" 3) interpolate:      {(t6 - t5):6.3f}")
    print(f" 4) display image:    {(t7 - t6):6.3f}")
    print(f"                     =======")
    print(
        f"total frame:          {(t7 - t2):6.3f} sec  rate:  {(1 / (t7 - t2)):5.1f} /sec"
    )
    print(f"                           free memory: {fm7/1000:6.3} Kb")
    print("")

Each phase of code execution is calculated from the time markers:

  • define displayio: the time in seconds to define the displayio elements before the primary loop begins
  • 1) data acquisition: elapsed time and the calculated ideal rate for acquiring and conditioning sensor data
  • 2) display stats: update the on-screen alarm, min, max, and average values
  • 3) interpolate: normalize the 8 x 8 sensor data and enlarge the image to a 15 x 15
  • 4) display image: using displayio, refresh the screen image
  • total frame: the elapsed time to generate a frame (steps 1 through 4); also includes the frame-per-second rate of the just-displayed frame

The PyGamer displays approximately 4 to 5 image frames per second depending on the quantity of changed display grid elements from one frame to the next. Here's a screen shot of a typical performance report:

Does performance reporting slow performance? Yes, but not significantly. Capturing the time markers and printing the performance report adds less than 0.005 seconds to each frame. That's a frame rate performance impact of approximately 2.5%.

Refer to the Performance Monitoring section for a further discussion of the method used and comparison of the Thermal Camera code performance on a variety of Adafruit development boards.

This guide was first published on Jun 09, 2021. It was last updated on 2021-06-09 17:11:13 -0400.

This page (Primary Process) was last updated on May 15, 2022.

Text editor powered by tinymce.