We've finally arrived at the portion of the code that controls the Thermal Camera's primary process. Before getting started in the main process, we need to define a couple of things to get the camera ready for looping.
After taking a performance time stamp at the beginning with the time marker variable mkr_t1, the default display mode flags and the initial ranges values are established. Next, the image_group display group is activated, a sample of the iron spectrum colors is displayed for 0.75 seconds, and a "ready" tone is sounded.
# ###--- PRIMARY PROCESS SETUP ---###
mkr_t1 = time.monotonic() # Time marker: Primary Process Setup
# pylint: disable=no-member
mem_fm1 = gc.mem_free() # Monitor free memory
DISPLAY_IMAGE = True # Image display mode; False for histogram
DISPLAY_HOLD = False # Active display mode; True to hold display
DISPLAY_FOCUS = False # Standard display range; True to focus display range
# pylint: disable=invalid-name
orig_max_range_f = 0 # Establish temporary range variables
orig_min_range_f = 0
# Activate display, show preloaded sample spectrum, and play welcome tone
display.root_group = image_group
update_image_frame()
flash_status("IRON", 0.75)
play_tone(880, 0.010) # Musical note A5
Primary Process Loop, Part I
Because of its complexity, the primary process loop is divided into two sections to make it easier to understand. The first section fetches the image sensor's data, analyzes and displays the sensor data as an image or histogram, and checks to see if any of the sensor elements have exceeded the alarm threshold. The second section looks at the buttons and joystick to select display modes and to run the Setup helper.
Retrieve Sensor Data, Display Image or Histogram, Check Alarm Threshold
At time mkr_t2, the image sensor's 64 data elements are moved into the sensor list when the DISPLAY_HOLD flag is false; otherwise a "-HOLD-" status message is displayed. To allow the sensor's temperature data to be used by the ultra fast ulab interpolation helper, the sensor list is copied into a ulab-compatible array, SENSOR_DATA. The data is also constrained ("clipped" in ulab nomenclature) to the valid temperature range of the AMG8833 sensor, 0°C to 80°C.
# ###--- PRIMARY PROCESS LOOP ---###
while True:
mkr_t2 = time.monotonic() # Time marker: Acquire Sensor Data
if DISPLAY_HOLD:
flash_status("-HOLD-", 0.25)
else:
sensor = amg8833.pixels # Get sensor_data data
# Put sensor data in array; limit to the range of 0, 80
SENSOR_DATA = np.clip(np.array(sensor), 0, 80)
Before the temperature data in the SENSOR_DATA array is altered for the interpolation process, the minimum, maximum, and average values of the array are captured in the variables v_min, v_max, and v_ave. The display is then updated with the Fahrenheit values of the current alarm setting as well as the converted minimum, maximum, and average Fahrenheit values. This section starts at time marker mkr_t4.
# Update and display alarm setting and max, min, and ave stats
mkr_t4 = time.monotonic() # Time marker: Display Statistics
v_max = np.max(SENSOR_DATA)
v_min = np.min(SENSOR_DATA)
v_ave = np.mean(SENSOR_DATA)
alarm_value.text = str(ALARM_F)
max_value.text = str(celsius_to_fahrenheit(v_max))
min_value.text = str(celsius_to_fahrenheit(v_min))
ave_value.text = str(celsius_to_fahrenheit(v_ave))
It's time to use bilinear interpolation to enlarge the sensor's 64 elements (8x8) into a grid of 225 elements (15x 15). The interpolation process commences at time marker mkr_t5 and begins by converting each of the 64 temperature values in the SENSOR_DATA array to a normalized value that ranges from 0.0 to 1.0 depending on the recorded temperature value as compared to the currently displayed temperature range. For example, a normalized value of 0.0 represents a temperature at the minimum of the currently displayed range, MIN_RANGE_C; a normalized value of 1.0 represents the maximum of the range, MAX_RANGE_C. Normalizing the temperature values makes it easier to display a full range of pseudocolors for any temperature range that the FOCUS mode may invoke.
After normalization, the known values from the SENSOR_DATA array are copied into the GRID_DATA array, starting at [0, 0], the upper left corner, and placed into the cells of the even columns. The odd rows in the GRID_DATA array are initially left blank. Once the GRID_DATA array is filled, the missing values are replaced with the results of the interpolation helper, ulab_bilinear_interpolation(). Refer to the 1-2-3s of Bilinear Interpolation section for details of the image enlargement process.
# Normalize temperature to index values and interpolate
mkr_t5 = time.monotonic() # Time marker: Normalize and Interpolate
SENSOR_DATA = (SENSOR_DATA - MIN_RANGE_C) / (MAX_RANGE_C - MIN_RANGE_C)
GRID_DATA[::2, ::2] = SENSOR_DATA # Copy sensor data to the grid array
ulab_bilinear_interpolation() # Interpolate to produce 15x15 result
This section checks the DISPLAY_IMAGE flag to see whether to display a sensor image or histogram. If DISPLAY_IMAGE is True, the update_image_frame() helper is used to display the data contained in the GRID_DATA array as a thermal image. When False, update_histo_frame() displays the data as a histogram distribution.
# Display image or histogram
mkr_t6 = time.monotonic() # Time marker: Display Image
if DISPLAY_IMAGE:
update_image_frame(selfie=SELFIE)
else:
update_histo_frame()
The next step in the primary process loop checks the returned maximum value against the current alarm threshold (ALARM_C). If the threshold is met or exceeded, the NeoPixels flash red and a warning tone is played through the speaker.
# If alarm threshold is reached, flash NeoPixels and play alarm tone
if v_max >= ALARM_C:
pixels.fill(RED)
play_tone(880, 0.015) # Musical note A5
pixels.fill(BLACK)
Primary Process Loop, Part II
The second portion of the primary process loop checks to see if any buttons have been pressed and sets the appropriate flags to select camera functions. This section also watches the SET button to activate the setup_mode() helper to permit changing camera parameters. Finally, all the time markers are analyzed and a code performance report is printed.
Watch the Buttons and Change Parameters
First the HOLD button (the PyGamer's BUTTON_A) is checked. If pressed, an acknowledgment tone is sounded and the boolean DISPLAY_HOLD parameter is toggled to the opposite state.
# See if a panel button is pressed
buttons = panel.events.get()
if buttons and buttons.pressed:
if buttons.key_number == BUTTON_HOLD:
# Toggle display hold (shutter)
play_tone(1319, 0.030) # Musical note E6
DISPLAY_HOLD = not DISPLAY_HOLD
If the IMAGE button (BUTTON_B) is pressed, a tone is played and the boolean DISPLAY_IMAGE value is toggled to the opposite state. After waiting until the button is released, the variable DISPLAY_IMAGE is checked. If True (display image), the histogram legend colors are disabled. If False (display histogram), the histogram legend colors are enabled.
if buttons.key_number == BUTTON_IMAGE:
# Toggle image/histogram mode (display image)
play_tone(659, 0.030) # Musical note E5
DISPLAY_IMAGE = not DISPLAY_IMAGE
if DISPLAY_IMAGE:
min_histo.color = None
max_histo.color = None
range_histo.color = None
else:
min_histo.color = CYAN
max_histo.color = RED
range_histo.color = BLUE
When the FOCUS button (PyGamer BUTTON_SELECT) is pressed, a tone is played and the boolean DISPLAY_FOCUS value is toggled to the opposite state.
If DISPLAY_FOCUS is True, the default display range values MIN_RANGE_F and MAX_RANGE_F are stored in temporary variables orig_min_range_f and orig_max_range_f. The display range is then updated with the current minimum and maximum values v_min and v_max. This change causes the color spectrum of the display to conform to the new range. The status "FOCUS" is then flashed on the display.
If DISPLAY_FOCUS is False, the previously stored range variables become the current display range. The display range reverts to the original default values and the colors match the original range. The display flashes the "ORIG" status message.
if buttons.key_number == BUTTON_FOCUS: # Toggle display focus mode
play_tone(698, 0.030) # Musical note F5
DISPLAY_FOCUS = not DISPLAY_FOCUS
if DISPLAY_FOCUS:
# Set range values to image min/max for focused image display
orig_min_range_f = MIN_RANGE_F
orig_max_range_f = MAX_RANGE_F
MIN_RANGE_F = celsius_to_fahrenheit(v_min)
MAX_RANGE_F = celsius_to_fahrenheit(v_max)
# Update range min and max values in Celsius
MIN_RANGE_C = v_min
MAX_RANGE_C = v_max
flash_status("FOCUS", 0.2)
else:
# Restore previous (original) range values for image display
MIN_RANGE_F = orig_min_range_f
MAX_RANGE_F = orig_max_range_f
# Update range min and max values in Celsius
MIN_RANGE_C = fahrenheit_to_celsius(MIN_RANGE_F)
MAX_RANGE_C = fahrenheit_to_celsius(MAX_RANGE_F)
flash_status("ORIG", 0.2)
When the SET button (BUTTON_START) is pressed, a tone is played. The setup_mode() helper is then executed returning new values for ALARM_F, MAX_RANGE_F, and MIN_RANGE_F that are promptly converted to Celsius.
if buttons.key_number == BUTTON_SET:
# Activate setup mode
play_tone(784, 0.030) # Musical note G5
# Invoke startup helper; update alarm and range values
ALARM_F, MAX_RANGE_F, MIN_RANGE_F = setup_mode()
ALARM_C = fahrenheit_to_celsius(ALARM_F)
MIN_RANGE_C = fahrenheit_to_celsius(MIN_RANGE_F)
MAX_RANGE_C = fahrenheit_to_celsius(MAX_RANGE_F)
Before looping to the start of the primary process loop to display the next image, the time marker mkr_t7 is used to record the time at the end of the code loop. The code performance time markers are analyzed and performance results printed to the REPL's serial output.
mkr_t7 = time.monotonic() # Time marker: End of Primary Process
gc.collect()
mem_fm7 = gc.mem_free()
# Print frame performance report
print("*** PyBadge/Gamer Performance Stats ***")
print(f" define display: {(mkr_t1 - mkr_t0):6.3f} sec")
print(f" free memory: {mem_fm1 / 1000:6.3f} Kb")
print("")
print(" rate")
print(f" 1) acquire: {(mkr_t4 - mkr_t2):6.3f} sec ", end="")
print(f"{(1 / (mkr_t4 - mkr_t2)):5.1f} /sec")
print(f" 2) stats: {(mkr_t5 - mkr_t4):6.3f} sec")
print(f" 3) convert: {(mkr_t6 - mkr_t5):6.3f} sec")
print(f" 4) display: {(mkr_t7 - mkr_t6):6.3f} sec")
print(" =======")
print(f"total frame: {(mkr_t7 - mkr_t2):6.3f} sec ", end="")
print(f"{(1 / (mkr_t7 - mkr_t2)):5.1f} /sec")
print(f" free memory: {mem_fm7 / 1000:6.3f} Kb")
print("")
Each phase of code execution is calculated from the time markers:
- define display: the time in seconds to define the displayio elements before the primary loop begins and the amount of available free memory.
- 1) acquire: elapsed time and the calculated ideal rate for acquiring and conditioning sensor data.
- 2) stats: update the on-screen alarm, min, max, and average values.
- 3) convert: normalize the 8 x 8 sensor data and enlarge the image to 15 x 15.
- 4) display: using displayio, refresh the screen image.
- total frame: the elapsed time to generate a frame (steps 1 through 4); also includes the frame-per-second rate of the just-displayed frame along with available free memory.
The PyGamer displays approximately 5 image frames per second depending on the quantity of changed display grid elements from one frame to the next. Here's a screen shot of a typical performance report:
Does printing the performance report to the REPL slow performance? Yes, but not significantly. Capturing the time markers and printing the performance report adds less than 0.005 seconds to each frame. That's a frame rate performance impact of approximately 2.9%. I think we can live with that.
Refer to the Performance Monitoring section for a further discussion of the method used and comparison of the Thermal Camera code performance on a variety of Adafruit development boards.
Text editor powered by tinymce.
