With just 64 display elements, the thermal camera can only display blocky object shapes. It's surprising how visual recognition improves when a bilinear interpolation technique is applied to the thermal sensor data to increase the resolution of the image.

Interpolation is a technique for enhancing limited data sets by estimating "in-between" values. It's often used for enlarging images to make patterns and objects easier to discern. Two varieties of interpolation are commonly used for images, bilinear and bicubic. The bilinear method, based on a linear equation like y = mx + b, is the simplest and the least computationally intensive. By comparison, bicubic interpolation is computationally more complicated, usually involving a polynomial function of the second degree or higher such as the quadratic form y = ax2 + bx + c. The bicubic method can produce enlarged images with smoother and clearer object edges than the bilinear method -- but at the price of increased computational power and elapsed processing time.

Given the computational power of the PyGamer's SAMD-51 processor, the simpler bilinear approach was chosen to enlarge the thermal camera's image. The AMG8833 sensor's image is enlarged from 8 x 8 (64 elements) to a display grid of 15 x 15 (225 cells). Let's talk about how that is done. For the sake of simplicity, the following conceptual example of the method is limited to a 4 x 4 sensor array (16 elements)  and 7 x 7 display grid (49 cells).

The first step in the bilinear interpolation process is to copy the contents of the sensor value array into an image grid array of (2n - 1) rows and (2n - 1) columns. In this example n = 4, so the display grid array will have 7 rows and columns.

The first row of the display grid array contains the contents of the first row of the sensor value array with a blank element between each known sensor value. A row is skipped and the next row of sensor data is copied into the display grid array. After the sensor data is placed in the display grid array, the interpolation will replace the unknown cells with a calculated value from the closest known cells using a two-pass process.

The first pass starts with the first unknown cell and calculates its value from the preceding and following known cells in that row. For example, the cell between columns 0 and 1 of row 0 is calculated using an average of the two adjacent cells. The unknown cell is updated with the value of 3. The process continues to calculate the remaining unknown cells in the evenly numbered rows. The missing values in the odd numbered rows will be calculated next.

The second pass starts by processing the unknown cells of row 1, calculating the value from the average of the cells directly above and below. For example, the first cell of row 1 is updated with the value 3. The second pass continues to update the remaining unknown cells in the display grid array.

Because an unknown cell is flanked by known cells, its calculated value is the average of the known cells -- the unknown cell is halfway between the known cells. If the target image is enlarged to create two unknown cells between known cells, then the value of each unknown cell is calculated based on its proportional display distance from each of the two known cells.

Here's the finished product, colored to roughly represent each cell's value. Compared to the original 4 x 4 sensor image, the newly interpolated 7 x 7 image has added detail with color gradients that can help to identify the object in the field of view.

This guide was first published on Jun 09, 2021. It was last updated on 2021-06-09 17:11:13 -0400.

This page (1-2-3s of Bilinear Interpolation) was last updated on Jun 29, 2022.