The Pi is only used to acquire the video. For processing, we download the video file to a PC and use full Python. This provides plenty of processing power and opens up the entire Python library ecosystem for use.

General Approach

The original research paper processed the video in a reasonably sophisticated way. For each frame, blobs were identified along with their general size using a feature detection algorithm. This provided information about both blob count and size, which allowed for comparing histograms.

For our processing, we are taking a much simpler approach. Consider a case of zero blobs. That should result in a video with all blank frames - nothing ever entered the laser light sheet. Total darkness. At the other extreme, tons of blobs would be frames full of light, tons of green laser light. So we can think in terms of "the more lit pixels in frame, the more blobs there are". We can compute a value for this for each frame as follows.

A single video frame showing blobs lit green in the laser sheet.

The same frame converted to gray scale.

The frame after applying a threshold. Only pixels with a value above the threshold remain.

Now the remaining pixels are counted and a percentage of "lit" pixels is determined. For example:

frame_count = np.count_nonzero(frame == True)
frame_percent = 100 * frame_count / (1920*1080)

These values are computed and saved for each frame. A final overall average can then be computed as well.

Required Python Libraries

We used these Python libraries to process the video data. They are all hosted on PyPi and can be installed using pip.

  • imageio - to load MP4 video files and extract frames
  • scikit-image - for image frame conversion and thresholding
  • numpy - for counting pixels
  • matplotlib - for producing plots

Processing Video

The following Python script was used to process MP4 videos taken for this experiment.

# SPDX-FileCopyrightText: 2020 Carter Nelson for Adafruit Industries
#
# SPDX-License-Identifier: MIT

import time
import imageio
from skimage.color import rgb2gray
import matplotlib.pyplot as plt
import numpy as np

THRESH = 0.3

RUN = int(input('Enter run number: '))

vid = imageio.get_reader('run_{:03d}.mp4'.format(RUN), 'ffmpeg')

#----------------
# MAIN PROCESSING
#----------------
frame_data = []
start = time.monotonic()
# go through video frame by frame
print("Processing", end='')
for frame in vid:
    print('.', end='', flush=True)
    frame_bin = rgb2gray(frame) > THRESH
    frame_count = np.count_nonzero(frame_bin == True)
    frame_percent = 100 * frame_count / (1920*1080)
    frame_data.append((frame_count, frame_percent))
# overall stats
avg_count = sum([x[0] for x in frame_data]) / len(frame_data)
avg_percent = 100 * avg_count / (1920*1080)

end = time.monotonic()
print("\nProcessing done in {} secs.".format(end - start))
print("Average Count = {}".format(avg_count))
print("Average Percent = {}".format(avg_percent))

#-------------
# SAVE TO FILE
#-------------
print("Saving data to file...")
with open('run_{:03d}.csv'.format(RUN), 'w') as fp:
    for frame, data in enumerate(frame_data):
        fp.write('{},{},{}\n'.format(frame, data[0], data[1]))

#---------
# PLOTTING
#---------
print("Generating plots...")
fig, ax = plt.subplots(1, figsize = (10,5))
ax.set_title("RUN {:03d}\nTHRESH = {}, AVG_CNT = {:4.2}, AVG_PER = {:.3}".format(RUN, THRESH,avg_count, avg_percent))
ax.set_xlabel("FRAME")
ax.set_ylabel("COUNT")
ax.plot([x[0] for x in frame_data])
fig.savefig('run_{:03d}_plot.png'.format(RUN))

print("DONE.")

This guide was first published on Oct 14, 2020. It was last updated on Dec 11, 2023.

This page (Processing Data) was last updated on Dec 11, 2023.

Text editor powered by tinymce.