CircuitPython Datalogging

Datalogging Example

Another handy task with GPS is logging all the raw output of the GPS module to a file.  This is useful if you're importing the GPS data into a tool like Google Earth which can process raw NMEA sentences.  You can perform this datalogging very easily with CircuitPython.

To store data you'll need to choose one of two options:

  • Wire up a SD card holder to your board's SPI bus, or use a board with SD card holder built-in like the Feather M0 Adalogger.  This is the recommended approach as it gives you a lot of space to store data and you can easily copy the data to your computer from the card.
  • Store data in your board's internal filesystem.  This requires a little more setup but allows you to save to a file on the internal filesystem of your CircuitPython board, right next to where code and other data files live.  This is more limited because depending on your board you might only have a few kilobytes or megabytes of space available and GPS sentences will quickly add up (easily filling multiple megabytes within a few hours of logging).

Install SD Card Library

If you're storing data on a SD card you must ensure the SD card is wired to your board and you have installed the Adafruit SD card library.  Luckily there's an entire guide to follow to learn about this process of connecting a SD card and installing the necessary library.  Be sure to carefully follow the guide so the card is connected, library installed, and you can confirm you're able to manually write data to the card from the Python prompt.

Enable Internal Filesystem Writes

If you're storing data on the internal filesystem you must carefully follow the steps in the CPU temperature logging guide to enable writing to internal storage.  If you're writing to a SD card skip these steps and move on to look at the datalogging code below.  Edit the boot.py on your board (creating it if it doesn't exist) and add these lines:

Download: file
import digitalio
import board
import storage
 
switch = digitalio.DigitalInOut(board.D5)
switch.direction = digitalio.Direction.INPUT
switch.pull = digitalio.Pull.UP
 
# If the D5 is connected to ground with a wire
# you can edit files over the USB drive again.
storage.remount("/", not switch.value)

Remember once you remount("/") you cannot edit code over the USB drive anymore!  That means you can't edit boot.py which is a bit of a conundrum. So we configure the boot.py to selectively mount the internal filesystem as writable based on a switch or even just alligator clip connected to ground.  Like the CPU temperature guide shows . In this example we're using D5 but select any available pin.

This code will look at the D5 digital input when the board starts up and if it's connected to ground (use an alligator clip or wire, for example, to connect from D5 to board ground) it will disable internal filesystem writes and allow you to edit code over the USB drive as normal.  Remove the alligator clip, reset the board, and the boot.py will switch to mounting the internal filesystem as writable so you can log images to it again (but not write any code!). 

Remember when you enable USB drive writes (by connecting D5 to ground at startup) you cannot write files to the internal filesystem and any code in your main.py that attempts to do so (like the example below) will fail.  Keep this in mind as you edit code--once you modify code you need to remove the alligator clip, reset the board to re-enable internal filesystem writes, and then watch the output of your program.

If you ever get stuck, you can follow the steps mentioned in https://learn.adafruit.com/cpu-temperature-logging-with-circuit-python/writing-to-the-filesystem to remove boot.py from the REPL if you need to go back and edit code!

Datalogging Example Code

The GPS library examples have a datalogging.py file you can edit and save as a main.py on your board:

# Simple GPS datalogging demonstration.
# This actually doesn't even use the GPS library and instead just reads raw
# NMEA sentences from the GPS unit and dumps them to a file on an SD card
# (recommended) or internal storage (be careful as only a few kilobytes to
# megabytes are available).  Before writing to internal storage you MUST
# carefully follow the steps in this guide to enable writes to the internal
# filesystem:
#  https://learn.adafruit.com/adafruit-ultimate-gps-featherwing/circuitpython-library
import board
import busio


# Path to the file to log GPS data.  By default this will be appended to
# which means new lines are added at the end and all old data is kept.
# Change this path to point at internal storage (like '/gps.txt') or SD
# card mounted storage ('/sd/gps.txt') as desired.
LOG_FILE = '/gps.txt'  # Example for writing to internal path /gps.txt
#LOG_FILE = '/sd/gps.txt'     # Example for writing to SD card path /sd/gps.txt

# File more for opening the log file.  Mode 'ab' means append or add new lines
# to the end of the file rather than erasing it and starting over.  If you'd
# like to erase the file and start clean each time use the value 'wb' instead.
LOG_MODE = 'ab'

# Define RX and TX pins for the board's serial port connected to the GPS.
# These are the defaults you should use for the GPS FeatherWing.
# For other boards set RX = GPS module TX, and TX = GPS module RX pins.
RX = board.RX
TX = board.TX

# If writing to SD card customize and uncomment these lines to import the
# necessary library and initialize the SD card:
#SD_CS_PIN = board.SD_CS  # CS for SD card (SD_CS is for Feather Adalogger)
#import adafruit_sdcard
#spi = busio.SPI(board.SCK, MOSI=board.MOSI, MISO=board.MISO)
#sd_cs = digitalio.DigitalInOut(SD_CS_PIN)
#sdcard = adafruit_sdcard.SDCard(spi, sd_cs)
#vfs = storage.VfsFat(sdcard)
#storage.mount(vfs, '/sd')  # Mount SD card under '/sd' path in filesystem.

# Create a serial connection for the GPS connection using default speed and
# a slightly higher timeout (GPS modules typically update once a second).
uart = busio.UART(TX, RX, baudrate=9600, timeout=3000)

# Main loop just reads data from the GPS module and writes it back out to
# the output file while also printing to serial output.
with open(LOG_FILE, LOG_MODE) as outfile:
    while True:
        sentence = uart.readline()
        print(str(sentence, 'ascii').strip())
        outfile.write(sentence)
        outfile.flush()

By default this example expects to log GPS NMEA sentences to a file on the internal storage system at /gps.txt.  New sentences will be appended to the end of the file every time the example starts running.

If you'd like to instead write to the SD card take note to uncomment the appropriate lines mentioned in the comments:

Download: file
# Path to the file to log GPS data.  By default this will be appended to
# which means new lines are added at the end and all old data is kept.
# Change this path to point at internal storage (like '/gps.txt') or SD
# card mounted storage ('/sd/gps.txt') as desired.
#LOG_FILE = '/gps.txt'  # Example for writing to internal path /gps.txt
LOG_FILE = '/sd/gps.txt'     # Example for writing to SD card path /sd/gps.txt

And further below:

Should all be uncommented and look as above.  This will configure the code to write GPS NMEA data to the /sd/gps.txt file, appending new data to the end of the file.

Once the example is running as a main.py on your board open the serial REPL and you should see the raw NMEA sentences printed out:

Check the gps.txt file (either under the root or /sd path depending on how you setup the example) in a text editor and you'll see the same raw NMEA sentences:

Awesome!  That's all there is to basic datalogging of NMEA sentences with a GPS module and CircuitPython!

This guide was first published on Aug 23, 2012. It was last updated on Aug 23, 2012. This page (CircuitPython Datalogging) was last updated on Jan 28, 2019.