Assembling Data

Let's say you are logging readings from a temperature sensor (the Si7021 for example). For some reason you need a reading number and a timestamp in addition to the reading, rounded to an integer.

You could do this the standard way:

import time
import board
import busio
import adafruit_si7021

i2c = busio.I2C(board.SCL, board.SDA)
sensor = adafruit_si7021.SI7021(i2c)

def read_temperature():
    return sensor.temperature

def now():
    return time.monotonic()

def non_iter():
    reading_number = 1
    while True:
        datapoint = (count, now(), int(read_temperature()))
        print(datapoint)
        count += 1
        time.sleep(20.0)

import time
import board
import busio
import adafruit_si7021

i2c = busio.I2C(board.SCL, board.SDA)
sensor = adafruit_si7021.SI7021(i2c)

def read_temperature():
    return sensor.temperature

def now():
    return time.monotonic()

def non_iter():
    reading_number = 1
    while True:
        datapoint = (count, now(), int(read_temperature()))
        print(datapoint)
        count += 1
        time.sleep(20.0)

Using iterators we can do this in a slightly different way:

import time
import board
import busio
import adafruit_si7021
from adafruit_itertools import count
from adafruit_itertools_extras import repeatfunc, roundrobin

i2c = busio.I2C(board.SCL, board.SDA)
sensor = adafruit_si7021.SI7021(i2c)

def read_temperature():
    return sensor.temperature

def now():
    return time.monotonic()

def iter():
    datapoints = zip(count(1), repeatfunc(now), map(int, repeatfunc(read_temperature)))
    while True:
        print(next(datapoints))
        time.sleep(20.0)

import time
import board
import busio
import adafruit_si7021
from adafruit_itertools import count
from adafruit_itertools_extras import repeatfunc, roundrobin

i2c = busio.I2C(board.SCL, board.SDA)
sensor = adafruit_si7021.SI7021(i2c)

def read_temperature():
    return sensor.temperature

def now():
    return time.monotonic()

def iter():
    datapoints = zip(count(1), repeatfunc(now), map(int, repeatfunc(read_temperature)))
    while True:
        print(next(datapoints))
        time.sleep(20.0)

The iterator based version is a couple lines shorter, but what's the advantage beyond that?

In the non-iterator version the data point is being constructed in the loop each time. In the iterator-based version, an iterator is constructed that will generate data tuples as requested. More importantly tuple generation is encapsulated in an iterator. That means it can be stored in a variable (as it is in this example, in datapoints), but also passed as an argument. This last thing is significant in that it lets you separate (decouple in programming lingo) the generate of data, from the use of it. The code in the loop doesn't care how to make a datapoint, it just asks for the next one. The iterator doesn't care how or when it's elements are being used, it just hands back the next one when asked. It doesn't even really know or care how the elements are being made. there's code that puts together the iterator, and other code that asks for elements. All the iterator does is construct a single element based on how it's been wired up. As such it serves as a great intermediary between the code that knows how to get/make the data and the code that knows how to use it.

Filtering by Divisibility

Here's an example from SICP*: let's say for some reason you need numbers that are not divisible by 7. 

First we can make a simple predicate to determine divisibility:

def divisible(x, y):
    return (x % y) == 0

To get numbers that are not divisible by 7 we can use filterfalse with a lambda:

>>> list(filterfalse(lambda x: divisible(x, 7), range(30)))
[1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 29]

In practice we probably don't want to pass in a range, but rather an infinite sequence of integers:

>>> take(20, filterfalse(lambda x: divisible(x, 7), count()))
[1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 22, 23]

The next step would be to package this up to pass to whatever function wants a sequence of integers that are not divisible by 7:

>>> no_sevens = filterfalse(lambda x: divisible(x, 7), count())
>>> take(20, no_sevens)
[1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 22, 23]

We can generalize by using a function that returns a lambda to construct the test:

>>> def divisibility_checker(x):
...     return lambda n: (n % x) == 0
>>> by7 = divisibility_checker(7)
>>> no_sevens = filterfalse(by7, count())
>>> take(20, no_sevens)
[1, 2, 3, 4, 5, 6, 8, 9, 10, 11, 12, 13, 15, 16, 17, 18, 19, 20, 22, 23]

Once more, this lets us separate the construction of the sequence from the use of it.

Using iterators also let you directly make requests like:

• Give me the next 5 items.
• Give me the 10th item.
• Skip 10 items and give me 5.
• Ignore items until one is over 100.

And so on.  All from the same iterator. All the sequencing, traversal, and computation is bundled up in the iterator and all your code has to do is ask for items from it. This can help make your code simpler and clearer.

^{* The Structure and Implementation of Computer Programs ... one book every programmer should read and study.}