Generators are a convenient way to make custom iterators. They take two forms: generator functions and generator expressions.

Generator Functions

These are functions that result in a generator object (basically an iterator) rather than a function object. That object can then be used in a for loop or passed to the next function. The differentiating feature of a generator function vs. a regular function is the use of the yield statement.

For example, if we wanted something like range that instead worked on floats, we could write one as a generator function:

def range_f(start, stop, step):
    x = start
    while x <= stop:
        yield x
        x += step

def range_f(start, stop, step):
    x = start
    while x <= stop:
        yield x
        x += step

Now we can write code such as:

for x in range_f(0, 1.0, 0.125):
    print(x)

for x in range_f(0, 1.0, 0.125):
    print(x)

That gives output of:

0
0.125
0.25
0.375
0.5
0.625
0.75
0.875
1.0

And we can use it with next:

r = range_f(0, 1, 0.125)
next(r)
#0
next(r)
#0.125
next(r)
#0.25
next(r)
#0.375
next(r)
#0.5

r = range_f(0, 1, 0.125)
next(r)
#0
next(r)
#0.125
next(r)
#0.25
next(r)
#0.375
next(r)
#0.5

The first time the resulting generator object is used, the function starts executing from the beginning. In this case, it creates a local variable x and sets it to the value of start.  The while loop executes, and the yield statement is encountered. This pauses execution of the function and produces its argument (the current value ofxin this case). The next time the generator object is used (iterating in a for loop or passed to a call to next), the function resumes execution immediately after the yield statement. In this case, x is advanced, and the while loop goes back to the condition check. This continues until the iteration is complete (x > stop in this example). When that happens, the function continues past the loop, possibly doing some wrap up before exiting. When it exits a StopIteration exception is raised.

next(r)
#1.0
next(r)
#Traceback (most recent call last):
#File "", line 1, in 
#StopIteration:

next(r)
#1.0
next(r)
#Traceback (most recent call last):
#File "", line 1, in 
#StopIteration:

Generator Expressions

Generator expressions use the same syntax as list comprehensions, but enclose it in parentheses rather than brackets:

(x for x in range(10))
#<generator object '<genexpr>' at 20002590>

(x for x in range(10))
#<generator object '<genexpr>' at 20002590>

Notice that the result isn't a list. It's a generator object as we saw above. To get values from it, we call next passing in the generator object:

g = (x for x in range(10))
next(g)
#0
next(g)
#1
next(g)
#2
next(g)
#3
next(g)
#4

g = (x for x in range(10))
next(g)
#0
next(g)
#1
next(g)
#2
next(g)
#3
next(g)
#4

We can also take g and use it in a for loop:

g = (x for x in range(10))
for x in g:
    print(x)
#0
#1
#2
#3
#4
#5
#6
#7
#8
#9

g = (x for x in range(10))
for x in g:
    print(x)
#0
#1
#2
#3
#4
#5
#6
#7
#8
#9

Now, think about how you would go about processing items from an infinitely long list. "But," you say, "where would be have an infinite amount of data in a microcontroller context?"  How about when you're taking temperature readings, and you continue taking temperature readings for the entire time the system is running. That is effectively an infinitely long list of data. And infinite really just means that: you have no idea how long it is.

While you could just read the temperature in the main while True loop (and that's perfectly fine for a simple application that is mainly just a while True loop) as your code grows and you want to organize it in a cleaner and more modular way (we are assuming that you're running on at least a SAMD51 chip, if not a Raspberry Pi, so you have plenty of room for large and well structured code). It can be cleaner to encapsulate that sequence of readings in an iterator. This lets it be passed around as needed and asked for another value where and when desired.

Another nice thing about using iterators is that they are now objects and can be worked with. There are ways to combine and ask for values from them in clean, readable ways.

Examples

One neat thing we can do with generators is to make infinite sequences. These don't have a condition to end the iteration. For example, we can make an infinite sequence of integers:

def ints():
    x = 0
    while True:
        yield x
        x += 1

def ints():
    x = 0
    while True:
        yield x
        x += 1

We can use this with next to get a ongoing sequence of integers, but that's not very interesting.

>>> i = ints()
>>> next(i)
0
>>> next(i)
1
>>> next(i)
2
>>> next(i)
3
>>> next(i)
4

Far more interesting is to use ints() in other generators. Here are some simple examples:

evens = (x for x in ints() if x % 2 == 0)
odds = (x for x in ints() if x % 2 != 0)
squares = (x**2 for x in ints())

evens = (x for x in ints() if x % 2 == 0)
odds = (x for x in ints() if x % 2 != 0)
squares = (x**2 for x in ints())

Note that each use of ints() returns a new and unique generator object.