This guide describes how to do cooperative multitasking in CircuitPython, using the
asyncio library and the
await language keywords. The
asyncio library is included with CPython, the host-computer version of Python. MicroPython also supplies a version of
asyncio, and that version has been adapted for use in CircuitPython.
Hey, why aren't you supporting preemptive hardware interrupts (irq)?
We looked at how MicroPython supports hardware interrupts and decided that the restrictions that are imposed make it harder to use and more prone to error than providing a better and more complete
asyncio experience. Preemptive interrupts can come in at any time, which is hard to control in an interpreted language. In MicroPython, memory cannot be allocated in an interrupt handler, and there's a lot of things that Python does that allocate memory. And since there's a garbage collector, it isn't like interrupt latency can be promised.
Instead, we think
asyncio + a "background task" that tracks GPIO change/fall/rise can be used to capture interrupts for when we are ready to process them. We have two native modules,
keypad, that can track your pin state changes in the background.
Also, we really, really want to keep CircuitPython code a true subset of CPython code so that examples can run on boards like Feather M4's or CircuitPlaygrounds as well as Raspberry Pi and desktop Python computers.
Hey, why aren't you supporting threads?
Python developers have dabbled with threads and pretty quickly determined that they are not a good way to have multiple tasks that can co-operate with shared memory. We believe that
asyncio is a better way to have concurrent tasks that can share the same memory space safely - and without having to learn about and debug concurrent processes, something that is so notoriously hard that CS students have to take a course on how to do safely (or at least, they should if they don't!).
The vast majority of microcontrollers that are supported have a single core, and those that have dual core, like Espressif chipsets, would probably benefit from pinning certain background tasks like WiFi handling to a separate core, rather than trying to balance processing manually.
Cooperative multitasking is a style of programming in which multiple tasks take turns running. Each task runs until it needs to wait for something, or until it decides it has run for long enough and should let another task run.
It's up to each task to decide when to yield control to other tasks, which is why it's cooperative. A task can freeze out other tasks, if it's not well behaved. This contrasts with preemptive multitasking, where tasks are interrupted without their knowledge to let other tasks run. Threads and processes are examples of preemptive multitasking.
Cooperative multitasking does not imply parallelism, where two tasks run literally simultaneously. The tasks do run concurrently. Their executions are interleaved: more than one can be active at a time.
In cooperative multitasking, a scheduler manages the tasks. Only one task runs at a time. When a task gives up control and starts waiting, the scheduler starts another task that is ready to run. The scheduler is fair and gives all tasks that are ready the chance to run. The scheduler runs an event loop which repeats this process over and over for all the tasks assigned to the event loop. You are already familiar with event loops, but you might not know the term. The
while True loop that is the main part of nearly all CircuitPython programs often serves as an event loop, monitoring for button presses, or simply running some code periodically on a schedule. The
loop() routine, a required part of every Arduino program, is also meant to be used as an event loop.
A task can wait for the completion of a sleep period, a network or file I/O operation, a timeout. It can also wait for some external asynchronous event such as a pin changing state.
The diagram below shows the scheduler, running an event loop, with three tasks: Task 1 is running, Task 2 is ready to run, and is waiting for Task 1 to give up control, and Task 3 is waiting for something else, and isn't ready to run yet.
A task is a kind of coroutine. A coroutine can stop in the middle of some code and return back to its caller. When the coroutine is called again, it starts where it left off.
You may be familiar with generators in Python, which are also a kind of coroutine. You can recognize a generator because it includes one or more
yield statements in it. When a coroutine gets to a
yield statement, it returns to its caller, optionally passing back a value. When the coroutine is called again, it starts at the next statement after the
yield. Early cooperative multitasking systems in Python took advantage of the generators mechanism, and used
yield to indicate when a task was giving up control.
Later, Python added the
await keywords specifically to support cooperative multitasking. A coroutine is declared with the keyword
async, and the keyword
await indicates that the coroutine is giving up control at that point.
The diagram below shows two coroutines,
g(), which are used as tasks. When Task 1 starts, it runs until it reaches an
await statement where it needs wait for something. It gives up control at that point. The scheduler then looks for another task to run, and chooses Task 2. Task 2 runs until it reaches its own
await. By that time, let's assume that whatever Task 1 was waiting for has happened. The scheduler sees that Task 1 is ready to run again, and starts up
f() where it left off.
This guide will explain
await, and their use with the
asyncio library in more detail, in some simple examples, starting on the next page.
Most of the examples in this guide require the CircuitPython version of the
asyncio library. The library is not built in to CircuitPython; you need to copy it onto CIRCUITPY to use it. The
asyncio library is available in the CircuitPython Library bundle, and is available on GitHub as well. You can also use the circup tool to get the library and keep it up to date. If you find a problem with the library that you think is a bug, please file an issue.
await (and therefore
asyncio) are available on most CircuitPython boards. SAMD21 ("M0") boards, such as the Trinket M0, Metro M0, or Feather M0 boards do not have enough flash or RAM. A few nRF boards with internal flash only also do not have enough flash.
asyncio and related libraries and modules will be enhanced over time. For details, check out the issue list in Adafruit's asyncio library.