This guide is under development. Its contents will evolve. There are missing sections. Your comments are welcome: bring them up in Discord, in the Forums, or via the Feedback link in the sidebar (note that Feedback is anonymous, so we can't respond to those comments).

This guide describes how to do cooperative multitasking in CircuitPython, using the asyncio library and the async and await language keywords. The asyncio library is included with CPython, the host-computer version of Python. MicroPython also supplies a version of asyncio, and that version has been adapted for use in CircuitPython.

You'll need CircuitPython 7.1.0-beta.0 or later to use asyncio.

FAQ

Hey, why aren't you supporting preemptive hardware interrupts (irq)?

We looked at how MicroPython supports hardware interrupts and decided that the restrictions that are imposed make it harder to use and more prone to error than providing a better and more complete asyncio experience. Preemptive interrupts can come in at any time, which is hard to control in an interpreted language. In MicroPython, memory cannot be allocated in an interrupt handler, and there's a lot of things that Python does that allocate memory. And since there's a garbage collector, it isn't like interrupt latency can be promised.

Instead, we think asyncio + a "background task" that tracks GPIO change/fall/rise can be used to capture interrupts for when we are ready to process them. We have two native modules, countio and keypad, that can track your pin state changes in the background.

Also, we really, really want to keep CircuitPython code a true subset of CPython code so that examples can run on boards like Feather M4's or CircuitPlaygrounds as well as Raspberry Pi and desktop Python computers.

Hey, why aren't you supporting threads?

Python developers have dabbled with threads and pretty quickly determined that they are not a good way to have multiple tasks that can co-operate with shared memory. We believe that asyncio is a better way to have concurrent tasks that can share the same memory space safely - and without having to learn about and debug concurrent processes, something that is so notoriously hard that CS students have to take a course on how to do safely (or at least, they should if they don't!).

The vast majority of microcontrollers that are supported have a single core, and those that have dual core, like Espressif chipsets, would probably benefit from pinning certain background tasks like WiFi handling to a separate core, rather than trying to balance processing manually.

Cooperative Multitasking

Cooperative multitasking is a style of programming in which multiple tasks take turns running. Each task runs until it needs to wait for something, or until it decides it has run for long enough and should let another task run.

It's up to each task to decide when to yield control to other tasks, which is why it's cooperative. A task can freeze out other tasks, if it's not well behaved. This contrasts with preemptive multitasking, where tasks are interrupted without their knowledge to let other tasks run. Threads and processes are examples of preemptive multitasking.

Cooperative multitasking does not imply parallelism, where two tasks run literally simultaneously. The tasks do run concurrently. Their executions are interleaved: more than one can be active at a time.

In cooperative multitasking, a scheduler manages the tasks. Only one task runs at a time. When a task gives up control and starts waiting, the scheduler starts another task that is ready to run. The scheduler is fair and gives all tasks that are ready the chance to run. The scheduler runs an event loop which repeats this process over and over for all the tasks assigned to the event loop. You are already familiar with event loops, but you might not know the term. The while True loop that is the main part of nearly all CircuitPython programs often serves as an event loop, monitoring for button presses, or simply running some code periodically on a schedule. The loop() routine, a required part of every Arduino program, is also mean to be used as an event loop.

A task can wait for the completion of a sleep period, a network or file I/O operation, a timeout. It can also wait for some external asynchronous event such as a pin changing state.

The diagram below shows the scheduler, running an event loop, with three tasks: Task 1 is running, Task 2 is ready to run, and is waiting for Task 1 to give up control, and Task 3 is waiting for something else, and isn't ready to run yet.

Coroutines

A task is a kind of coroutine. A coroutine can stop in the middle of some code and return back to its caller. When the coroutine is called again, it starts where it left off.

You may be familiar with generators in Python, which are also a kind of coroutine. You can recognize a generator because it includes one or more yield statements in it. When a coroutine gets to a yield statement, it returns to its caller, optionally passing back a value. When the coroutine is called again, it starts at the next statement after the yield. Early cooperative multitasking systems in Python took advantage of the generators mechanism, and used yield to indicate when a task was giving up control.

Later, Python added the async and await keywords specifically to support cooperative multitasking. A coroutine is declared with the keyword async, and the keyword await indicates that the coroutine is giving up control at that point.

The diagram below shows two coroutines, f() and g(), which are used as tasks. When Task 1 starts, it runs until it reaches an await statement where it needs wait for something. It gives up control at that point. The scheduler then looks for another task to run, and chooses Task 2. Task 2 runs until it reaches its own await. By that time, let's assume that whatever Task 1 was waiting for has happened. The scheduler sees that Task 1 is ready to run again, and starts up f() where it left off.

This guide will explain asyncawait, and their use with the asyncio library in more detail, in some simple examples, starting on the next page.

The asyncio library is not in the CircuitPython library bundle at the moment due to a bundling issue. You can get it from the GitHub link below.

asyncio Library

Italicized remarks are not true at the moment.

Most of the examples in this guide require the CircuitPython version of the asyncio library. The library is available in the CircuitPython Library bundle, and is available on GitHub as well. You can also use the circup tool to get the library and keep it up to date. If you find a problem with the library that you think is a bug, please file an issue.

Hardware

async and await (and therefore asyncio) are available on most CircuitPython boards. SAMD21 ("M0") boards, such as the Trinket M0, Metro M0, or Feather M0 boards do not have enough flash or RAM. A few nRF boards with internal flash only also do not have enough flash. 

Not all CircuitPython boards have the resources to use multitasking. Pick a board with adequate resources for your needs.

Future Enhancements

CircuitPython asyncio and related libraries and modules will be enhanced over time. For details, check out the issue list in Adafruit's asyncio library. For countio, see its issues, including this one.

This guide was first published on Nov 23, 2021. It was last updated on 2021-11-23 19:34:33 -0500.

This page (Overview) was last updated on Dec 04, 2021.

Text editor powered by tinymce.