Frozen Modules

In the previous page you saw how to load modules and packages from Python code files on a board's filesystem.  This is a quick and easy way to load modules, but there are some big limitations with storing and running code from the filesystem.  The biggest limitation is that raw Python code has to be loaded and processed by MicroPython's interpreter.  This process takes some time and memory which on some boards is quite limited.  You might even find that some code files are so large they can't be loaded in memory and processed by MicroPython's interpreter!

However there is a way to pre-process modules so they're 'baked-in' to MicroPython's firmware and use less memory.  MicroPython can squeeze almost all of the Python language into kilobytes of memory by 'freezing' Python code so that it's stored as more efficient byte-code or even compiled into native code instead of being stored as raw Python code.  Once the code is frozen it can be quickly loaded and interpreted by MicroPython without as much memory and processing time.

With MicroPython you can freeze your own modules and add them to a custom build of the MicroPython firmware for your board.  This allows you to write more complex modules than you might be able to run directly from .py Python code files as previously shown.

Be aware that freezing modules is a bit advanced and requires you to make a custom build of MicroPython's firmware.  For some boards like the ESP8266 it's relatively easy to build MicroPython firmware, but for others it might require more advanced knowledge of setting up and using compiler toolchains, etc.

Before you look at freezing modules make sure you can compile MicroPython firmware for your board.  For the ESP8266 check out this handy guide on how to compile MicroPython firmware using a special Linux-based virtual machine.  For other boards like the pyboard, WiPy, etc. for now you'll need to check the MicroPython source to see how to setup their toolchains and compile firmware.

Also be aware freezing modules is somewhat new for boards like the ESP8266 and the process might change over time.  If the steps below have issues, check the latest documentation, code on GitHub, and MicroPython forums to see if more up to date information on freezing modules is availble.

For this guide we'll look specifically at the ESP8266 and how to add a frozen module to the board's MicroPython firmware.  Start by looking at the source of the MicroPython ESP8266 firmware.  If you're using the Vagrant virtual machine from the previously mentioned firmware building guide then you'll want to enter the VM with SSH and navigate to the ~/micropython/esp8266 folder.  Notice there are two subdirectories, scripts and modules:

There's an important difference between these two folders.  The scripts folder is for Python code that will be 'baked-in' to the firmware but not frozen into more efficient and smaller code.  Everything in the scripts folder is just stored as the raw Python code in the board's flash memory.  This saves you from having to copy that code onto the board's filesystem, but doesn't save a lot of memory or processing time.

The modules folder is for Python code that will be frozen into more efficient bytecode.  This is where you want to place scripts that you'd like freeze to save memory.

Try adding a new frozen module as a quick test.  Inside the modules folder create a test.py file and add the following code to it:

def add(a, b):
        return a + b

def subtract(a, b):
        return a - b

Now rebuild the ESP8266 MicroPython firmware.  From inside the VM's ~/micropython/esp8266 folder run:

Notice in the output the MPY modules/test.py line, this shows the test.py module was picked up by the build and frozen into bytecode:

If you don't see this line for your module then try running the make clean command and then make command again.

Now copy out the newly built firmware and flash it to the ESP8266.  Remember you're now using a development build so debug output will be enabled and features like the WebREPL must be manually started.

Connect to the REPL of the board with the new firmware and try using the test module's functions by running:

import test
test.add(1, 1)
test.subtract(1, 1)

You should see the module import and functions work just like if the .py file was on the filesystem!

You can also freeze Python packages inside the modules folder.  Just create a directory for each package and put inside it an __init__.py and other source files for the package.

Using the scripts folder to save Python module sources in flash memory is just like using the modules folder above.  However note with the scripts folder you cannot put packages inside it, only single .py source files will currently work. 

Remember the scripts folder is only for scripts that will be saved as-is to flash memory.  The modules folder is for freezing modules and packages so they're in flash memory and more efficient to load and store.  As you develop your own complex MicroPython scripts and modules consider freezing them in the MicroPython firmware to reduce memory usage and increase performance!

Last updated on Aug 29, 2016 Published on Aug 29, 2016