Let's start looking a little more under the hood at how Python venv actually works. This will help to better understand the various options for using venv on Raspberry Pi.
What did the activate script do?
A Python venv is "activated" by running the aptly named activate script found in the venv bin folder. It's just a shell script, so it is a plain text file you can open in any text editor. It's not crazy long or complex, only about 70 lines of shell code - half of which define the deactivate function.
This script does some cute things like change the prompt to include the venv name. However, the key thing done is to alter the PATH variable such that the venv Python path shows up before the system path.
Observe the behavior before activating the venv:
pi@raspberrypi:~ $ which python /usr/bin/python pi@raspberrypi:~ $ echo $PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games pi@raspberrypi:~ $
There is no mention of the venv in PATH and as a result the python command is finding the system level installation in /usr/bin. (pip would have similar behavior)
Now activate the venv:
pi@raspberrypi:~ $ source foobar/bin/activate (foobar) pi@raspberrypi:~ $
And note the change in behavior:
(foobar) pi@raspberrypi:~ $ which python /home/pi/foobar/bin/python (foobar) pi@raspberrypi:~ $ echo $PATH /home/pi/foobar/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games (foobar) pi@raspberrypi:~ $
The python command is now pointing to the venv installation. This is a result of /home/pi/foobar/bin being added to PATH. The paths in PATH are searched in order, left to right. So by having the venv path before the system path, the venv is found first.
Why was activate "sourced"?
Shell scripts are usually "run". But above we instead "sourced" the activate script. Why was that done? It has to do with context. When a script is "run", it gets launched in a new shell context, and when the script is done, that context goes away. However, when a script is "sourced", it is run in the current context.
This is important for things like environment variables, which can be set and accessed very simply in bash:
pi@raspberrypi:~ $ FOO=23 pi@raspberrypi:~ $ echo $FOO 23 pi@raspberrypi:~ $
Now consider this simple script, called test.sh, which similarly sets a variable:
pi@raspberrypi:~ $ ls -l test.sh -rwxr-xr-x 1 pi pi 7 Oct 23 11:50 test.sh pi@raspberrypi:~ $ cat test.sh BAR=42 pi@raspberrypi:~ $
If this script is "run", the BAR variable does not persist:
pi@raspberrypi:~ $ ./test.sh pi@raspberrypi:~ $ echo $BAR pi@raspberrypi:~ $
However, if the script is "sourced", the BAR variable does persist:
pi@raspberrypi:~ $ source test.sh pi@raspberrypi:~ $ echo $BAR 42 pi@raspberrypi:~ $
Since "activating" a Python virtual environment needs to alter the current context, the activate script should be sourced, not run.
Use without activating
Activating a Python venv is really just a convenience. It's possible to use a venv without activating it by explicitly invoking the Python interpreter found in the venv. This is done by using absolute paths.
To demonstrate, let's use this simple Python script, called test.py, that tries to import the click module:
import click
print("Done.")
The click module is pip installed into the foobar venv:
(foobar) pi@raspberrypi:~ $ pip install click
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting click
Downloading https://www.piwheels.org/simple/click/click-8.1.7-py3-none-any.whl (97 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.9/97.9 kB 47.2 kB/s eta 0:00:00
Installing collected packages: click
Successfully installed click-8.1.7
(foobar) pi@raspberrypi:~ $ pip list
Package Version
---------- -------
click 8.1.7
pip 23.0.1
setuptools 66.1.1
(foobar) pi@raspberrypi:~ $
So running with the venv activated has the expected output:
(foobar) pi@raspberrypi:~ $ python test.py Done. (foobar) pi@raspberrypi:~ $
However, if the venv is deactivated, the script fails:
(foobar) pi@raspberrypi:~ $ deactivate
pi@raspberrypi:~ $ python test.py
Traceback (most recent call last):
File "/home/pi/test.py", line 1, in <module>
import click
ModuleNotFoundError: No module named 'click'
pi@raspberrypi:~ $
This is because click was only installed in the foobar venv. Once deactivated, we are back to using the system level install, which does not have click installed.
However, by using an absolute path to the venv, it works again:
pi@raspberrypi:~ $ /home/pi/foobar/bin/python test.py Done. pi@raspberrypi:~ $
Virtual Environment Configuration
A file named pyvenv.cfg that lives in the venv folder controls some basic virtual environment configuration.
pi@raspberrypi:~ $ ls foobar/ bin/ include/ lib/ pyvenv.cfg
This file contains a simple list of key = value pairs.
home = /usr/bin include-system-site-packages = false version = 3.11.2 executable = /usr/bin/python3.11 command = /usr/bin/python3 -m venv /home/pi/foobar
Probably the most important one is the include-system-site-packages option. This does pretty much what it says. If set to true, then the venv has access to modules (aka libraries, aka packages) installed at the system level. The default is false.
For example, gpiozero is usually pre-installed at the system level on most Raspberry Pi's:
pi@raspberrypi:~ $ python Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import gpiozero >>>
But with the default false setting of include-system-site-packages, the foobar venv does not have access to it:
pi@raspberrypi:~ $ source foobar/bin/activate (foobar) pi@raspberrypi:~ $ python Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import gpiozero Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'gpiozero' >>>
If the pyvenv.cfg in the foobar venv folder is changed to (only change is false to true):
home = /usr/bin include-system-site-packages = true version = 3.11.2 executable = /usr/bin/python3.11 command = /usr/bin/python3 -m venv /home/pi/foobar
Then the venv does have access:
pi@raspberrypi:~ $ source foobar/bin/activate (foobar) pi@raspberrypi:~ $ python Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import gpiozero >>>
Enabling access to system site packages can also be done when initially creating the venv by using the --system-site-packages command line argument:
python3 -m venv --system-site-packages foobar
Pip vs. apt
So there is pip for installing Python stuff. And there is also apt and apt-get that also get used for "installing stuff". Keep in mind these are two totally separate package management tools. Pip is specific to Python while apt and apt-get are operating system tools. So while activating a Python venv will change the behavior of pip installs (they'll go into the venv), there is no change to apt and apt-get behavior. Using apt/apt-get with or without a venv active will have the same behavior.
Text editor powered by tinymce.
