Ollama is an inference engine, which is a tool for managing and running local LLMs. It also provides a Python library for interacting with local LLMs, allowing them to be integrated into projects. Ollama runs on different hardware and operating systems (OS) including Mac, Windows, and Linux.

Ollama supports many different models, but the majority of them are made for running on fancy GPUs and higher powered computers than the Raspberry Pi. However, there are some that are small enough to work on the Raspberry Pi 5.

Install Ollama

To install Ollama, download and run the shell script from ollama.com/download.

On a Raspberry Pi 5, the process takes 3-5 minutes or longer for slower networks. During installation, it will print messages regarding the steps being taken and progress as well as a warning about not finding a GPU.

After it is finished, verify that that Ollama installed successfully by running the help command:

Install Ollama Python Bindings Library

Activate the same virtual environment created in the steps on the Pi & Blinka Setup page if it is not already active.

Next, install ollama with pip using this command.

For more detailed instructions, and a demo script to verify the installation worked see the Python Integration page in the Local LLMs on Raspberry Pi guide.