Local LLMs on Raspberry Pi

Explore the possibilities of running LLMs locally on the Raspberry Pi & similar hardware.

by Tim C

published September 10, 2025, last edited September 18, 2025

posted in Raspberry Pi Programming/ Linux

Save Download

Intermediate

Project guide

Qwen3

The Qwen family of models were launched by Alibaba Cloud in 2023. Qwen3 is the latest generation of the models. The smallest Qwen3 model is the 0.6b version which works nicely on the Raspberry Pi 4 & 5 with 8 GB RAM.

Run The Model

To run the model simply issue the ollama run command specifying the model name.

Copy Text

ollama run qwen3:0.6b

ollama run qwen3:0.6b

The first time you run the model, it will download the model and its associated files. The qwen3 0.6b model and associated files are about 525mb.

ollama CLI output from downloading the Qwen3 0.6b model

Once you see the "Send a message" prompt, the download is complete.

Type in whatever prompt you want to give to the model and see the output get generated.

qwen3:0.6b eval stats Pi5

Copy Text

$ ollama run --verbose qwen3:0.6b
>>> please write a short poem about the Raspberry Pi. No more than one paragraph.
Thinking...
Okay, the user wants a short poem about the Raspberry Pi in one paragraph. Let me 
start by recalling what I know about the Raspberry Pi. It's a popular open-source 
platform for learning programming, right? So the poem should highlight its purpose, 
maybe mention its versatility.

First line: "A humble device, a puzzle for the curious," that sets the scene. Then, 
talk about its uses—like coding, robotics, or home automation. Need to keep it 
concise. Maybe mention specific projects like a simple server or a robot. End with 
something that ties it back to the Raspberry Pi's community. Avoid being too 
technical, but still informative. Check the paragraph is one, so structure it well. 
Make sure it's engaging and flows naturally.
...done thinking.

A humble device, a puzzle for the curious,  
A tiny world where code and art blend.  
With a single power, it hums through the night,  
A gateway to the unknown, and its light.

total duration:       12.609667335s
load duration:        154.679968ms
prompt eval count:    26 token(s)
prompt eval duration: 420.988582ms
prompt eval rate:     61.76 tokens/s
eval count:           200 token(s)
eval duration:        12.033309246s
eval rate:            16.62 tokens/s

$ ollama run --verbose qwen3:0.6b
>>> please write a short poem about the Raspberry Pi. No more than one paragraph.
Thinking...
Okay, the user wants a short poem about the Raspberry Pi in one paragraph. Let me 
start by recalling what I know about the Raspberry Pi. It's a popular open-source 
platform for learning programming, right? So the poem should highlight its purpose, 
maybe mention its versatility.

First line: "A humble device, a puzzle for the curious," that sets the scene. Then, 
talk about its uses—like coding, robotics, or home automation. Need to keep it 
concise. Maybe mention specific projects like a simple server or a robot. End with 
something that ties it back to the Raspberry Pi's community. Avoid being too 
technical, but still informative. Check the paragraph is one, so structure it well. 
Make sure it's engaging and flows naturally.
...done thinking.

A humble device, a puzzle for the curious,  
A tiny world where code and art blend.  
With a single power, it hums through the night,  
A gateway to the unknown, and its light.

total duration:       12.609667335s
load duration:        154.679968ms
prompt eval count:    26 token(s)
prompt eval duration: 420.988582ms
prompt eval rate:     61.76 tokens/s
eval count:           200 token(s)
eval duration:        12.033309246s
eval rate:            16.62 tokens/s

Page last edited September 10, 2025