Local LLMs on Raspberry Pi

Explore the possibilities of running LLMs locally on the Raspberry Pi & similar hardware.

by Tim C

published September 10, 2025, last edited September 18, 2025

posted in Raspberry Pi Programming/ Linux

Save Download

Intermediate

Project guide

SmolLM3

SmolLm3 is one of the Smol Models, which is a family of efficient and lightweight AI models from Hugging Face. They are fully open models, with open weights and full training details, including public data mixture and training configs published.

Run The Model

To run the model simply issue the ollama run command specifying the model name:

Copy Text

ollama run alibayram/smollm3:latest

ollama run alibayram/smollm3:latest

The first time you run the model, it will download the model and its associated files. The smollm3 model and associated files are around 1.9gb in size. The download time will depend on your network speed.

ollama CLI output from installing the smollm3 model

Once you see the "Send a message" prompt, the download is complete.

Type in whatever prompt you want to give to the model and see the output get generated.

SmolLM3 eval stats Pi5

Copy Text

$ ollama run --verbose alibayram/smollm3:latest
>>> please write a short poem about the Raspberry Pi. No more than one paragraph.
<think>

</think>
In a tiny box, a world awakens,
A Raspberry Pi, with heart and brain.
Tiny fingers dance on its screen,
Unlocking secrets of code's refrain.

With Linux as its soul, it sings,
Of circuits and connections so fine.
From coding to computing, it reigns,
A champion of the tech age divine.

total duration:       14.236032244s
load duration:        185.644813ms
prompt eval count:    24 token(s)
prompt eval duration: 165.400373ms
prompt eval rate:     145.10 tokens/s
eval count:           72 token(s)
eval duration:        13.884483189s
eval rate:            5.19 tokens/s

$ ollama run --verbose alibayram/smollm3:latest
>>> please write a short poem about the Raspberry Pi. No more than one paragraph.
<think>

</think>
In a tiny box, a world awakens,
A Raspberry Pi, with heart and brain.
Tiny fingers dance on its screen,
Unlocking secrets of code's refrain.

With Linux as its soul, it sings,
Of circuits and connections so fine.
From coding to computing, it reigns,
A champion of the tech age divine.

total duration:       14.236032244s
load duration:        185.644813ms
prompt eval count:    24 token(s)
prompt eval duration: 165.400373ms
prompt eval rate:     145.10 tokens/s
eval count:           72 token(s)
eval duration:        13.884483189s
eval rate:            5.19 tokens/s

Page last edited September 10, 2025