The previous pages feature two interesting examples of the care that's required with numbers in a computer language.
- The unit test uses the peculiar sounding
assertAlmostEqual
method. - One of the interim test failures reported a number as
-253.14999999999998
where the buggy code would be expected to produce-253.15
.
It's very common for languages to use data types to represent numbers which have a fixed size. For an integer this limits the range of numbers. For a real number which are typically represented in floating-point form this limits the precision and range. The example below shows how 6.25 is represented exactly in floating-point - the relatively uncommon half precision (FP16) format is used for brevity.

Binary computers using the ubiquitous IEEE 754 standard cannot represent 253.15 exactly in the same way that 1/3 cannot be represented exactly as a decimal number. These miniscule errors are often compounded and enlarged by repeated or large calculations.
The assertAlmostEqual() method is specifically for floating-point comparisons where the equivalence test needs to take into account the precision limitations of floating-point arithmetic. The assertEqual() method could be used with the correct, exact number produced by the computer's arithmetic but this is too fragile as a change in
- calculation order,
- rounding mode or
- intermediate precision
may alter the resulting number.
Python 3
Python hides the detail of the number representation from the programmer most of the time. The type()
function will reveal whether a number is an int
or a float:
-
int
- arbritrary precision (length) integers; -
float
- 64bit floating-point.
Care is still needed with very large integers to avoid loss of precision from unintentional conversion to a float
. It's common to accidentally use /
for (floating-point) division when //
(floor division) was intended - the latter retains the int
type if both operands are int
s.
CircuitPython
CircuitPython is a variant of MicroPython and both maintain Python's approach of largely hiding the number representation from the programmer.
Small Boards
This category, sometimes referred to as the "non-Express boards", comprises the Gemma M0, Trinket M0 and Feather M0 non-Express boards.
-
int
- 31bit, -1073741824 to 1073741823. AnOverflowError
exception is raised beyond this range. -
float
- 30bit storage but calculations are performed with 32bit precision.
Large Boards
Everything not in the above small category which runs CircuitPython.
-
int
- arbritrary precision integers. -
float
- 30bit storage but calculations are performed with 32bit precision.
C/Arduino
For comparison:
- int 16bit on Arduino Uno (and other ATmega based boards) and 32bit on Arduino Due and SAMD based boards.
- float - 32bit.
Zuse Z1
An older comparison:
- float - 24bit storage but calculations are performed with 28bit equivalent precison.

Decimal Precision
The equivalent decimal precision in digits (significant figures) for a range of floating-point sizes is shown in the table below.
The Schmidt JavaScript Visualisation of 32bit IEEE-754 floating-point is a useful online tool to understand floating-point. Stored values in CircuitPython will always have the lower two bits not set. The animated example below shows how the value 3.14 is represented and how it would be represented in CircuitPython with two fewer bits.

The next page discusses the practical issues of representing time in CircuitPython due to the underlying number representation.
Page last edited March 08, 2024
Text editor powered by tinymce.