The previous pages feature two interesting examples of the care that's required with numbers in a computer language.

1. The unit test uses the peculiar sounding assertAlmostEqual method.
2. One of the interim test failures reported a number as -253.14999999999998 where the buggy code would be expected to produce -253.15.

It's very common for languages to use data types to represent numbers which have a fixed size. For an integer this limits the range of numbers. For a real number which are typically represented in floating-point form this limits the precision and range. The example below shows how 6.25 is represented exactly in floating-point - the relatively uncommon half precision (FP16) format is used for brevity.

Binary computers using the ubiquitous IEEE 754 standard cannot represent 253.15 exactly in the same way that 1/3 cannot be represented exactly as a decimal number. These miniscule errors are often compounded and enlarged by repeated or large calculations.

The assertAlmostEqual() method is specifically for floating-point comparisons where the equivalence test needs to take into account the precision limitations of floating-point arithmetic. The assertEqual() method could be used with the correct, exact number produced by the computer's arithmetic but this is too fragile as a change in

• calculation order,
• rounding mode or
• intermediate precision

may alter the resulting number.

Python 3

Python hides the detail of the number representation from the programmer most of the time. The type() function will reveal whether a number is an int or a float:

• int - arbritrary precision (length) integers;
• float - 64bit floating-point.

Care is still needed with very large integers to avoid loss of precision from unintentional conversion to a float. It's common to accidentally use / for (floating-point) division when // (floor division) was intended - the latter retains the int type if both operands are ints.

CircuitPython

CircuitPython is a variant of MicroPython and both maintain Python's approach of largely hiding the number representation from the programmer.

Small Boards

This category, sometimes referred to as the "non-Express boards", comprises the Gemma M0, Trinket M0 and Feather M0 non-Express boards.

• int - 31bit, -1073741824 to 1073741823. An OverflowError exception is raised beyond this range.
• float - 30bit storage but calculations are performed with 32bit precision.
   

Large Boards

Everything not in the above small category which runs CircuitPython.

• int - arbritrary precision integers.
• float - 30bit storage but calculations are performed with 32bit precision.

C/Arduino

For comparison:

• int 16bit on Arduino Uno (and other ATmega based boards) and 32bit on Arduino Due and SAMD based boards.
• float - 32bit.

Zuse Z1

An older comparison:

• float - 24bit storage but calculations are performed with 28bit equivalent precison.

Decimal Precision

The equivalent decimal precision in digits (significant figures) for a range of floating-point sizes is shown in the table below.

The Schmidt JavaScript Visualisation of 32bit IEEE-754 floating-point is a useful online tool to understand floating-point. Stored values in CircuitPython will always have the lower two bits not set. The animated example below shows how the value 3.14 is represented and how it would be represented in CircuitPython with two fewer bits.

The next page discusses the practical issues of representing time in CircuitPython due to the underlying number representation.