Floating-point error made physical: every 8-bit float as a steel pin on a number line, then a dive into binary64 for 0.1 + 0.2, absorption at 2^24 and special values
This rail holds every non-negative 8-bit float up to 16 (1 sign, 4 exponent and 3 mantissa bits, bias 7), one steel pin per bit pattern. Drag the carriage along the rail. It can only stop on a pin, because nothing in between exists.
binary8 has 256 patterns: 240 finite values (including +0 and −0), 2 infinities and 14 NaNs. The largest finite value is 240.
Read the bit pattern as an unsigned integer and add one: you land on the next float up, every time. Step from 1.625 to 1.875, then once more: the mantissa rolls over to 000 and the carry flips the exponent from 0111 to 1000. Click any switch to flip a bit yourself.
This is why positive floats sort correctly as integers, and why nextafter() is a single integer increment.
Each binade [2^e, 2^(e+1)) holds exactly 8 values here, so the gap doubles every time the exponent goes up. Pick a binade: the readout shows its spacing, one ULP. Relative precision stays the same while absolute precision halves.
binary32 has 2^23 values per binade, and about half of all its finite values lie in [−1, 1].
Amber wires mark the decimals you typed; the white wire is the exact sum of the two stored values. In binary8, 0.1 + 0.2 rounds to 0.3125 and so does 0.3, so the comparison is true. Now try 0.1 + 0.6: the sum rounds to 0.75, but 0.7 rounds to 0.6875.
Whether a + b == c holds depends on where three separate roundings land, not on the decimals.
The same sum in binary64. The rail zooms in 2^50 times around 0.3 until single doubles separate into pins 2^−54 apart. The exact sum of the stored 0.1 and 0.2 falls exactly halfway between two pins, ties-to-even picks the upper one, and you get 0.30000000000000004: one ULP above 0.3.
The two results differ only in the last mantissa bit; the readout shows both bit patterns in hex. The exact sum of the stored values is 0.3 plus about 1.67 × 10^−17.
A binary32 counter just below 2^24 = 16777216. Below it the pins are 1 apart; above it they are 2 apart. Add 1 until you reach 2^24: the exact result 16777217 is a tie, rounds back down to the even pin, and the counter never moves again.
binary64 stalls the same way at 2^53, which is why JavaScript's Number.MAX_SAFE_INTEGER is 2^53 − 1.
Add 0.1 ten times in binary64 and you stop one pin short of 1: 0.9999999999999999. Kahan summation carries each addition's rounding error in a second variable and lands on 1 exactly. Switch between the two and watch where the carriage stops.
Dhahran, 25 Feb 1991: a Patriot battery's 24-bit fixed-point clock lost 0.34 s in 100 hours of uptime, and its radar looked over half a kilometre away from the incoming Scud. 28 soldiers died (GAO IMTEC-92-26).
The exponent's two extremes are special. All zeros gives ±0 and the subnormals, evenly spaced down to zero, so x − y is 0 only when x == y. All ones gives ±inf, and NaN, which is not equal to anything, itself included.
−0 == 0 is true, but 1 / −0 is −inf. Subnormal arithmetic can be far slower on some CPUs, which is why flush-to-zero modes exist.
Flip any of the eight switches or drag the carriage. Every pattern lands on a pin, hits the end stop or leaves the rail entirely, and the readout gives its exact value.
Every binary8 value is exact in binary64, so the readout's decimals are exact.