Skip to content
hpcnotes
Loading…
HPC·Arithmetic
2026.09.11· 1 min

Rounding Errors

The error is the algorithm. If you need bits, say how many, and where they die.
IEEE 754 Floats........................................................................................................................................................................................................Previous
Newton's Method........................................................................................................................................................................................................Next
#hpc
Floating-Point Numbers........................................................................................................................................................................................................2026.09.11
IEEE 754 Floats........................................................................................................................................................................................................2026.09.11
Newton's Method........................................................................................................................................................................................................2026.09.11
Fast Inverse Square Root........................................................................................................................................................................................................2026.09.11
Integer Numbers........................................................................................................................................................................................................2026.09.11
Suggest an editMarkdownHistory