where \(p\) -bits are allocated to the mantissa (defining the value itself), \(e\) -bits are allocated to the exponent (its order of magnitude), and its sign fits in the remaining bit. Alongside the blueprint for constructing floating-point
data, the IEEE-754 [1]} norm specifies floating-point formats commonly used in CPUs, such as binary32 (\(e=8\) , \(p=23\) ) and binary64 (\(e=11\) , \(p=52\) ), also known as single and double precision or “floats” and “doubles”. Recently, deep learning has led to the emergence of reduced formats such as Microsoft’s ms-fp8 Minifloats (8 bits), IEEE-754 binary16 (16 bits), bfloat16 (16 bits) [2]}, or posits (8, 16, 32, and 64 bits) [3]}. However, reduced formats implicitly store less information compared to double precision and should therefore be used carefully.
