If you consider the dynamic range of varying bit depths, 1 bit being roughly equivalent to 6dB of dynamic range, then it makes sense that the higher the bit depth the higher the dynamic range. With 24 bit depth, the dynamic range (theoretically) is 144dB. Bearing in mind that our hearing does not even come close to a 144dB range, it makes sense to use a dynamic range beyond our hearing’s dynamic range for the very simple reason that audio captured at this bit resolution will fall below our hearing’s minimum range and above its maximum range.
To accommodate internal processing within a digital system a much higher headroom is required for the simple reason that processing will require additional bits. By adding more than one 24 bit numbers together it is obvious that more bits are required. Dynamic processing, by its very nature, requires higher bit counts as the process itself generates bits, or subs of, that need managing otherwise there will be sonic compromises.
The 32 bit system seems to handle these processes well and it has become a minimum standard. Of course, we now have higher bit internal processing.
Fixed Point systems use the 32 bits in the standard way and the maths is simply a scale that provides a dynamic range of 192dB (32×6). The usual procedure is to allow the 24 bit signal to work closely at the top of the 32 bit processing. This makes complete sense as it provides a higher headroom and a lower noise floor.
Floating-point still uses the 32 bit system but arranges the bits in a different manner. The signal is still kept in 24 bit but the remaining bits are allocated to denote scaling factors. This basically means that the 24-bit can be used in a more flexible and dynamic manner allowing for a massive dynamic range. This equates to a never-ending scale of headroom and a noise floor that is so low as to be negligible.
Relevant content: