What they are referencing to is that only one type of DIMM is allowed per system: UDIMM, RDIMM, or LRDIMMs. They cannot be mixed. The R720XD does support mixing dimms, as clarified on the bottom of page 42, as well as the top of 43, here. 
It looks like Dell doesn't sell 2RX4 8GB RAM modules any more. They recommend 2RX8 and 1Rx4 for the R720XD.  Can I mix and match my old 2RX4 with the new 2RX8? I believe the manual says no, but it seemed like an off-hand recommendation, and not a big bold warning.  I have 12 modules installed.   DUAL IN-LINE MEMORY MODULE, 8GB, 1600, 2RX4, 2G, DDR3L, R
r720XD mixing 2RX4 with 2RX8?