TLDR: A NVIDIA P4 card works in a Dell R620 out of the box just fine (when its a good card). After exhausting all the steps outlined above with regards to checking slots, BIOS settings, firmware revisions on both the P4 and the R620, the vendor kindly sent me another P4 to swap out the troublesome card. Feeling optimistic I initially placed the card in slot 2 of the 3 as the older Quadro card was back in slot 3 of the chassis and after running though the BIOS settings and calibration test it passed POST. It booted into Linux without error but the NVIDIA driver didnt detect either card during the module load. Command nvidia-smi returned no card found. Halted system. Removed both GPU's, Put P4 in slot 3, rebooted, calibrated power, viola this time the card was detected during module load and vida-smi returned info about the P4. Ran some quick functional checks using stable-diffusion to make sure it was working as expected. Halted system Reinstalled the Quadro in slot 2 and left the P4 card in slot 3. Rebooted, calibrated power, both cards detected and functional by the NVIDIA drivers. Final verdict: POST fault was due to a bad NVIDIA P4 card. Replaced card, system working as expected. For the forum and history books should anyone else ask and because I've seen some questions online about VBIOS version, idle temperature, and fan noise. First an "unsupported" NVIDIA P4 card works in a Dell R620 out of the box.  NVIDIA VBIOS version: 86.04.8C.00.10 Idle temperature: 37C R620 fan noise at idle (relative): louder than without a GPU, but nowhere near "screaming". Not significantly louder running GPU intensive tasks like stable-diffusion or 3D rendering. No out of chassis mods or firmware hacks required ala ATFM2's post above. A classic solution but not required if you have a working card. I understand page 18 lists supported GPU cards but the documentation is dated before these later cards came out, and also only what Dell tested/supported. Both my cards are not supported according to the documentation but they do work. The key to look for is physical size of the card and power draw has to be less than 75W for the PCIe slot. A R620 physically cant fit a two slot card (most current models) and would exceed the power capacity of the bus. There is no PCIe power port on the R620. You would have to go with ATFM2's out of the chassis approach to go bigger. Thank you to all those who took the time to respond with troubleshooting advice!
Good evening! I have a Dell Power Edge R620 with the latest BIOS 2.9.0 that will not boot with a NVIDIA Tesla P4 card (75W passive cooling) installed. It gives a PCIe Training Error Slot # (varies by slot I installed it in) during POST. Various web forms report success with this card in a R620 though its not on the officially supported list but I cant seem to get it to boot/pass POST. I reset the BIOS to defaults and removed all the other PCIe cards (a Dell USB 3.0 card and NVIDIA Quadro K620), swapped the GPU between slots 1, 2, 3 and no change in the error message other than the slot # reported. I normally use EFI to boot a Linux OS, and tried switching to BIOS - no change in the error message. I unplugged the machine, held the power button for 10 seconds and after power on it appeared to pass the first POST with the P4 installed, all other cards still removed, it calibrated the power settings then after it rebooted it halted again with the same error.  I've tried enabling/disabling the 4Gb extension - same error. Is there any other settings that could be causing a conflict or that I can set? The system currently has a NVIDIA Quadro K620 that also has 75W draw using passive cooling installed and running fine so I don't think its a power draw issue. The P4 is a circa 2016 GPU I bought online, its not Dell branded, but I would think within the support range of the later 2020 Dell 2.9.0 BIOS. So bad Tesla P4 GPU or is it simply incompatible with the R620? Would back flashing to an older Dell BIOS be an option?  
NVIDIA Tesla P4 GPU in Dell R620 won't pass POST