I've figured out the problem. First off, if you have a voltage issue (or any other that makes the server not boot) specifically with the ethernet daughter card on a R740, DON'T keep plugging in replacements, because chances are that your ethernet firmware on the system board is corrupted. This means that when you plug a new (or refurbished) card in, it will download the firmware that's present on your system board and basically ruin your card. What you need to do (or at least the solution we've found), is to plug the new replacement card into a server that you know works, so it can download working firmware upon first connection. After that you can plug it into the misbehaving server and be on your way. 
Hi, I'm having an issue with one of our watercooled R740 servers. After a BIOS (2.16.1) and Ethernet card (20.5.16) update, the node refused to turn on. Stripping it down to minimal configuration revealed nothing so we assumed the problem was with the system board. After replacing the system board and loading the node with miniimal config again, it turned out to be a problem with the ethernet card, because the server starts fine without the card, but fails to do so with it (showing the same pfault error). After realizing this, we replaced the card as well with the same model. This did not help either. We've tried to set the node up in any configuration we could think of (1 CPU, both CPUs, 1 DIMM in A1, 2 DIMMs in A1 and B1, backplane connected/disconnected, both PSUs replaced with ones from a known working server, reseating every single cable on the board etc). Nothing solves the issue and it always boils down to the ethernet card. With it the system just doesn't boot. When I look in the iDRAC webgui, I can see that the iDRAC does not recognize the state of the CPU, DIMMs and PSUs when the ethernet card is connected, not sure if that is related to the issue.Other steps we took were updating the CPLD, OS Drivers Pack, iDRAC and the OS Collector, draining the node after every component configuration change and rebooting the iDRAC after it reports the first pfault error. We have not tried the ethernet card in a different server in fears that it might cause issues.We are just really confused why the problem seems to stay with the chassis, even when the system board and ethernet card are completely different. If anyone has an answer to this, please let me know. Thanks
R740 Pfault fail-safe voltage is outside of range