Hello,   I did some a bit of digging through our KBs and past cases, and couldn't find an instruction set that seemed helpful. I did, however, find a previous case and based on what I was seeing there, it looks like you'll need to contact support. From what I'm seeing here, they needed to escalate and set up a remote session to correct it.
When I go to the Avamar Administrator console (JAVA version) and then select the option TOOLS - Manage ESRS, to add it to the newly added Secure Connect Gateway 5, it fails with the following error message: "Registered to ESRS Gateway failed: Failed to register to ESRS. Certificate not verified." The same Avamar machine could successfully register to the older ESRS 3.x. Note: Avamar is the Virtual Edition version 19.4.0-116_HF332282
Add of AVAMAR fails