Chris, it has been rebooted twice.  Anyway I moved the iDRAC connection inside the firewall (was a public IP) and it seemed to start working again.  I don't see why this should make any difference, but the iDRAC log showed a large number of failed log on attempts on the public IP.  Perhaps that had something to do with it.  Anyway, I didn't understand why it stopped working and I don't understand why started working again.  I can live with the mystery unless it happens again.       
This worked fine until last week..... Using iDRAC 6 Ent and ESXi 5.0 on a R710, I was able to log into the ESXi console and select troubleshooting options to restart the ESXi management agents.  Starting last week, I can bring up the ESXi console via iDRAC, but when I press F2, the logon prompt does not come up.  After the initial F2 which brings up the black and yellow screen, no keystrokes seem to get passed to the console session.  I have tried using both the Java and native consoles, but neither seems to work.  I have tried this from several different computers.  I don't get it.   
iDrac 6 and VMWare ESXi console