OK, I just found someone that has the exact same problem, and he fixed it, I'll explain what he did, for anyone that has the same problem with using the iDRAC 6 Enterprise Virtual Console with Java 8. The problem is that Oracle in their (unknown wisdom) decided that self signed certificates are taboo, and must NEVER be used, thus basically forcing people to go out and buy a signed certificate from an accredited authority, but there is a way to get round the Java security blocking the virtual console. First, if you try to launch the console you'll get an error that Java has blocked the certificate, so in that case you'll need to open the Java control panel which in windows 10 is in the Windows control panel under Java or Java(32), then when the Java control panel is open, click the 'Security' tab, then at the bottom, click 'Edit Site List', then click 'Add' then enter your server's LAN IP address, then click OK, then click OK again, but if you try to use the console, you'll eventually get a 'Connection Failed' error, this is because of the Java crypto algorithm used, and Dell uses RC4, so this is easily remedied in the java.security file. Second, open a command prompt with elevated privileges, IE cmd with admin access, by opening the windows search then type cmd and right click the cmd line and select 'Run as administrator', then navigate to the java security file which in Windows 10 is at:- C:\Program Files (x86)\Java\jre1.8.0_251\lib\security In the command prompt type cd "C:\Program Files (x86)\Java\jre1.8.0_251\lib\security"   Don't forget the quotes, otherwise windows will give you an error. If you're reading this when there's been an update, then replace jre1.8.0.251 with the version number you're currently using. Third, in the command prompt type this:- notepad java.security Press enter, then press CTRL-F and look for jdk.tls When it's found look for the line:- jdk.tls.disabledAlgorithms=SSLv3, DES, RC4, MD5withRSA, DH keySize < 1024, \EC keySize < 224, 3DES_EDE_CBC, anon, NULL Delete the bit that has RC4, (including the comma) so that it looks like this:- jdk.tls.disabledAlgorithms=SSLv3, DES, MD5withRSA, DH keySize < 1024, \EC keySize < 224, 3DES_EDE_CBC, anon, NULL Save it and then you can successfully connect to your server using the iDRAC 6 Enterprise Virtual Console. I just tried it and it works.
After updating iDRAC6 FW and the LCC FW, it seems now that Java wanted to update too so now it's version 8.0.251, and now I can't get the virtual console to connect, because Java blocks it, I even added my server IP to Java's exception list, but nope, Java still blocks it, I don't want to remove v8, and re-install v7, because a lot of apps that use Java, insist on using v8, is there a way to force Java 8 to accept the self signed certificates that iDRAC creates every time the console is launched? It seems now that Oracle in their (unknown wisdom) have decided that self signed certificates are taboo, and should NEVER be used, which forces us to buy an SSL certificate from a reputable cert service even though we're only using local LAN access only, IE, iDRAC is only accessible from within the LAN, as nothing is being forwarded to the iDRAC dedicated IP. Java 7 had the option in the security tab of 'Medium' settings which allows the virtual console to run, but Java 8 has removed that setting, and now it's only high or very high.
iDRAC 6 virtual console does not work with Java 8