I think the setting in the current security zone for Forms Based Authentication is what changes that behavior.  Putting the server in the Intranet zone or making it a Trusted Site will often help.  In any case you can cancel out of the pop-up window and just login to the web page itself.
Hello all, I am using OMSA version 6.4, and I have noticed that when logging into it, there are two login screens presented, and the first one never actually does anything.  I always have to end up backing out of it, and then I am able to log into the second screen just fine.  What is the point of the first screen?
OMSA displays two separate login screens, and I've never understood why. 