From Lee's perspective, " Lee owed, and gave, his allegiance to his home State of Virginia, rather than to the Union as a whole."
That's all.
Others have a different perspective. He had served in the US Army for decades and had taken an oath to protect and defend the UNITED States.