The 29-year-old is the Hoops' third summer signing following the arrivals of Manchester City defender Dedryck Boyata and Saidy Janko from the Sky Blue's city rivals United.
And he is likely to be joined in the East End of Glasgow by Nadir Ciftci, with Celtic closing in on the Dundee United striker's capture after the Tannadice finally accepted their third bid on Tuesday.
Bailly, though, is delighted his deal is already signed and sealed.
Bailly started his career with Standard Liege before moving to Genk in 2002. A move to German outfit Borussia Munchengladbach followed six years later.
The keeper also spent time in Switzerland on loan at Neuchatel Xamax and has also represented Belgium at every level from Under-16 up to the national side's senior team, winning eight caps.
But now his focus is on dislodging Gordon from Deila's line-up.
"I have a lot of respect for Craig because he is a great goalkeeper with a lot of experience in a long career so it will be nice to be training with him," said Bailly.
"I'm 29 but I can still learn a lot from him. I think I'm in my prime just now as 29 or 30 years old is a great age for a goalkeeper.
The keeper has already met his new team-mates up at Lennoxtown and is looking forward to settling in.
He said: "I was up at the training complex yesterday morning and I met the players. Every country is different but this is my first time in Scotland and if you are happy in your football then everything else is all right.