Were it just a matter of a contract between an airline an one individual, you'd be right. However the airline also has a contract with all the other paying passengers to get them where they're going. There's also the special legal nature of aircraft, boats and so forth. Contract or no, the captain has the final say in who's on and who's off.