He would have needed to beat Shanks before he lost his arm to claim the title of WBS if he never fought Shanks afterwards . Or Shanks never claimed to be a swordsman in the first place , and was more a swashbuckling brawler . Or Shanks said You can have the title , I don ' t care . Or as you say people gave him the benefit of the doubt . Or another person Rayleigh perhaps held the title the whole time they would duel . And by the time Mihawk took the mantle of strongest swordsman by defeating Rayleigh , Shanks had lost an arm and so was no longer considered a swordsman but is still much stronger of an overall fighter . Fujitora weilds a sword but that doesn ' t mean Mihawk is stronger than Fujitora . In the same vein , Shanks can be stronger than Mihawk without Mihawk ' s title of strongest swordsman being called into question .