The reason why wasn’t up for debate. Petunia Dursley recognized the one that called himself "Merlin’. But, in her foolishness, the woman didn’t keep her revelation for herself until she was safe. She instead confronted Merlin. Unsurprisingly, Merlin silenced her before she could reveal his identity.