Podrick Payne was rather shit at fighting, but he could get better with practice. He was stubborn and loyal enough, despite or maybe because he was from a branch of House Payne. After the trial of Tyrion Lannister, King's Landing wasn't safe for him anymore. Podrick was the page of a known kinslayer and kingslayer, so Jaime had him sent away with Brienne of Tarth.