"why didn't she just give the guy a good, 'swift' slap across the face and a thorough verbal dressing down?"

Because that woud have been assault, and opened her up to possible criminal charges and almost certain civil action.  Taylor Swift did the right thing.  She strikes me as a class act.