I'm not gonna play
-Refusal to go along with something. 

-Refusal to participate in an activity.

-What you say to a drunken sailor who challenges you to a duel of some kind.
"That's right, Billy, I'm back and I want YOU!"

"I'm not gonna play. Why not that guy?"

"Because I already had him. THAT'S RIGHT KIDS I already had him!"