Can I use my powers for good? I hesitate to ask this question, but I read a lot of the career advice from MathOverflow and math.stackexchange, and I couldn't find anything similar. 

Four years after the PhD, I am pretty sure that I am going to leave academia soon. I do enjoy teaching and research, but the alpha-maleness, massive egos and pressure to publish are really unappealing to me, and I have never felt that I had the mathematical power to prove interesting results. However, I am really having trouble thinking of anything else to do. Most people seem to think that the main careers open to mathematicians are in banking and finance. I really want to work in some field where I can use mathematics, but it is also important to me to feel like I am contributing something positive or at least not actively doing harm. For this reason, financial speculation is very unappealing to me, although I do find the underlying mathematics quite fascinating.

Here is my question: what careers which make a positive contribution to society
might be open to academic mathematicians who want to change careers?
If you are in the US, there are several thousand institutions of higher learning, and at many of them there is very little "pressure to publish". At others, the "pressure to publish" can be met by publishing a textbook or some work of scholarship that does not require proofs of interesting (original) results. High schools also need qualified Mathematics teachers. Consider staying in academia, just moving to a different part of it, as an option for using your powers to do good. 

I suspect, but cannot be sure, that much of what I've written applies outside the US as well.
The question you seem to be asking is "Can I be paid to do research, without any formal education, if I am very good?" The likely answer is no.  I know many serious researchers across countless fields, and the only ones I am aware of who can make a living without any formal training are inventors who got quite lucky in terms of their ideas and their marketability. Many great artists, mathematicians, and scientists did other things to survive while doing the work they were truly passionate for, and we romanticise many very inspirational people. Research is probably much more a function of diligence and persistence, even when things are depressingly boring, than it is of intelligence or genius. I would suggest reading some of the good bits of advice many modern prodigies have given on the subject. Terence Tao in particular has several good pieces on accelerated education, working hard, and being a genius.

Yes, school can be terribly dull if you are quick. But being able to succeed in that environment is an invaluable human skill, which will prove useful in a research career when looking for funding, or when setting up research groups and similar things, as well as really showing that you have a sense of humility. We can complain about how slow and tedious bureaucracy can be, but if you really are quite good, it is worth your time to learn and understand how to survive in it. Mathematics is not simply something devoid of social interaction or human involvement. Pure and applied mathematics stem from human ideas and are inherently connected to social and cultural concepts, and these connections are often a part of what traditional formal education gives us. I would absolutely advise you to go further than what you see in a classroom, do independent and guided research as early as possible, and test out of whatever classes you can. But skipping the process entirely will handicap your ability to contribute meaningfully to mathematical research.