Does posting paper online anonymously prevent me from publishing it in journal? I am a high school student and I am trying to publish my first research paper in mathematics.

However, I have already had my full paper posted without author information on the website of another organization. Now this paper (in pdf) can be found by Google. Does this prevent me from publishing the paper in a journal because of plagiarism-check issues?

I can prove that I am the author in the following way: that organization where I posted the paper can provide proof that I uploaded it and that the paper contained no significant plagiarism because it was checked by Turnitin.com's software before upload. The plagiarism-check report and the paper were posted online at the same time. How can I provide a proof like this to a journal? (I have not submitted the paper yet.)

(For lack of experience, I did not know that I should not have done this.)
There are two complicating factors involved here. It may or may not be possible, but it depends on these issues at least. Even if it is possible, there may be some "risk". 

When the "organization" published your paper, did you assign your copyrights to them? It is possible that you did, and also possible that you retained all rights and just gave them a limited license to publish. The outcome is very different for these two situations. What did you sign or implicitly transmit to them to make it possible for them to publish. If they hold copyright or an exclusive license then there are things you cannot do yourself. Copyright law varies widely from place to place but is generally becoming more restrictive. 

The second issue is that many places won't publish anything that has already been published. Many of them actually want you to sign over the copyrights and don't want to be limited by some prior license that you gave to another organization. This is very strict with some of them. But it is a limitation imposed only by their now policies and these vary. It would be a mistake to try to publish with such a journal without informing them of the situation. 

But even if neither of the above considerations apply, you might have difficult claiming that a work is yours. If the original publisher makes the claim then it will be believed (most likely), but if only you make the claim it might be open to dispute unless you are backed up. In an ideal world (not this world, unfortunately) this would be a simple matter. 

Therefore, the place to start is with the original publishing organization. You don't describe its nature. I worry about reputable organizations publishing the work of students anonymously as you have stated unless the is very controversial and you need some form of anonymity for personal protection. But you don't describe the circumstances. 

But is that organization that holds the key. If you ask and they say no, without a legal reason, then you proceed with some repetitional risk. 

An additional complication is that if you simply reuse what you previously published, without citing it properly, you can still be accused of self plagiarism. Normally one cites and quotes ones earlier work, rather than just reusing it. 



By the way, turnitin isn't a sufficient check to prove the absence of plagiarism. Plagiarism is much more complex and subtle than what can be judged by an algorithm.