Can I use a stamped arxiv paper as a proof of priority if it has been rejected? Sometimes arxiv moderators may not agree with content of a submitted manuscript. Although it's rare, the submitted manuscript may be rejected even if it contains novel or original results.

During submission the manuscript is stamped by the arxiv submission system with the classification and date of submission.

Can I use a stamped arxiv manuscript as a proof (priority of a physics or math formula, new method or some other idea) if it has been rejected? If yes, how this could be done? Can I publish the stamped paper online and claim it as an official arxiv document?
I recommend against it. You can put online anything you like and use it to claim priority, but if the paper is hosted on your personal website then you lose the credibility that the date stamp of an official arXiv-hosted paper comes with (since just the stamp itself would be easy to forge), so in the event of a priority dispute this would not be a foolproof form of evidence.

Also, as has been discussed in other questions on academia.se (can't look it up right now), people who are new to academic publishing worry excessively about their ideas being stolen by others. Especially given that your manuscript was rejected by arXiv, the chances of this happening are effectively zero. You would be much better off using your energy to write a good manuscript rather than worrying about datestamps.