What should you do if you learn after posting a paper to Arxiv that it is not a novel idea? Consider the following scenario: 

Assume you try to  publish a paper in a journal, but before sending the paper, you place the paper on Arxiv. 
Then, the journal rejects the paper because of it lacks novelty due to an improper literature survey. 
Furthermore, Arxiv does not permit permanently deleting papers. 

What can you do?
Leaving the paper on Arxiv will haunt you for all time. 

Should you replace the pdf with a dummy pdf?
The previous versions will be intact and the people searching for her papers comes to know all those flaws.  Is there any way to remove the paper from Arxiv?
Adding something like "After submission of the initial version, it has come to our attention that our main result is a rediscovery of a prior result by XXX (citation)." to the comment section of the arXiv metadata and the frontpage of the article itself should be an elegant way to handle this.