Proof Check: $x \leq y+ \epsilon$ for all $\epsilon >0$ iff $x \leq y$. Synopsis
I want to be sure I'm utilizing proof by contradiction correctly, so please check my proof of the exercise below. It's relatively simple, so it shouldn't take you too much time.
Exercise

Let $x$ and $y$ be real numbers. Show that $x \leq y + \epsilon$ for all real numbers $\epsilon > 0$ if and only if $x \leq y$.

Proof
Suppose $x \leq y + \epsilon$ for all $\epsilon > 0$ and $x > y$. Then $x - y > 0$ and $x - y + \epsilon > \epsilon$ for all $\epsilon > 0$. But if $x \leq y + \epsilon$, then $y - x + \epsilon \geq 0$. So $0 \leq y-x+\epsilon < y-x+(x-y+\epsilon) = \epsilon$, a contradiction. For the converse, suppose $ x \leq y$. Then it is obvious that $x \leq y+\epsilon$. This concludes our proof.
Update
This proof is obviously wrong. It is not a contradiction that $\epsilon >0$. For some reason, I deluded myself that my conclusion stated that $\epsilon < 0$, but that's just due to my occasional stupidity and habitual lack of double checking. Instead, consider some $\epsilon$ such that $0 < \epsilon < x-y$. Then $x \leq y + \epsilon < y+x-y < x$, a contradiction. Thank you to the various people who commented on the issues with my proof. This was a very stupid mistake, and I don't even know how I overlooked what I did.

No, it is not correct. I tried to prove it by contradiction, and you got the conclusion that 0<\varepsilon, claiming that that's a contradiction. Why? There is no contradiction there.
If $x>y$, let $\varepsilon=\frac12(x-y)$. Then $\varepsilon>0$ and therefore $x\leqslant y+\varepsilon$. But this means that$$x\leqslant y+\frac12(x-y)=\frac12(x+y)<\frac12(x+x)=x.$$This gives you a contradiction: $x>x$.
