Calculus: Why do we ignore dx? I'm new to calculus and I find it really confusing why we just ignore the dx at the end.

For example, when working on derivation of $x^2$,
at the last step, we're left with 
$f'(x)= 2x + dx$

But I've heard people in videos say: "Since $dx$ is super super super small, we can safely ignore it. But just ignoring it bugs me. If we choose to ignore it, should it not actually look like this: 
$f'(x) \approx 2x $

Thanks for your time. Cheers :)

Warning: I'm about to use the terms "secant" and "tangent" with geometric meanings I'll link to, and not their meanings in trigonometry. I didn't choose the terminology.
For $y=x^2$ the secant from $(x,\,y)$ to a point $(x+\Delta x,\,y+\Delta y)$ with $\Delta x\ne0$ has gradient $\frac{\Delta y}{\Delta x}=2x+\Delta x$. The tangent at $(x,\,y)$ is the line through it whose gradient is the $\Delta x\to0$ limit $2x$ of the secant's gradient $2x+\Delta x$. We are not ignoring the $\Delta x$ term in the secant's gradient; we are interested in the tangent's gradient, which is denoted $\frac{dy}{dx}$. This is not literally a ratio of two quantities $dy,\,dx$, but has this notation because it is a limit of a ratio.
As has been discussed, the fact that $\lim_{\Delta x\to0}\frac{\Delta y}{\Delta x}=\frac{dy}{dx}$ is finite implies the small-$\Delta x$ little-$o$ result $\Delta y\in\frac{dy}{dx}\Delta x+o(\Delta x)$.
