Using $dx$ for $h$. Is it mathematically correct to write
$$f'(x)=\lim_{dx\to0}\frac{f(x+dx)-f(x)}{dx},$$
rather than
$$f'(x)=\lim_{h\to0}\frac{f(x+h)-f(x)}{h}?$$
If not, what is the difference?  If so, why isn't this notation used from the beginning?  My feeling for the latter is that it would align the derivative more with the inverse of the indefinite integral.

No it doesn't matter, as pointed out what you write doesn't matter, as long as the concept gets across, $dx$, $f$, $john$ $\oplus$, you can use any of them, $h$ is just the common one and used because everyone recognises it but in certain context, to make clear, you must use other letters as you're doing more than one thing at once.
