The misuse of Dirac's $\delta$ In physics and engineering it is common practice to use the Dirac delta distribution to represent "densities" of discrete random variables. It is a very useful construct and you can do many things with it easily. 

$$f_{\pmb x}(x)=\sum_{n=1}^{\infty}P(\pmb x = x_n) \,\delta(x-x_n) \;=\; \sum_{n=1}^{\infty}p_n \,\delta(x-x_n)$$

$$E_{f_{\pmb x}}\{\delta(g(\pmb x))\} \;=\; \sum_i \frac{f_{\pmb x}(x_i)}{{|g'(x_i)|}}\enspace, \quad \text{ with } g(x_i)=0 \text{ and } g'(x_i)\neq 0$$

$$\pmb y = g(\pmb x) \quad\Rightarrow \quad
  f_{\pmb y}(y) \;=\; E_{f_{\pmb y}}\{\delta(\pmb y - y)\} \;=\; E_{f _{\pmb x}}\{\delta(g(\pmb x)-y)\}$$

But the mathematicians always say it would be mathematically objectionable or even incorrect, because a density function made up of delta distributions is not continuous and not integrable. However the definition of the delta distribution precisely defines its integral. So what is the problem here? 

Is there an example where the use of the Dirac delta function can lead do wrong results?
What the mathematicians are saying is not that the Dirac delta function is not continuous or integrable which requires first the object under discussion be an $\mathbf R\to\mathbf R$ function, but that it is not even an $\mathbf R\to\mathbf R$ function. However, the Dirac delta function is rigorously defined, only not as an $\mathbf R\to\mathbf R$ function but a class of linear functionals, which is a linear function from a function space into the set of real (complex) numbers $\mathbf R$,  called distribution or generalized function.
I am very happy to consider $\delta$ as a function, without having to mess with functional analysis that drives intuition away. Until I stumble upon the square $(\delta(x))^2$, which is a suspicious object: indeed, a single Internet search reveals trouble. $^{[1]}$

There is also another issue: if $\phi\colon \mathbb R^n\to \mathbb R$, I would be very happy to consider $\delta(\phi(x_1\ldots x_n))$. That's another case in which the $\delta$ "function" does not behave like a function at all. 



Note [1]. At a more down-to-earth level, Note 2 in this answer shows that $\delta$ cannot be considered as an element of $L^2(\mathbb R)$, not even in a "weak" sense, so $\delta(x)^2$ should have an infinite integral, to begin with.