What is the motivation for analytic solutions in Mathematical Physics? I am trying to understand why one cares about solving PDE's with an analytic/theoretical solution when one can use numerical methods?
If you tell me, "only mathematicians try to find theoretical solutions and understand them", I can live with that, after all that is part of what mathematics is all about. But it seems that physicists and engineers also care about theoretical solutions. What is their motivation? 
To expand my question, consider any application involving Bessel functions. Even the simplest PDE will lead to some nasty series. When one has an actual PDE, or something real, it will be a lot messier, so I doubt that anyone building something specific based on the Bessel series will work with it analytically. 
What is to be gained by solving the PDE for a vibrating membrane? Is it because the theoretical solutions imply certain physical laws  that govern the process? Perhaps this is what the numerical approach is missing. 

The point of PDE theory is not to find solutions to individual differential equations, any more than the point of real analysis is to compute integrals. Here are two more specific examples:
1) What sorts of conditions can we impose to ensure that the solution is actually smooth, rather than merely having derivatives up to $k$th order? That isn't something that can be observed from numeric solutions, and it's a tricky problem in general. On the other hand, elliptic regularity gives convenient conditions for smoothness with mild assumptions. One of the most common elliptic operators is the Laplacian, and so there are various applications in mathematical physics. 
2) De Rham cohomology on a smooth (oriented, closed) manifold reflects its topological properties, despite simply being a system of differential equations. The dimension of the solution space has huge topological significance, but writing down those exact solutions is not particularly useful (or feasible) in most cases. There are also many generalizations of this idea, e.g., the Atiyah-Singer index theorem.
