Is there a formula for solving integrals of this form? I was wondering if there was a substitution formula to solve integrals of this form:
$\int f(g(x))g''(x)dx$

What you can do depends a lot on the form of $g(x)$. I doubt very much that there is a general solution.
However there's two steps that I'd look at doing: (These are only valid if $g(x)$ is smooth and monotonic)
First an integration buy parts, then a substitution.
Taking $u = f(g(x))$ and $dv = g''(x)\;dx$ we'd get that:
$$
\int f(g(x))g''(x)\;dx = f(g(x))g'(x) - \int f'(g(x))g'(x)^2\;dx
$$
Now making the substitution $u = g(x)$ (which will only be valid for some $g(x)$), so that $du = g'(x)\;dx$, we get:
$$
\int f(g(x))g''(x)\;dx = f(g(x))g'(x) - \int f'(u) g'( g^{-1}(u) )\;du
$$
Whether or not this is an improvement will depend on what $g'(g^{-1}(u))$ is like.
