How find this integral $\int\frac{\sqrt{x-1}\arctan{(x\sqrt{x-1})}}{x}dx$ Find this integral
$$\int\dfrac{\sqrt{x-1}\arctan{(x\sqrt{x-1})}}{x}dx$$

My try: let 
$$\arctan{(x\sqrt{x-1})}=t$$
and that's very ugly,Thank you
This problem is not really very hard yet it is messy since one needs to deal with roots of third order polynomials. Yet all what is involved in here is one substitution, then integration by parts and then decomposition into partial fractions. We have:
\begin{equation}
\int \frac{\sqrt{x-1}}{x} \arctan\left[ x \sqrt{x-1}\right] dx = 
\int\frac{2 u^2}{1+u^2} \arctan\left[ u(1+u^2)\right] du
\end{equation}
where we substituted for $u=\sqrt{x-1}$. Now we integrate by parts:
\begin{eqnarray}
\int\frac{2 u^2}{1+u^2} \arctan\left[ u(1+u^2)\right] du = 2(u-\arctan(u)) \cdot \arctan\left[ u(1+u^2) \right] - 2 \int\frac{u-\arctan(u)}{1+(1+u^2)^2 u^2} (1+3 u^2) du
\end{eqnarray}
where we used the fact that $\int (2 u^2)/(1+u^2) du = 2(u-\arctan(u))$.

Now there are two integrals to be computed . 

Let us start from the easier one:
\begin{eqnarray}
&amp;&amp;\int \frac{u(1+3 u^2)}{1+(1+u^2)^2 u^2} du =\\
&amp;&amp; u \arctan\left[ u(1+u^2)\right] - \int \arctan\left[ u(1+u^2) \right]du=\\
&amp;&amp; u \arctan\left[ u(1+u^2)\right] -\frac{\pi}{2} u- \frac{1}{2 \imath} \sum\limits_{\xi=1}^3 \log(u-u^{(+)}_\xi) (u-u^{(+)}_\xi)+ \frac{1}{2 \imath} \sum\limits_{\xi=1}^3 \log(u-u^{(-)}_\xi) (u-u^{(-)}_\xi)
\end{eqnarray}
where in the last line we used the identity $\arctan(x)= 1/(2 \imath) \log[(1+\imath x)/(1-\imath x)]$. 

Here $\left\{ u^{(+)}_\xi\right\}_{\xi=1}^3$ are roots of the equation $u^3+u-\imath=0$ and $\left\{ u^{(-)}_\xi\right\}_{\xi=1}^3$ are roots of the equation $u^3+u+\imath=0$.

Now the second integral:
\begin{eqnarray}
&amp;&amp;\int\frac{(1+3 u^2) \arctan[u]}{1+(1+u^2)^2 u^2} du=\\
&amp;&amp;\arctan[u(1+u^2)] \arctan[u] - \frac{\pi}{2} \arctan[u] + \frac{1}{4} \sum\limits_{\xi=1}^3 \sum\limits_{\eta \in\{-1,1\}} \sum\limits_{\eta_1 \in\{-1,1\}}
F^{(u^{(\eta)}_\xi,-\eta_1 \imath)}(u)\\
\end{eqnarray}
where 
\begin{equation}
F^{(a,b)}(u):=\log(a+u) \log[\frac{x+b}{b-a}] + Li_2\left[ \frac{x+a}{a-b}\right]
\end{equation}

Bringing everything together we have:
\begin{eqnarray}
&amp;&amp;\int\frac{\sqrt{x-1}}{x} \arctan\left[ x \sqrt{x-1}\right] dx=\pi \left( \sqrt{x-1} - \arctan[\sqrt{x-1}]\right)+\\
&amp;&amp;
-\imath \sum\limits_{\xi=1}^3 \left(\log[\sqrt{x-1}-u^{(+)}_\xi](\sqrt{x-1}-u^{(+)}_\xi)-
 \log[\sqrt{x-1}-u^{(-)}_\xi](\sqrt{x-1}-u^{(-)}_\xi)\right)+\\
&amp;&amp;\frac{1}{2} \sum\limits_{\xi=1}^3 \sum\limits_{\eta \in\{-1,1\}} \sum\limits_{\eta_1 \in\{-1,1\}}
F^{(u^{(\eta)}_\xi,-\eta_1 \imath)}(\sqrt{x-1})
\end{eqnarray}

In[1296]:= x =.;
eX = D[Pi Sqrt[x - 1] - Pi ArcTan[Sqrt[x - 1]] - 
     I Sum[Log[
          Sqrt[x - 1] - Root[#^3 + # - I &amp;, xi]] (Sqrt[x - 1] - 
           Root[#^3 + # - I &amp;, xi]) - 
        Log[Sqrt[x - 1] - Root[#^3 + # + I &amp;, xi]] (Sqrt[x - 1] - 
           Root[#^3 + # + I &amp;, xi]), {xi, 1, 3}] + 
     1/2 Sum[eta eta1 (F[-Root[#^3 + # - eta I &amp;, xi], -eta1 I, Sqrt[
          x - 1]]), {xi, 1, 3}, {eta, -1, 1, 2}, {eta1, -1, 1, 2}], 
    x] - Sqrt[x - 1]/x ArcTan[x Sqrt[x - 1]];
x = RandomReal[{0, 10}, WorkingPrecision -&gt; 50];
N[eX, 50]

Out[1299]= 0.*10^-50 + 0.*10^-50 I
$\because$ according to http://integrals.wolfram.com/index.jsp?expr=%28x-1%29%5E%281%2F2%29%2Fx&amp;random=false, $\int\dfrac{\sqrt{x-1}}{x}dx=2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1}+C$

$\therefore\int\dfrac{\sqrt{x-1}\tan^{-1}(x\sqrt{x-1})}{x}dx$

$=\int\tan^{-1}(x\sqrt{x-1})~d(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})$

$=(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})\tan^{-1}(x\sqrt{x-1})-\int(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})~d(\tan^{-1}(x\sqrt{x-1}))$

$\because$ according to http://www.wolframalpha.com/input/?i=d%2Fdx%28arctan%28x%28x-1%29%5E%281%2F2%29%29%29, $\dfrac{d}{dx}(\tan^{-1}(x\sqrt{x-1}))=\dfrac{3x-2}{2\sqrt{x-1}(x^3-x^2+1)}$

$\therefore(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})\tan^{-1}(x\sqrt{x-1})-\int(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})~d(\tan^{-1}(x\sqrt{x-1}))$

$=2\sqrt{x-1}\tan^{-1}(x\sqrt{x-1})-2\tan^{-1}\sqrt{x-1}\tan^{-1}(x\sqrt{x-1})-\int(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})\dfrac{3x-2}{2\sqrt{x-1}(x^3-x^2+1)}dx$

$=2\sqrt{x-1}\tan^{-1}(x\sqrt{x-1})-2\tan^{-1}\sqrt{x-1}\tan^{-1}(x\sqrt{x-1})-\int\dfrac{3x-2}{x^3-x^2+1}dx+\int\dfrac{(3x-2)\tan^{-1}\sqrt{x-1}}{\sqrt{x-1}(x^3-x^2+1)}dx$

$\int\dfrac{3x-2}{x^3-x^2+1}dx$ is just an integral of rational functions and should have close form.

For $\int\dfrac{(3x-2)\tan^{-1}\sqrt{x-1}}{\sqrt{x-1}(x^3-x^2+1)}dx$ ,

Let $u=\sqrt{x-1}$ ,

Then $x=u^2+1$

$dx=2u~du$

$\therefore\int\dfrac{(3x-2)\tan^{-1}\sqrt{x-1}}{\sqrt{x-1}(x^3-x^2+1)}dx$

$=\int\dfrac{2(3u^2+1)\tan^{-1}u}{(u^2+1)^3-(u^2+1)^2+1}du$

Then you can separate it to the terms of $\int\dfrac{\tan^{-1}u}{au+b}du$ or $\int\dfrac{\tan^{-1}u}{pu^2+qu+r}du$ by partial fraction. According to http://pi.physik.uni-bonn.de/~dieckman/IntegralsIndefinite/IndefInt.html, they relate to the polylogarithm function.