Find a particular solution $y_p$ for $5y''+8y'+8y=\cos^{2}(x)$ I'm getting $A = \frac{-3}{68}$ and $B = \frac{1}{34}$ for the guess $y_p = A\cos(2x)+B\sin(2x)$ using the method of undetermined coefficients.  Apparently this answer is wrong (with $A$ and $B$ plugged in).  Can anyone help me find the right one?

Thanks!
Note that the right hand side is ${\cos ^2}(x)$, so want to choose something similar as our particular solution. But $\cos (2x) = {\cos ^2}(x) - {\sin ^2}(x) = 2{\cos ^2}(x) - 1$ so
$${\cos ^2}(x) = \frac{1}{2} + \frac{1}{2}\cos (2x)$$
and we can look for a particular solution of the form
$${y_p} = A + B\sin (2x) + C\cos (2x).$$

Then you need to use the initial conditions to determine the coefficients or plug ${y_p}$ into the differential equation, collect like terms, and equate the coefficients.
See below for the details. Your equation is

$$\left( {5{D^2} + 8D + 8} \right)y = {\cos ^2}\left( x \right)$$

Make that into 

$$\left( {5{D^2} + 8D + 8} \right)y = \frac{{1 + \cos \left( {2x} \right)}}{2}$$

and apply $D$

$$\eqalign{
  &amp; \left( {5{D^2} + 8D + 8} \right)y = \frac{{1 + \cos \left( {2x} \right)}}{2}  \cr 
  &amp; \left( {5{D^2} + 8D + 8} \right)y' =  - \sin \left( {2x} \right)  \cr 
  &amp; \left( { - 5{D^2} - 8D - 8} \right)y' = \sin \left( {2x} \right) \cr} $$

Thus, we have $$\eqalign{
  &amp;  - 5{D^2} - 8D - 8 = \left( { - 5{D^2} - 8} \right) - 8D  \cr 
  &amp; \varphi \left( D \right) =  - 5D - 8  \cr 
  &amp; \xi \left( D \right) =  - 8 \cr} $$

and then 

$$y' = \frac{{3\sin 2x + 4\cos 2x}}{{100}}{\text{ }}$$

and you solution is $$y = \frac{{ - 3\cos 2x + 4\sin 2x}}{{200}} + C{ _1}$$

I leave it to you to find $C_1$.



I know the following is not completely rigorous, but it has its justifications. 

As a generalization of Tapu's solution, suppose we're given

$$\phi(D)=\sum_{k=0}^m a_kD^k$$

where $D=\dfrac d {dx}$. Since $D^2\sin(ax)=-a^2\sin (ax)$, we have that

$$\phi(D^2)\sin(ax)=\phi(-a^2)\sin(ax)$$

But we may write 

$$\phi (D) = \sum\limits_{k = 0}^{m'} {{a_{2k}}{D^{2k}}}  + \sum\limits_{k = 0}^{m'} {{a_{2k + 1}}{D^{2k + 1}}} $$

whence

$$\phi (D) = \varphi \left( {{D^2}} \right) + D\xi \left( {{D^2}} \right)$$

thus an equation of the form

$$\phi (D)y = \sin ax$$

becomes $$\left[ {\varphi \left( {{D^2}} \right) + D\xi \left( {{D^2}} \right)} \right]y = \sin ax$$

We apply the "conjugate" of the operator to the equation, to get $$\eqalign{
  &amp; \left[ {{\varphi ^2}\left( {{D^2}} \right) - {D^2}{\xi ^2}\left( {{D^2}} \right)} \right]y = \left[ {\varphi \left( {{D^2}} \right) - D\xi \left( {{D^2}} \right)} \right]\sin ax  \cr 
  &amp; \left[ {{\varphi ^2}\left( {{D^2}} \right) - {D^2}{\xi ^2}\left( {{D^2}} \right)} \right]y = \varphi \left( { - {a^2}} \right)\sin ax - a\xi \left( { - {a^2}} \right)\cos ax  \cr 
  &amp; \left[ {{\varphi ^2}\left( {{D^2}} \right) - {D^2}{\xi ^2}\left( {{D^2}} \right)} \right]y = \alpha \sin ax + \beta \cos ax  \cr 
  &amp; \Phi \left( {{D^2}} \right)y = \alpha \sin ax + \beta \cos ax \cr} $$

Assuming  $$\Phi \left( { - {a^2}} \right) \ne 0$$ we may write $$\frac{{\sin ax}}{{\Phi \left( { - {a^2}} \right)}} = \frac{{\sin ax}}{{\Phi \left( {{D^2}} \right)}}$$ so that

$$\eqalign{
  &amp; y = \frac{{\alpha \sin ax + \beta \cos ax}}{{\Phi \left( { - {a^2}} \right)}}  \cr 
  &amp; y = \frac{{\varphi \left( { - {a^2}} \right)\sin ax - a\xi \left( { - {a^2}} \right)\cos ax}}{{{\varphi ^2}\left( { - {a^2}} \right) + {a^2}{\xi ^2}\left( { - {a^2}} \right)}} \cr} $$

For the cosine, you get $$y = \frac{{\varphi \left( { - {a^2}} \right)\cos ax + a\xi \left( { - {a^2}} \right)\sin ax}}{{{\varphi ^2}\left( { - {a^2}} \right) + {a^2}{\xi ^2}\left( { - {a^2}} \right)}}$$

As an example, consider

$$(D^2-3D+2)y=\sin 3x$$

Then $\phi(D)=D^2-3D+2=D^2+2-3D=\varphi(D^2)-D \xi(D^2)$ where $\varphi(D)=D+2$ and $\xi(D)=-3$. Thus $$y = \frac{{ - 7\sin 3x - 3\left( { - 3} \right)\cos 3x}}{{49 + 9{{\left( { - 3} \right)}^2}}} = \frac{{9\cos 3x - 7\sin 3x}}{{130}}$$