How to determine the optimal control law? 
Given the differential equation
$$\dot x = -2x + u$$
determine the optimal control law $u = - kx$ that minimizes the performance index
$$J = \int_0^{\infty} x^2 \, \mathrm d t$$

My approach was to find the state feedback $k$. But since the value of $R$ (positive semidefinite Hermitian) is not given, that means $R=0$. How do I determine the optimal control for this system where $R=0$?

Using the state feedback law $u = - \kappa \, x$, where $\kappa$ is to be determined, we obtain
$$\dot x =  -(\kappa + 2) \, x$$
Integrating,
$$x (t) = \exp \left( -(\kappa + 2) t \right) \, x_0$$
where $x_0$ is the initial condition. Hence,
$$\int_0^{\infty} \left( x (t) \right)^2 \, \mathrm d t = \cdots = \dfrac{x_0^2}{2 (\kappa + 2)}$$
where the integral converges if $\kappa > -2$. Note that there is no minimum, but
$$\lim_{\kappa \to \infty} \dfrac{x_0^2}{2 (\kappa + 2)} = 0$$
