Let $f$ be a non-negative function defined on the interval $[0,1].$If $\int_0^x\sqrt{1-(f'(t))^2}dt=\int_0^xf(t)dt,0\leq x \leq 1$ and $f(0)=0,$then Let $f$ be a non-negative function defined on the interval $[0,1].$If $\int_0^x\sqrt{1-(f'(t))^2}dt=\int_0^xf(t)dt,0\leq x \leq 1$ and $f(0)=0,$then
$(A)f(\frac{1}{2})<\frac{1}{2}$ and $f(\frac{1}{3})>\frac{1}{3}$
$(B)f(\frac{1}{2})>\frac{1}{2}$ and $f(\frac{1}{3})>\frac{1}{3}$
$(C)f(\frac{1}{2})<\frac{1}{2}$ and $f(\frac{1}{3})<\frac{1}{3}$
$(D)f(\frac{1}{2})>\frac{1}{2}$ and $f(\frac{1}{3})<\frac{1}{3}$

$\int_0^x\sqrt{1-(f'(t))^2}dt=\int_0^xf(t)dt$
Differentiating both sides wrt $x,$
$\sqrt{1-(f'(x))^2}=f(x)$
Squaring and rearranging,
$(f(x))^2+(f'(x))^2=1$
Differentiating it gives us
$f'(x)=0$ or $f''(x)+f(x)=0$
I am not able to solve it further.

The function $f$ is real-valued by hypothesis. Since $\sqrt{1 - f'(x)^{2}} = f(x)$ for $0 \leq x \leq 1$, the square root on the left is real-valued, i.e., $0 \leq f'(x)^{2} \leq 1$ for $0 \leq x \leq 1$, or
$$
-1 \leq f'(x) \leq 1\qquad 0 \leq x \leq 1.
\tag{1}
$$
Since $f(0) = 0$, the fundamental theorem gives
$$
f(x) = f(x) - f(0) = \int_{0}^{x} f'(t)\, dt,\qquad 0\leq x \leq 1.
\tag{2}
$$
This eliminates three of your four choices. (Why?)
To establish that the remaining choice is true, you need a bit more analysis: $f$ is differentiable by (tacit) hypothesis, hence continuous. Consequently, it is not the case that $f(x) \equiv 1$ (because $f(0) = 0$). It follows that $|f(x)| < x$ for all $0 < x \leq 1$. (Why?)
