If $f^2$ is differentiable, how pathological can $f$ be? Apologies for what's probably a dumb question from the perspective of someone who paid better attention in real analysis class.
Let $I \subseteq \mathbb{R}$ be an interval and $f : I \to \mathbb{R}$ be a continuous function such that $f^2$ is differentiable.  It follows by elementary calculus that $f$ is differentiable wherever it is nonzero.  However, considering for instance $f(x) = |x|$ shows that $f$ is not necessarily differentiable at its zeroes.  
Can the situation with $f$ be any worse than a countable set of isolated singularities looking like the one that $f(x) = |x|$ has at the origin?

To expand on TZakrevskiy's answer, we can use one of the intermediate lemmas from the proof of Whitney extension theorem.

Theorem (Existence of regularised distance) Let $E$ be an arbitrary closed set in $\mathbb{R}^d$. There exists a function $f$, continuous on $\mathbb{R}^d$, and smooth on $\mathbb{R}^d\setminus E$, a large constant $C$, and a family of large constants $B_\alpha$ ($C$ and $B_\alpha$ being independent of the choice of the function $f$) such that

*

*$C^{-1} f(x) \leq \mathrm{dist}(x,E)\leq Cf(x)$

*$|\partial^\alpha f(x)| \leq B_\alpha~ \mathrm{dist}(x,E)^{1 - |\alpha|}$ for any multi-index $\alpha$.


(See, for example, Chapter VI of Stein's Singular Integrals and Differentiability Properties of Functions.)
Property 1 ensures that if $x\in \partial E$ the boundary, $f$ is not differentiable at $x$. On the other hand, it also ensures that $f^2$ is differentiable on $E$. Property 2, in particular, guarantees that $f^2$ is differentiable away from $E$.
So we obtain

Corollary Let $E\subset \mathbb{R}^d$ be an arbitrary closed set with empty interior, then there exists a function $f$ such that $f^2$ is differentiable on $\mathbb{R}^d$, $f$ vanishes precisely on $E$, and $f$ is not differentiable on $E$.

