Finding smallest vector satisfying an equation with a dot product I was wondering whether you could help me understand a solution to one of the problems from the 6.036 Introduction to Machine Learning from MIT Open Learning Library. The task is to find the smallest vector (with respect to the L2 norm) $\theta$ which satisfies the equation
$$\theta \cdot x = \frac{1}{y}$$
where $x$ is an arbitrary vector (with the same dimension as $\theta$), y is a constant taking either value $+1$ or $-1$ and $\cdot$ is the dot product.
The provided solution is:
$$\theta = \frac{x}{\| x \|^2} \times \frac{1}{y}$$
Even after looking at the solution, I am still not sure how this problem was supposed to be solved. It seems that the solution could be derived using manipulations similar to:

*

*$\theta \times (x \cdot x) = x \times {1 \over y}$

*$\theta \times \| x \| ^ 2 = x \times {1 \over y}$

*$\theta = {x \over \| x \| ^ 2} \times {1 \over y}$
However, I struggle to see what could be the justification for transforming the original equation to the form in the first step, so I suspect this might not be the right way.

GeometryLover's solution is the way that I would solve the problem; it captures the geometric intuition behind the solution and can easily be made fully rigorous.
I want to point out, though, that it's also possible to brute-force the solution by solving the optimization problem exactly as stated, which might prove useful for other, less simple problem of similar flavor:
We are asked to solve
$$\min_{\theta} \|\theta\|^2 \quad \mathrm{s.t.}\quad \theta \cdot x = \frac{1}{y}.$$
This is a convex quadratic program whose unique solution can be found using the method of Lagrange multipliers: the constrained problem is equivalent to
$$\operatorname{ext}_{\theta,\lambda}  \|\theta\|^2 - \lambda\left(\theta\cdot x - \frac{1}{y}\right)$$
whose optimality conditions are
\begin{align*}
2\theta - \lambda x &= 0\\
\theta \cdot x - \frac{1}{y} &= 0.
\end{align*}
The first equations gives you $\theta = \lambda x/2$, and all we need to do is to find $\lambda$, which we can do by plugging into the second equation:
$$\frac{\lambda \|x\|^2}{2} - \frac{1}{y} = 0$$
or
$$\lambda = \frac{2}{y\|x\|^2}$$
which gives
$$\theta = \frac{x}{y\|x\|^2}.$$
