Finding smallest vector satisfying an equation with a dot product I was wondering whether you could help me understand a solution to one of the problems from the 6.036 Introduction to Machine Learning from MIT Open Learning Library. The task is to find the smallest vector (with respect to the L2 norm) $\theta$ which satisfies the equation
$$\theta \cdot x = \frac{1}{y}$$
where $x$ is an arbitrary vector (with the same dimension as $\theta$), y is a constant taking either value $+1$ or $-1$ and $\cdot$ is the dot product.
The provided solution is:
$$\theta = \frac{x}{\| x \|^2} \times \frac{1}{y}$$
Even after looking at the solution, I am still not sure how this problem was supposed to be solved. It seems that the solution could be derived using manipulations similar to:

*

*$\theta \times (x \cdot x) = x \times {1 \over y}$

*$\theta \times \| x \| ^ 2 = x \times {1 \over y}$

*$\theta = {x \over \| x \| ^ 2} \times {1 \over y}$
However, I struggle to see what could be the justification for transforming the original equation to the form in the first step, so I suspect this might not be the right way.

$ \theta \cdot x = | \theta | | x | \cos \psi = \dfrac{1}{y}$
where $\psi$ is the angle between the vector $x$ and the vector $\theta$.
Since $x$ and $y$ are fixed, this implies the minimum $| \theta |$ occurs when
$\cos \psi $ is maximum, and this occurs when $\psi = 0$ so that $\cos \psi = 1$.
And with this, $\theta$ will be aligned with $x$ , i.e. along $x$.
Thus $\theta = \alpha x$ , for a certain $\alpha \gt 0$ that we're about to find.
Plug this expression for $\theta$ in the original equation, you get,
$\theta \cdot x = \alpha x \cdot x = \dfrac{1}{y} $
Hence $\alpha = \dfrac{1}{x \cdot x } \times \dfrac{1}{y} $
Finally we note that $x \cdot x = | x |^2 $
This gives the answer mentioned in the lecture.
