Hyperplanes and Support Vector Machines I have the following question regarding support vector machines: So we are given a set of
training points $\{x_i\}$ and a set of binary labels $\{y_i\}$.
Now usually the hyperplane classifying the points is defined as:
$ w \cdot x + b = 0 $
First question: Here $x$ does not denote the points of the training set, but the points on the separating (hyper)plane, right?
In a next step, the lecture notes state that the function:
$f(x) = \mathrm{sign}\, (w \cdot x + b)$
correctly classify the training data.
Second question: Now I don't understand that, since it was stated earlier that $ w \cdot x + b = 0 $, so how can something which is defined to be zero have a sign?
Two additonal questions:
(1) You have added that a slack variable might be introduced for non-linearly separable data - how do we know that the data is not linearly separable, as far as I understand the mapping via kernel has as a purpose to map the data into a vector space where in the optimal case it would be linearly separable (and why not using a non-linear discriminant function altogether instead of introducing a slack variable?)
(2) I've seen that for the optimization only one of the inequalities $w \cdot x = b \geqslant 1$ is being used as a linear constrained - why?

To answer both questions at once: The set $\{x \in \mathbb R^n \mid w'x+b=0\}$ defines the separating hyperplane. This hyperplane separates the space $\mathbb R^n$ into two halfspaces $H_+ = \{x \in \mathbb R^n \mid w'x+b>0\}$ and $H_- = \{x \in \mathbb R^n \mid w'x+b<0\}$.
Thus, for an unknown example $x$ computing $f(x)= sign(w'x+b)$ tells you which halfspace (defined by the hyperplane) the example lies in and thus which label should be assigned.
