Derivative of sigmoid function that contains vectors Could someone show me how to take the derivative of this function with respect to $w_i$?
$f(w) = \frac{1}{1+e^{-w^Tx}}$
$w$ and $x$ are both vectors $\in \mathbb{R}^D$
How would this be different from taking the derivative with respect to $w$ itself?

With
$x = (x_1, x_2, \ldots, x_n)^T \tag 1$
and
$w = (w_1, w_2, \ldots, w_n)^T, \tag 2$
we have
$w^Tx = \displaystyle \sum_1^n w_i x_i; \tag 3$
we observe that
$\dfrac{\partial (w^Tx)}{\partial w_j} = x_j, \; 1 \le j \le n; \tag 4$
we may write
$f(w) = \dfrac{1}{1 + e^{-w^Tx}} = (1 + e^{-w^Tx})^{-1}, \tag 5$
and deploy the chain rule:
$\dfrac{\partial f(w)}{\partial w_j} = \dfrac{df(w)}{d(w^Tx)} \dfrac{\partial (w^Tx)}{\partial w_j}$
$= -(1 + e^{-w^Tx})^{-2} (-e^{-w^Tx}) \dfrac{\partial (w^Tx)}{\partial w_j} = (1 + e^{-w^Tx})^{-2} (e^{-w^Tx}) x_j = \dfrac{e^{-w^Tx} x_j}{(1 + e^{-w^Tx})^{-2}}. \tag 6$
The above shows how to form the derivatives with respect to the individual $w_i$; to take the derivative with respect to $w$, we again call on the chain rule, but rather than (6) we have
$\dfrac{\partial f(w)}{\partial w} = \dfrac{df(w)}{d(w^Tx)} \dfrac{\partial (w^Tx)}{\partial w} = (1 + e^{-w^Tx})^{-2} (e^{-w^Tx})\dfrac{\partial (w^Tx)}{\partial w}, \tag 7$
where it remains to evaluate
$\dfrac{\partial (w^Tx)}{\partial w}; \tag 8$
but this is straightforward; we form the difference
$(w + \Delta w)^T x - w^Tx = w^Tx + \Delta w^T x - w^Tx = \Delta w^T x, \tag 9$
whence
$(w + \Delta w)^T x - w^Tx - \Delta w^T x = 0, \tag{10}$
which yields
$\Vert (w + \Delta w)^T x - w^Tx - \Delta w^T x \Vert = 0, \tag{11}$
independently of $\Vert \Delta w \Vert$; since it then follows that
$\dfrac{\Vert (w + \Delta w)^T x - w^Tx - \Delta w^T x \Vert}{\Vert \Delta w \Vert} = 0, \; \forall \Delta w \ne 0, \tag{12}$
we may conclude that the linear map
$\Delta w \to \Delta w^T x \tag{13}$
is the sought-for derivative (8); it follows then that (7) may be written
$\dfrac{\partial f(w)}{\partial w} = \dfrac{df(w)}{d(w^Tx)} \dfrac{\partial (w^Tx)}{\partial w} =  (1 + e^{-w^Tx})^{-2} (e^{-w^Tx})(\cdot)^Tx; \tag{14}$
we note that this is the linear mapping from $\Bbb R^n \to \Bbb R$ given by
$\Delta w \mapsto \dfrac{ e^{-w^Tx}}{ (1 + e^{-w^Tx})^2} \Delta w^T x. \tag{15}$
We may cast this in a somewhat more normative form via the observation that, since $\Delta w^T x \in \Bbb R$ is a scalar quantity, 
$\Delta w^T x = (\Delta w^T x)^T = x^T (\Delta w^T)^T = x^T \Delta w, \tag{16}$
and therefore (15) becomes
$\Delta w \mapsto \dfrac{ e^{-w^Tx}}{ (1 + e^{-w^Tx})^2} x^T \Delta w, \tag{17}$
so we at last find
$\dfrac{\partial f(w)}{\partial w} = \dfrac{df(w)}{d(w^Tx)} \dfrac{\partial (w^Tx)}{\partial w} = \dfrac{ e^{-w^Tx}}{ (1 + e^{-w^Tx})^2} x^T, \tag{18}$
is the derivative we seek.
