In the Cooperative stage of Kohonen's SOM, the neighborhood for a winning neuron(output node). In most cases, the neighborhood function happens to be the Gaussian Function.For example,$$h_j,_i = exp(-d_j,_i^2/2*\sigma^2)$$where $h_j,_i$ is the window function ($i$ is the index of the winning neuron and $j$ is the index of the encompassing neuron), $d_j,_i$ is the lateral distance between the winning neuron $i$ and the excited neuron $j$.Now the user who is going to implement the SOM has the input data and the randomly initialized weight vectors. How is $d_j,_i$ determined? Is it randomly initialized too? Also what is the relation between the weight vectors of the output nodes and their positions?The figure below is just to give a visual representation of the organization of the input and the output nodes 
How are the positions of the output nodes determined in the Kohonen - Self Organizing Maps algorithm?
machine learning;neural network;clustering;dimensionality reduction;gaussian
The distance is calculated according to a distance function (euclidian, manhattan, mahalanobis and so on. As this is an unsupervised model (hence the self-organizing part in the name) there are no output neurons. Would you clarify the term?I'll just recap the training a bit:Inititalizing: We have a grid (let's assume 2d) of neurons $n_i = (w_i, k_i)$, where $w_i$ is a randomly initialized weight and $k_i$ the position on the grid.We'll now pick a training sample $x_i$ randomly. For this instance $x_i$ we pick the neuron $n_m$ ($m$ for minimum) where the distance between the weight vector and the instance  is minimal given a distance function $d$, so $n_m = \text{argmin}_{n_j} \; d(x_j, W(n_j))$ where $W(\cdot)$ gives the weight for the respective neuron.We now pick a set of neurons for which we will adapt the weight vector given a neighbourhood function (for example your gaussian, or a cone). The weights that should have their weights updated are given by $N^{+t} = \{ n_i = (w_i,k_i) \mid d_A(k_m,k_i) \leq \delta^t \}$, where $t$ is the step in time (I'll mention that later), the reach $\delta^t$ the neighbourhood should have and of course the position on the grid $k_m$ for the winning neuron.We then update the weights of the neurons in the neighbourhood of the winning neuron $n_m$ according to some updating rule. This can be interpreted as moving the weights closer to the input we currently look at (because we chose the neuron with the closest weight vector). A possible update rule could be: $w_s^{t+1} = w_m^t + \epsilon^t \cdot h_{mi}^t \cdot (x_j - w_m^t)$, where $\epsilon^t$ is a time-dependent learning rate and $h_{si}^t$ weights the distance from the winning neuron to the neuron we are changing right now, also time-dependent.In order to somewhat guarantee a topological mapping two things should be considered now:The learning rate, which is part of the update rule, has to be decreased over time, starting with a relatively high value.The neighbourhood radius has to be decreased as well, also starting from a relatively high value.Let's visualize this on this picture real quick. The red nodes are inputs. The dark grey is the winning neuron for the input that is considered right now (the picture does not make it clear that we look at one example/input at a time!). The light grey nodes are the ones that are within the neighbourhood of the winning neuron and hence are updated slightly according to some updating rule.Disclaimer: this was from the top of my head right now, please check any formulas or hypotheses I propose here before using it in any kind of work. (Which you should always do!).See also wikipedia, they also have some explanation of the algorithm that might better suit you.Image sourceAdditioanl source (German Wikipedia)