I've written a toy neural network in Java. I ran it several million times with the same outputs with only the randomized weights changing from run to run. The average of all of the outputs is not 0.5, as I would have expected. The code is in this Github Repository.The Layer class:public class Layer extends AbstractLayer {    private double[][] weights;    public Layer(int neurons, int prevLayerNeurons) {        super(neurons);        weights = new double[neurons][prevLayerNeurons + 1];        randomize(weights);    }    protected void randomize(double[][] x) {        for (int j = 0; j < x.length; j++) {            for (int i = 0; i < x[j].length; i++) {                weights[j][i] = Math.random();            }        }    }    @Override    public double[] compute(double inputs[]) {        Preconditions.checkArgument(inputs.length == weights[0].length - 1, incorrect number of inputs);        double[] ret = new double[neurons];        for (int i = 0; i < neurons; i++) {            double acc = 0;            for (int j = 0; j < inputs.length; j++) {                acc += inputs[j] * weights[i][j];            }            acc -= weights[i][weights[i].length - 1];            acc = sigmoid(acc);            ret[i] = acc;        }        return ret;    }    private double sigmoid(double x) {        return 1 / (1 + Math.exp(-x));    }}And the Network class:public class Network {    private List<AbstractLayer> layers;    public Network(int[] neuronCounts) {        layers = new LinkedList<AbstractLayer>();        layers.add(new InputLayer(neuronCounts[0]));        for (int i = 1; i < neuronCounts.length; i++) {            layers.add(new Layer(neuronCounts[i], neuronCounts[i - 1]));        }    }    public boolean[] run(double[] inputs) {        double[] tmps = inputs;        for (int i = 0; i < layers.size(); i++) {            tmps = layers.get(i).compute(tmps);        }        boolean[] rets = new boolean[tmps.length];        for (int i = 0; i < rets.length; i++) {            rets[i] = tmps[i] > 0.5;        }        return rets;    }    public AbstractLayer getLayer(int i) {        return layers.get(i);    }}These are the most relevant classes, but there are several others in the repository. As it stands now, the class Bootstrap will run a 3-layer network with a fixed arrangement and set of inputs.I'd appreciate any comments on the structure of the program, and on the correctness (or incorrectness) of the output.
Simple Java Neural Network
java;machine learning;neural network
Finding the average outputLet's figure out what your average output should be.  Your neural network has 3 inputs in the first layer, 2 nodes in the second layer, and one output.  Each weight is randomized to a value from 0..1, so call it 0.5 on average.The inputs you use in the program are: 1, 0, 1.  On each layer, you also have a bias input of -1.  So with average weights of 0.5, the the input layer will do the following:inputs = 1, 0, 1, -1output = sigmoid(1*0.5 + 0*0.5 + 1*0.5 - 1*0.5) = sigmoid(0.5) = 0.62The second layer has two nodes, each with average input 0.62.  It will do:inputs = 0.62, 0.62, -1output = sigmoid(0.62*0.5 + 0.62*0.5 - 1*0.5) = sigmoid(0.12) = 0.53So your average output should be 0.53.  I modified your program to sum the outputs and found that this was close (actual was 0.528).  Now as for the percentage of time the output is above 0.5, that depends on the distribution of the output, and I don't know that it is easy to compute by hand.  But your program shows through experimentation that the answer is roughly 60% of the time.Random weightsI think that your choice of random weights from the range 0..1 is the source of your confusion.  If you were to choose random weights in the range -1..1 like this:weights[j][i] = Math.random()*2 - 1.0;then your output would be 0.5 on average and the percentage of outputs greater than 0.5 would be 50% (I modified your program to verify this).  Perhaps that is what you were expecting.As far as your neural net code goes, it appears to be correct as far as I can tell.