I have a very simple feed forward neural network with keras that should learn a sinus. Why is the predictive power so bad and what is generally the best way to pinpoint issues with a network?In the below I have one input neuron, 10 in the hidden laywer and one output. I would expect the network to perform much more accurately.import numpy as npfrom keras.layers import Dense, Activationfrom keras.models import Sequentialx = np.arange(100)y = np.sin(x)model = Sequential([    Dense(10, input_shape=(1,)),    Activation('sigmoid'),    Dense(1),    Activation('sigmoid')])model.compile(loss='mean_squared_error', optimizer='SGD', metrics=['accuracy'])model.fit(x, y, epochs=10, batch_size=1)scores = model.evaluate(x, y, verbose=0)print(Baseline Error: %.2f%% % (100-scores[1]*100))print(model.predict(np.array([.5])))Output  1/100 [..............................] - ETA: 0s - loss: 1.2016 - acc: 0.0000e+00 79/100 [======================>.......] - ETA: 0s - loss: 0.4665 - acc: 0.0127    100/100 [==============================] - 0s - loss: 0.5044 - acc: 0.0100     Baseline Error: 99.00%[[ 0.35267803]]
predict sinus with keras feed forward neural network
tensorflow;keras
Accuracy is a metric meant for classification problems, look at the mean squared error instead. Your network is too small for a highly fluctuating function that you want to learn, if you divide your x by a smaller amount it would be easier to learn. Second of all, adding another layer and having an identity activation at the end will help quite a bit. Also taking batches bigger than 1 will make the gradient more stable. With a 1000 epochs I get to 0.00167 as mean squared error.x = np.arange(200).reshape(-1,1) / 50y = np.sin(x)model = Sequential([Dense(40, input_shape=(1,)),Activation('sigmoid'),Dense(12),Activation('sigmoid'),Dense(1)    ])model.compile(loss='mean_squared_error', optimizer='SGD', metrics=['mean_squared_error'])for i in range(40):    model.fit(x, y, nb_epoch=25, batch_size=8, verbose=0)    predictions = model.predict(x)    print(np.mean(np.square(predictions - y)))The biggest issue is that the signal in your original small dataset is very difficult to learn, you can see this when you plot it, it will just collapse to the mean.