I'm building a neural net for classifying characters in pictures. The input can be any character a-z, lowercase and uppercase. I only care about classifying the characters, and not the case, so the neural net has an output vector of length 26; one for each character. It makes sense, intuitively, to have a hidden layer of size 26*2 just upstream of the output layer. It also makes intuitive sense for this layer not to be fully connected to the output layer, but instead having two and two hidden nodes connect to each output node.I have some questions:a) Does this make sense? I'm getting about 75 % success rate on a pretty hard data set with just one hidden layer, but I'm not certain on how to improve from there.b) If so, What activation function should I use from the hidden layer with 26*2 nodes to the output layer? Maybe I should use an OR function for this, since both the lowercase and the uppercase version of a character should output for a single character.c) Would it be wiser to have 26*2 output nodes instead, and just combine lowercase and uppercase outputs after the neural net?
Character recognition neural net topology/design
machine learning;neural network;deep learning;theano;image recognition
Your design makes some sense, but there is no need to limit connections even if you expect to represent probabilities of upper/lower case separately, because they will interact usefully. E.g if the character could most likely be one of o, O, Q, G then this might be useful information to choose the correct one. If you went ahead, you would need to train this network without the final layer (so that it learns the representations you expect, not some other group of 52 features), then add the final layer later, with no need for special connection rules, just use existing ones. Initially you would training the new layer separately from the full output of the 52-class net i.e. probability values, not selected class. Then you would combine with the existing net and fine-tune the result by running a few more epochs with a low learning rate on the final model.That all seems quite complex, and IMO unlikely to gain you much accuracy (although I am guessing, it could be great - so if you have time to explore ideas, you could still try). Personally I would not take your hidden layer idea further. The full 52-class version with simple logic to combine results is I think simpler. This is also not necessary, the neural net can learn to have two different-looking images be in the same class quite easily, provided you supply examples of them in training. However, it may give you useful insights into categorisation failures in training or testing.It is not clear from the question, but if you are not already using convolutional neural network for lower layers, then you should do so. This will make the largest impact on your results by far.