I'm using the MLPRegressor in sklearn to train a network with approx 1000 inputs and a continuous output variable.  Essentially, the issue is one of image classification (1000 pixels) with the distribution of the pixels related to a continuous variable output.I can generate an arbitrarily large training set, but run into memory problems running this on a single machine when loading arrays into numpy.  I've selected hyper parameters that give a reasonable result with a limited (5k) training set, but am trying to work out the best way to train the network with two orders of magnitude more data.  Is there a way to pass mini-batches to sklearn myself so that I can manage the memory and IO?  Is training and retraining the network helpful if I can setup a means to ensure I read batches for each training step in a randomised fashion? I'm guessing this is a common problem, but I'm struggling to find a sensible solution.
MLP batch iteration in python
python;neural network;scikit learn