In standard question answering, we do not care how the context is presented to the model, and for the models that achieve state of the art results, e.g. [11,<cite> 2]</cite> , they process the full context before making any decisions. We show that it is possible to modify these models to be incremental while achieving similar performance.