1. A computer-implemented method comprising: receiving an image; determining an activity being performed in the received image, wherein the activity is configured to be performed on a surface; selecting (i) a particular activity solver, from a set of activity solvers stored in memory, that provides solution data for the activity being performed in the received image and (ii) configuration data associated with the activity being performed in the received image; based on the configuration data associated with the activity being performed in the received image, (i) generating a rectified image of the surface of the activity performed in the received image, and (ii) determining an activity state of the activity being performed in the received image; wherein determining an activity state of the activity being performed in the received image is based on the rectified image in addition to the configuration data associated with the activity being performed in the received image; providing data indicating the activity state to the particular activity solver, selected from the set of activity solvers, that provides solution data for the activity being performed in the received image; in response to providing data indicating the activity state to the particular activity solver, obtaining solution data, from the selected particular activity solver, for the activity being performed in the received image in accordance with the data indicating the activity state provided to the particular activity solver; and providing, for display, a representation of the solution data, obtained from the selected particular activity solver, for the activity being performed in the received image.