1. A computer-implemented system comprising: one or more computers; one or more data storage devices in data communication with the one or more computers, storing: a training data repository that includes client training data comprising a first plurality of training data sets belonging to a client entity and received over a network; a plurality of training functions; and instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising: generating a plurality of trained predictive models using the plurality of training functions and a first sample of the client training data; determining a respective accuracy of each of the plurality of trained predictive models using a different, second sample of the client training data; receiving, over the network one or more new training data sets belonging to the client entity, wherein each of the one or more new training data sets is new relative to the first plurality of training data sets; updating the client training data to include the one or more new training data sets; generating a plurality of new trained predictive models using the plurality of training functions and a different, third sample of the client training data; determining, a respective accuracy of each of the plurality of new trained predictive models using a different, fourth sample of the client training data; generating a respective effectiveness score for each of the plurality of trained predictive models and each of the plurality of new trained predictive models using the determined accuracy of its respective trained predictive model; receiving, over the network from a client computing system, a first prediction request and first input data; selecting a first trained predictive model to service the first prediction request from among the plurality of trained predictive models and the plurality of new trained predictive models based on the respective effectiveness scores; running the first trained predictive model on the first input data to generate a predictive output; and providing, to the client computing system, the predictive output in response to the first prediction request.