Furthermore, every production deployment of machine learning has to take the cost incurred by model training and inference into account, which often has to be traded off against the business benefits provided by the model [1]}. The time for inference is also crucial, as the response latency of a model has a major impact on its online performance [2]}. We therefore ask the following research question:
