On Efficient Approximate Queries over Machine Learning Models
The question of answering queries over ML predictions has been gaining
attention in the database community. This question is challenging because the
cost of finding high quality answers corresponds to invoking an oracle such as
a human expert or an expensive deep neural network model on every single item
in the DB and then applying the query. We develop a novel unified framework for
approximate query answering by leveraging a proxy to minimize the oracle usage
of finding high quality answers for both Precision-Target (PT) and
Recall-Target (RT) queries. Our framework uses a judicious combination of
invoking the expensive oracle on data samples and applying the cheap proxy on
the objects in the DB. It relies on two assumptions. Under the Proxy Quality
assumption, proxy quality can be quantified in a probabilistic manner w.r.t.
the oracle. This allows us to develop two algorithms: PQA that efficiently
finds high quality answers with high probability and no oracle calls, and PQE,
a heuristic extension that achieves empirically good performance with a small
number of oracle calls. Alternatively, under the Core Set Closure assumption,
we develop two algorithms: CSC that efficiently returns high quality answers
with high probability and minimal oracle usage, and CSE, which extends it to
more general settings. Our extensive experiments on five real-world datasets on
both query types, PT and RT, demonstrate that our algorithms outperform the
state-of-the-art and achieve high result quality with provable statistical
guarantees.