The most related work to ours [1]}, [2]} also study
the problem of estimating the bias of predictor \(f\)  of \(Y\)  from \(X\)  as a function of \(A\)  with few or no samples from the joint distribution over \((X, Y, A)\) .
In [1]} the authors aim to estimate the disparity of a model \(f\)  predicting \(Y\)  from \(X\) , along \(A\)  by considering predicting \(A\)  from \(X\)  via a threshold \(q\) , and then assuming \(A=a\)  whenever \(P[A = a | X = x] > q\) . This can work well with unlimited data from the joint distribution over \(A, X\) , but can both overestimate and underestimate the bias because of the misclassification of \(A\)  using said thresholding rules.
