Adaptive Fairness-Aware Online Meta-Learning for Changing Environments
The fairness-aware online learning framework has arisen as a powerful tool
for the continual lifelong learning setting. The goal for the learner is to
sequentially learn new tasks where they come one after another over time and
the learner ensures the statistic parity of the new coming task across
different protected sub-populations (e.g. race and gender). A major drawback of
existing methods is that they make heavy use of the i.i.d assumption for data
and hence provide static regret analysis for the framework. However, low static
regret cannot imply a good performance in changing environments where tasks are
sampled from heterogeneous distributions. To address the fairness-aware online
learning problem in changing environments, in this paper, we first construct a
novel regret metric FairSAR by adding long-term fairness constraints onto a
strongly adapted loss regret. Furthermore, to determine a good model parameter
at each round, we propose a novel adaptive fairness-aware online meta-learning
algorithm, namely FairSAOML, which is able to adapt to changing environments in
both bias control and model precision. The problem is formulated in the form of
a bi-level convex-concave optimization with respect to the model's primal and
dual parameters that are associated with the model's accuracy and fairness,
respectively. The theoretic analysis provides sub-linear upper bounds for both
loss regret and violation of cumulative fairness constraints. Our experimental
evaluation on different real-world datasets with settings of changing
environments suggests that the proposed FairSAOML significantly outperforms
alternatives based on the best prior online learning approaches.