Post-hoc Concept Bottleneck Models
Concept Bottleneck Models (CBMs) map the inputs onto a set of interpretable
concepts (``the bottleneck'') and use the concepts to make predictions. A
concept bottleneck enhances interpretability since it can be investigated to
understand what concepts the model "sees" in an input and which of these
concepts are deemed important. However, CBMs are restrictive in practice as
they require concept labels in the training data to learn the bottleneck and do
not leverage strong pretrained models. Moreover, CBMs often do not match the
accuracy of an unrestricted neural network, reducing the incentive to deploy
them in practice. In this work, we address the limitations of CBMs by
introducing Post-hoc Concept Bottleneck models (PCBMs). We show that we can
turn any neural network into a PCBM without sacrificing model performance while
still retaining interpretability benefits. When concept annotation is not
available on the training data, we show that PCBM can transfer concepts from
other datasets or from natural language descriptions of concepts. PCBM also
enables users to quickly debug and update the model to reduce spurious
correlations and improve generalization to new (potentially different) data.
Through a model-editing user study, we show that editing PCBMs via
concept-level feedback can provide significant performance gains without using
any data from the target domain or model retraining.