Unlike the traditional information bottleneck methods [1]}, [2]} that uses a fixed \(\beta \) , we follow the suggestion of  [3]} to adaptively update \(\beta ^S/\beta ^T\)  via dual gradient descent. The motivation behind is intuitive: the more bias should be given to the feature purification when the encoded information overflow the bottleneck and vice versa, to enforce a specific constraint \(I_c\)  on the mutual information. Specifically, we train the network to minimax the overall loss \(\mathcal {L}_{Overall}(F,C,D)\)  by alternating between optimizing \(F\) , \(C\) , \(D\) , \(\beta ^S\)  and \(\beta ^T\)  until the loss converges.
