However, the increased use of MAB algorithms for conducting
experiments, has been followed by the realisation that using adaptively-collected data for important secondary inferential objectives may be statistically challenging. Most specifically,  traditional inference methods are typically not valid or reliable when used in MAB-collected data. It is well recognized that the sequential dependence induced by adaptive algorithms can lead to considerable biases in classical estimators [1]}, [2]}, [3]}, that persist even in the infinite data limit, with asymptotic distributions differing from the predicted central limit theorem (CLT) behavior [4]}, [5]}. More recently, issues have been detected in hypothesis testing as well [1]}, [7]}, attracting a substantial attention within the machine learning (ML) community [8]}, [9]}, [5]}. These issues include type-I error inflation (i.e., higher than a pre-defined target \(\alpha \) ) and reduced statistical power to reject the null (or equivalently, increased type-II error).
