It is not surprising that, although more limited in applications, linear bandits are much better understood in theory than contextual linear bandits.
Indeed, algorithms for linear bandits often leverage the fixed action space property,
and cannot be easily extended to contextual linear bandits.
To give a concrete example, the algorithm Phased Elimination (PE) [1]}, [2]} leverages the fixed action space by exploring a (small) core set of actions to achieve good estimates of the rewards for all actions. This algorithm achieves a high probability regret bound of \(O(d\sqrt{T\log T})\) .
Nevertheless, despite several attempts over the last decade [3]}, [4]}, [5]}, the best known regret upper bounds for contextual linear bandits have a \(\log \)  (or iterated \(\log \) ) multiplicative gap over the \(O(d\sqrt{T\log T})\)  bound both in high probability and in expectation. Similarly,
the best known algorithms for several linear bandits problems (e.g., with misspecification, adversarial corruption, and others [1]}, [7]}, [8]}, [9]}, [10]}), perform better (in the worst-case) than the corresponding algorithms for contextual linear bandits.
