Causal Bandits.
There have been a few research works of studying how to learn optimal interventions sequentially by representing the relationship between interventions and outcomes as a causal graph along with associated conditional distributions. [1]} introduced the causal bandit problems in which interventions are treated as arms in a bandit problem but their influence on the reward, along with any other observations, is assumed to conform to a known causal graph. Specifically they focus on the setting that observations are only revealed after selecting an intervention (and hence the observed features cannot be used as context) and the distribution of the parents of the reward is known under those interventions.
[2]} developed a way to choose an intervention subset based on the causal graph structure as a brute-force way to apply standard bandit algorithms on all interventions can suffer huge regret.
[3]} studied a relaxed version of the structural causal bandit problem when not all variables are manipulable. [4]} considered best intervention identification via importance sampling. Instead of forcing a node to take a specific value, they adopted soft intervention that changes the conditional distribution of a node given its parent nodes. [5]} proposed two algorithms, causal upper confidence bound (C-UCB) and causal Thompson Sampling (C-TS), and showed that they have improved cumulative regret bounds compared with algorithms that do not use causal information. They focus on causal relations among interventions and use causal graphs to capture the dependence among reward distribution of these interventions.
