\(\bullet \)  Batch Learning.
In batch learning, instead of observing the reward at the end of each round to decide what action to play next, the learning agent is constrained to split the rounds into a fixed number of \(M\)  batches, use a predetermined policy within each batch, and it can only observe the action outcomes and switch the policy at the end of each batch.
This is a central problem in online learning [1]}, [2]}, [3]}, [4]} as limited policy adaptivity enables parallelism and facilitates deployment of learning algorithms for large-scale models.
