To mitigate this, typically one of a few related strategies are proposed:
policy or learning update constraints preventing deviations
from states and actions well-covered by the dataset or satisfying certain uncertainty bounds
[1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, [9]}, [10]},
pessimism bias to battle value over-estimation [11]}, [12]},
large and diverse datasets to improve state space coverage [13]}, or learned
models to fill in gaps with synthesized data [14]}, [15]}. While many of these enjoy theoretical justification in the tabular or linear cases [16]}, guarantees for the practically relevant non-linear case are mostly lacking.
