An alternative approach is to use imitation learning (also known as behavioral cloning) to mimic an expert-provided policy. A popular option is to use supervised learning to directly learn the expert's action in a state; see [1]}, [2]}, [3]}, [4]} for a few recent examples.
[5]} turn imitation learning into an induced reinforcement learning problem, and then applied an off-the-shelf RL algorithm to learn the expert's policy.
