Imitation learning learns a policy from a dataset of expert demonstrations via supervised learning.
The expert is a policy that acts in a (reward-free) Markov decision process (MDP) defined by a tuple \(\mathcal {M} = (\mathcal {S}, \mathcal {A}, P(s^{\prime }\mid s,a), P(s_0))\) , where \(\mathcal {S}\)  is the set of states, \(\mathcal {A}\)  is the set of actions, \(P(s^{\prime }\mid s,a)\)  is the transition probability, and \(P(s_0)\)  is a distribution over initial states.
The expert's interaction with the environment produces a trajectory \(\tau = (s_0, a_0, \dots , a_{T-1}, s_T)\) .
The expert may maximize the expectation over some reward function, but this is not necessary (and some tasks cannot be expressed through Markov rewards [1]}).
In the simplest form of imitation learning, a behavioral cloning policy \(\pi _{\eta }(a\mid s)\)  parametrized by \(\eta \)  is learned by minimizing the loss \(- \sum _{s, a \in \mathcal {D}} \log \pi _{\eta }(a\mid s)\) , where \(\mathcal {D}\)  is the dataset of state-action pairs collected by the expert's policy.
