We used a 5x5 grid world Pac-Man to evaluate our approach because it is used in other Policy Shaping papers [1]}, [2]}.
The goal is to eat all the food pellets without being caught by ghosts. Once clear the game, +500 reward is given while -500 reward is given if Pac-Man is caught by ghosts. Also, each pellet awards +10 points, and each time step costs 1 point (rewarded -1 point.)
The state representation includes Pac-Man's position, the position and orientation of the ghost, and each food pellet's presence.
Similar to [1]}, we also use Oracle to simulate human feedback because it allows us to sweep parameters of feedback likelihood (\(L\) ) that specifies how often a trainer gives feedback and consistency level (\(C\) ). The Oracle was created by using Q-learning[4]} on the environment before the experiments. We also use Q-Learning as the underlying RL algorithm. Further details for the experiment are in Appendix REF .
