\(Q\) -function reuse is the process of training an agent on a simple source Markov decision process (MDP) \(\mathcal {M}_S\)  and then transferring its learned \(Q\) -function to a more complex, yet related target MDP \(\mathcal {M}_T\) . The goal is to improve the sample complexity when compared to just training in \(\mathcal {M}_T\)  from the start. Sample complexity is loosely defined as how much data an agent must collect in order to learn a good policy [1]}.
If the agent was trained in \(\mathcal {M}_S\)  until convergence to the optimal policy, transferring the \(Q\) -function from \(\mathcal {M}_S\)  to \(\mathcal {M}_T\)  can sometimes be thought of as a near-optimal \(Q\) -function initialization in \(\mathcal {M}_T\) , since \(\mathcal {M}_S\)  is related to \(\mathcal {M}_T\) .
Therefore, we propose that one method to study \(Q\) -function reuse is to study the effectiveness of near-optimal \(Q\) -function initialization.
