For the model updating, we aim to select optimal updating strategies according to model structure changes and human expert feedback. Similarly, the core idea of the Markov Decision Process (MDP) is deciding which action to perform based on state changes in the environment and rewards obtained after performing the action. Thus, to ensure that the identity recognition model can be updated timely and reasonably by human expert feedback, we transform the sequential interactions between the human expert and the identification model as an MDP.
Model structures and parameters are regarded as state \(s\) . And we set five actions for model updating, including Maintain, Increase Density, Decrease Density, Expand Node, and Collapse Node. Deep Q-learning Network (DQN) [1]} is employed to select the updating strategy \(a\)  for the feedback in each iteration.
The detailed updating process is demonstrated in section REF .
The procedure is iteratively executed to timely improve the model performance without retraining the model.
