Interpretable Hidden Markov Model-Based Deep Reinforcement Learning Hierarchical Framework for Predictive Maintenance of Turbofan Engines
An open research question in deep reinforcement learning is how to focus the
policy learning of key decisions within a sparse domain. This paper emphasizes
combining the advantages of inputoutput hidden Markov models and reinforcement
learning towards interpretable maintenance decisions. We propose a novel
hierarchical-modeling methodology that, at a high level, detects and interprets
the root cause of a failure as well as the health degradation of the turbofan
engine, while, at a low level, it provides the optimal replacement policy. It
outperforms the baseline performance of deep reinforcement learning methods
applied directly to the raw data or when using a hidden Markov model without
such a specialized hierarchy. It also provides comparable performance to prior
work, however, with the additional benefit of interpretability.