Scalable Online Disease Diagnosis via Multi-Model-Fused Actor-Critic Reinforcement Learning
For those seeking healthcare advice online, AI based dialogue agents capable
of interacting with patients to perform automatic disease diagnosis are a
viable option. This application necessitates efficient inquiry of relevant
disease symptoms in order to make accurate diagnosis recommendations. This can
be formulated as a problem of sequential feature (symptom) selection and
classification for which reinforcement learning (RL) approaches have been
proposed as a natural solution. They perform well when the feature space is
small, that is, the number of symptoms and diagnosable disease categories is
limited, but they frequently fail in assignments with a large number of
features. To address this challenge, we propose a Multi-Model-Fused
Actor-Critic (MMF-AC) RL framework that consists of a generative actor network
and a diagnostic critic network. The actor incorporates a Variational
AutoEncoder (VAE) to model the uncertainty induced by partial observations of
features, thereby facilitating in making appropriate inquiries. In the critic
network, a supervised diagnosis model for disease predictions is involved to
precisely estimate the state-value function. Furthermore, inspired by the
medical concept of differential diagnosis, we combine the generative and
diagnosis models to create a novel reward shaping mechanism to address the
sparse reward problem in large search spaces. We conduct extensive experiments
on both synthetic and real-world datasets for empirical evaluations. The
results demonstrate that our approach outperforms state-of-the-art methods in
terms of diagnostic accuracy and interaction efficiency while also being more
effectively scalable to large search spaces. Besides, our method is adaptable
to both categorical and continuous features, making it ideal for online
applications.