Learning Optimal Treatment Strategies for Sepsis Using Offline Reinforcement Learning in Continuous Space
Sepsis is a leading cause of death in the ICU. It is a disease requiring
complex interventions in a short period of time, but its optimal treatment
strategy remains uncertain. Evidence suggests that the practices of currently
used treatment strategies are problematic and may cause harm to patients. To
address this decision problem, we propose a new medical decision model based on
historical data to help clinicians recommend the best reference option for
real-time treatment. Our model combines offline reinforcement learning with
deep reinforcement learning to address the problem that traditional
reinforcement learning in healthcare cannot interact with the environment,
enabling our model to make decisions in a continuous state-action space. We
demonstrate that, on average, the treatments recommended by the model are more
valuable and reliable than those recommended by clinicians. In a large
validation dataset, we found that patients whose actual doses from clinicians
matched the AI's decisions had the lowest mortality rates. Our model provides
personalized, clinically interpretable treatment decisions for sepsis that can
improve patient care.