FIELD OF THE DISCLOSURE

The present disclosure relates generally to techniques for supervised machine learning and, more particularly, to techniques for improving model performance by simultaneous incorporation of partially available privileged information and label uncertainty.

BACKGROUND

The background description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventor, to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.

The digitization of health data provides an important opportunity for new knowledge discovery and improved care delivery, through the development of electronic analytics and clinical decision support systems (CDSS) that leverage vast data to support increasing aspects of healthcare. By synthesizing digital information into actionable knowledge, these systems have the potential to provide important insights into the health status of patients, helping physicians make more timely and accurate medical diagnosis and evidence-based treatment decisions. Yet, in modern settings, providers are inundated by the vast amounts of clinical data being generated during routine patient care. CDSS that can integrate these disparate data sources (structured and unstructured) in real-time and effectively assist clinicians in medical decisions show promise for transforming how patient care is delivered. These systems could support precision medicine by improving rapid and accurate diagnosis and make just-in-time treatment recommendations that improve care quality and clinical outcomes.

An example medical condition that could benefit from a clinical application of CDSS is in acute respiratory distress syndrome (ARDS) care. ARDS is a fulminant inflammatory lung injury that is the downstream consequence of critical illnesses such as sepsis, pneumonia, or trauma, evolving over hours to days. When diagnosed early, multiple evidence-based management strategies can be provided to patients with ARDS that improve clinical outcomes. However, more than half of patients with ARDS are not immediately recognized by practicing clinicians and, as a consequence, fail to receive these life-saving treatments. Patients who develop ARDS first present with serious illnesses including sepsis, pneumonia or traumatic injury. As their illness evolves, massive streams of clinical data are collected from their bedside monitoring devices and recorded in electronic health records as part of routine care. These data are asynchronous, with varying temporal frequency (from millisecond to daily) and stored in varying formats. Within 24-72 hours after presentation, some patients progress and develop ARDS. However, some clinicians may fail to recognize ARDS if they are unable to synthesize the clinical data, recognize this evolution and obtain and correctly interpret chest imaging and make an ARDS diagnosis.

The inability of healthcare providers to process the massive data-streams generated during the care of critically ill patients has been specifically cited as a potential reason for ARDS misdiagnosis. Delay in recognition of this condition, followed by delay in the institution of appropriate treatment is associated with increased mortality. Yet, only after a healthcare provider correctly interprets a chest x-ray and synthesize this information with other clinical data will they correctly diagnosis ARDS. There is a need for a CDSS that models the diagnostic process, such that the accuracy and reliability of ARDS diagnosis can be greatly enhanced, thereby ensuring patients receive timely treatments that improve clinical outcomes.

The example of ARDS demonstrates many of the challenges facing designers of CDSS and other systems that are called upon to assess large amounts of data, that is, big data analytics systems for personalized medicine and other personalized applications.

Identifying relevant clinical information from massive data streams is one major challenge. In many healthcare settings, for example, there is an abundance of rich, continuously measured data collected thorough all stages of care. However, there is a lack of reliable systems to: 1) extract clinically relevant features from these data; and 2) integrate these features with clinical data to provide quantitative predictions/recommendations about patient's health status, thereby assisting clinicians to improve the quality of decision making. In case of ARDS, for example, patients at risk for ARDS are typically monitored with continuous electrocardiogram (ECG) and photoplethysmography (PPG) in the form of pulse oximetry, in addition to discrete laboratory and vital sign data. Potential patterns in these signals that could signify developing respiratory insufficiency and ARDS are not extracted or used in real-time decision making.

Another challenge is having a system exhibiting robustness to uncertainty, a particular issue for health data. For the diagnosis of ARDS, as well as many other clinical diseases and syndromes, medical experts can confidently label a subset of patients as “positive” and “negative.” However, there may also be many cases, particularly at the early stages of diagnosis, where even medical experts may have far less confidence in labeling patients. Computational models that can account for this uncertainty may be more robust to the use of noisy or incomplete health data for medical diagnosis.

Another challenge is the potential for using data that is unavailable in live clinical environments, but available in training phase. In a conventional treatment flow for a patient, a chest radiograph is the main confirmatory data necessary for a definitive diagnosis of ARDS. Patients known to have early evidence of lung injury on chest x-ray are at high risk for progression to ARDS. However, a chest radiograph may not always be available in real-time, and even the need to perform a chest radiograph may not be recognized at early stages. In addition, the frequency at which chest radiographs are obtained is variable and not amendable to high density collection, in contrast to physiologic monitoring data. Therefore, ideally a clinical decision support system would not rely on chest radiographic data to determine a patient's ARDS risk. However, chest radiographs still can provide critical information about a patient's current state of illness, and this information may be of great value during retrospective training of algorithms that can detect high risk patients.

Another challenge is the availability of reliable methods for interpreting medical data, such as chest x-ray interpretation. At the time that a chest x-ray is made available, radiology interpretation plays the major role in diagnosis of ARDS. However, due to the noisy nature of these images, the expertise of the interpreter, and other factors, the reliability of chest radiograph interpretation for ARDS is low among clinicians. There is a clear need for image processing algorithms to quantitatively express and extract the changes in the chest x-rays (compared to normal cases) and evaluate the presence and severity of ARDS.

Another challenge is how to incorporate cognitive/behavioral data. While the importance and significance of cognitive and behavioral data have been recognized by the healthcare community, its use has not been realized in CDSSs. There is a need for decision support system capable of resolving these short comings and, in particular, for clinical decision support systems capable of utilizing cognitive and behavioral data collected upon patient presentation and capable of integrating this data with all other clinical data to improve characterization of patient's health status.

SUMMARY OF THE INVENTION

In examples, the present techniques provide systems having advanced computational algorithms to integrate and analyze data to consider multiple interdependent, heterogeneous sources and forms of patient data. In some examples, these systems leverage longitudinal clinical data for early disease detection. For example, clinical diagnosis may be achieved using a classification model that determines a patient's status (e.g., ARDS or non-ARDS) based on a set of clinical features. Classifying patients at time points prior to their development of a condition, time points after development, and tracking illness recovery, are important for informing treatment decisions—tasks less easily addressed using time-to-event models.

Using a classification model, the present techniques introduce new learning paradigms, including privileged learning and learning with uncertain clinical data. A known issue with classification systems is that observations on individual patients over time can be correlated, breaking the assumption of independence necessary for standard statistical inference. Some have proposed model autocorrelation in longitudinal patient data explicitly. In examples herein, systems deploy a sampling strategy when building datasets for training machine-learning algorithms that limits the number of highly correlated observations within individual patients to limit overfitting.

In examples, the present techniques use a classification model with uncertainty in training labels. Datasets in many applications may have uncertainty in their training labels or the input data (i.e. features) might be corrupted. For instance, in medical applications, precisely determining the prognosis of patients might be difficult as patients might not describe what has happened to them or how they feel, results of the laboratories might be erroneous and so on. In examples, the present techniques provide a support vector machine (SVM) based approach with embedded uncertainty in training labels.

In examples, the present techniques use privileged information to improve and tune classification models during training. For example, training is implemented using a learning paradigm in which the learner is provided with not only a set of training examples, but also a set of additional “privileged information” that can help significantly improve the learning process. Learning using privileged information (LUPI) considers the fact that while privileged information may not be available at the testing stage, the abundance of such information in the training phase can help tune and improve classification models. Various sources of privileged information can be found in healthcare applications, for example, including in retrospective databases and not just real-time environments. Example privileged information include (i) physiological waveform signals (e.g., Electrocardiogram (ECG), Arterial Blood Pressure (ABP), pulse oximetry using photoplethysmography (PPG), and respiratory rate (RR)), (ii) clinical/demographic data (e.g., electronic medical records (EMR) data such as chief complaint, medications, comorbidities, age, gender, injury scores, respiratory support (i.e. ventilator support) and laboratory results), (iii) medical imaging data (e.g., chest image data for ARDS diagnosis applications), (iv) cognitive and behavioral data (e.g., Glasgow Coma Scale (GCS) data, level of orientation, motor strength, pronator drift, arousal, speech and behavior observations and pain rating, and composite scores including the confusion assessment method (CAM) and Richmond Agitation Sedation Scale (RASS)), and (v) expert adjudicated patient outcomes (e.g., healthcare professional reviews of hospitalizations of other patients with diagnosed conditions such as ARDS onset, healthcare professionals rating of their confidence in assessments to capture clinical uncertainty, healthcare professional annotations, etc.)

Learning using privileged information is highly useful for time-course predictive scenarios such as clinical decisions making, in which the retrospectively training data sets contain highly informative additional information that cannot be used by conventional machine learning. For example, in the case of the early detection and prediction of ARDS, while the conventional machine learning would only consider the features extracted from clinical/physiological data for early detection of ARDS, i.e., the present techniques can consider the available data, along with the chest x-rays that are not available at the time of the actual decision making. In other words, since the chest x-rays are available in the training datasets, they can serve as the privileged information when creating the classification models that map the clinical/physiological data to the presence or absence of ARDS, i.e., an {−1, 1} determination. Indeed, in some examples, the present techniques apply the formulation of LUPI for SVM, generating new, optimized SVM techniques that shows superior results to previous SVM techniques.

In examples, the present techniques integrate streaming clinical data to detect ARDS by tracking evolving cardiovascular and respiratory system physiology in real-time, before any chest x-ray is ordered. The present techniques can then recommend a chest radiograph when a patient's ARDS risk reaches a specific threshold for further evaluation of possible developing ARDS. The present techniques can then automatically analyze the radiograph for features consistent with developing ARDS and incorporate these additional features into the machine-learning algorithm to then make an updated clinical diagnosis of ARDS.

In an example, a computer-implemented method of developing a classifier model for identifying a medical condition of a subject, the method comprises: receiving training data for a plurality of subjects, where the training includes sensed physiological data, imaging data, demographic data, and/or other clinical data, where at least some of the training data is for subjects having the medical condition and at least some of the training data is for subjects not having the medical condition, wherein the training data includes privileged information where at least a portion of the privileged information is label uncertain privileged information; identifying the label uncertain privileged information; applying a penalty parameter to each label uncertain privileged information, wherein the penalty parameter establishes a soft-margin decisional boundary for the label uncertain privileged information; performing an optimization on (i) the label uncertain privileged information having the penalty parameters, (ii) the privileged information in the training data that has certain labels, and (iii) non-privileged information in training data, to develop a mapping to an outcome determination of the medical condition; and generating a classifier model based on the mapping to classify testing data as corresponding to the presence of the medical condition or to the absence of the medical condition.

In some examples, the computer-implemented method includes applying the training data to a support vector machine (SVM) framework configured to map training data to a vector space and generate a decision hyperplane classifier model for identifying the presence of the medical condition and for identifying the absence of the medical condition.

In some examples, the computer-implemented method includes configuring the support vector machine (SVM) framework to map privileged information in the training data and non-privileged information in the training data to the vector space for generating the decision hyperplane classifier model.

