In this paper we provide a categorisation and implementation of digital ink features for behaviour characterisation. Based on four feature sets taken from literature, we provide a categorisation in different classes of syntactic and semantic features. We implemented a publicly available framework to calculate these features and show its deployment in the use case of analysing cognitive assessments performed using a digital pen.
Digitalising popular existing cognitive assessments, such as the Clock Drawing Test (CDT), has been topic of recent debate. There are clear benefits resulting from digitalisation, such as increased diagnostic accuracy @cite_33 . only recently presented their work on how to infer congitive status from subtle behaviours observed in digital ink @cite_16 . Based on such ink features machine learning models can be trained @cite_0 , which can also be explained by existing, validated scoring schemes @cite_40 . Examples of complex digitalised cognitive assessments include the Rey-Osterrieth Complex Figure test @cite_4 , which can be used for various purposes, such as diagnosing the periphery @cite_23 .
@cite_4 @cite_33 @cite_0 @cite_40 @cite_23 @cite_16
2463614937 2108192156 2606695625 2000292092
The Clock Drawing Test (CDT) is a rapid, inexpensive, and popular neuropsychological screening tool for cognitive conditions. The Digital Clock Drawing Test (dCDT) uses novel software to analyze data from a digitizing ballpoint pen that reports its position with considerable spatial and temporal precision, making possible the analysis of both the drawing process and final product. We developed methodology to analyze pen stroke data from these drawings, and computed a large collection of features which were then analyzed with a variety of machine learning techniques. The resulting scoring systems were designed to be more accurate than the systems currently used by clinicians, but just as interpretable and easy to use. The systems also allow us to quantify the tradeoff between accuracy and interpretability. We created automated versions of the CDT scoring systems currently used by clinicians, allowing us to benchmark our models, which indicated that our machine learning models substantially outperformed the existing scoring systems. The digital clock drawing test is a fielded application that provides a major advance over existing neuropsychological testing technology. It captures and analyzes high precision information about both outcome and process, opening up the possibility of detecting subtle cognitive impairment even when test results appear superficially normal. We describe the design and development of the test, document the role of AI in its capabilities, and report on its use over the past seven years. We outline its potential implications for earlier detection and treatment of neurological disorders. We set the work in the larger context of the THink project, which is exploring multiple approaches to determining cognitive status through the detection and analysis of subtle behaviors. The conventional Clock Drawing Test (cCDT) is a rapid and inexpensive screening tool for detection of moderate and severe dementia. However, its usage is limited due to poor diagnostic accuracy especially in patients with mild cognitive impairment (MCI). The diagnostic value of a newly developed digital Clock Drawing Test (dCDT) was evaluated and compared with the cCDT in 20 patients with early dementia due to AD (eDAT), 30 patients with amnestic MCI (aMCI), and 20 cognitively healthy controls (HC). Parameters assessed by dCDT were time while transitioning the stylus from one stroke to the next above the surface (i.e. time-in-air), time the stylus produced a visible stroke (i.e. time-on-surface), and total-time during clock drawing. Receiver-operating characteristic (ROC) curves were calculated and logistic regression analyzes have been conducted for statistical analysis. Using dCDT, time-in-air was significantly increased in eDAT (70965.8 ms) compared to aMCI (54073.7 ms; p = .027) and HC (32315.6 ms; p < .001). In addition, time-in-air was significantly longer in patients with aMCI compared to HC (p = .003), even in the aMCI group with normal cCDT score (54141.8 ms; p < .001). Time-in-air using dCDT allowed discrimination of patients with aMCI from HCs with a sensitivity of 81.3 and a specificity of 72.2 while cCDT scoring revealed a sensitivity of 62.5 and a specificity of 83.3 . Most interestingly, time-in-air allowed even discrimination of aMCI patients with normal cCDT scores (80 from all aMCI patients) from HCs with a clinically relevant sensitivity of 80.8 and a specificity of 77.8 . A combination of dCDT variables and cCDT scores did not improve the discrimination of patients with aMCI from healthy controls. In conclusion, assessment of time-in-air using dCDT yielded a higher diagnostic accuracy for discrimination of aMCI patients from HCs than the use of cCDT even in those aMCI patients with normal cCDT scores. Modern digitizing devices offer the opportunity to measure subtle changes of visuo-constructive demands and executive functions that may be used as a fast and easy to perform screening instrument for the early detection of cognitive impairment in primary care. Many machine learning and pattern classification methods have been applied to the diagnosis of Alzheimer’s disease (AD) and its prodromal stage, i.e., mild cognitive impairment (MCI). Recently, rather than predicting categorical variables as in classification, several pattern regression methods have also been used to estimate continuous clinical variables from brain images. However, most existing regression methods focus on estimating multiple clinical variables separately and thus cannot utilize the intrinsic useful correlation information among different clinical variables. On the other hand, in those regression methods, only a single modality of data (usually only the structural MRI) is often used, without considering the complementary information that can be provided by different modalities. In this paper, we propose a general methodology, namely Multi-Modal Multi-Task (M3T) learning, to jointly predict multiple variables from multi-modal data. Here, the variables include not only the clinical variables used for regression but also the categorical variable used for classification, with different tasks corresponding to prediction of different variables. Specifically, our method contains two key components, i.e., (1) a multi-task feature selection which selects the common subset of relevant features for multiple variables from each modality, and (2) a multi-modal support vector machine which fuses the above-selected features from all modalities to predict multiple (regression and classification) variables. To validate our method, we perform two sets of experiments on ADNI baseline MRI, FDG-PET, and cerebrospinal fluid (CSF) data from 45 AD patients, 91 MCI patients, and 50 healthy controls (HC). In the first set of experiments, we estimate two clinical variables such as Mini Mental State Examination (MMSE) and Alzheimer’s Disease Assessment Scale - Cognitive Subscale (ADAS-Cog), as well as one categorical variable (with value of ‘AD’, ‘MCI’ or ‘HC’), from the baseline MRI, FDG-PET, and CSF data. In the second set of experiments, we predict the 2-year changes of MMSE and ADAS-Cog scores and also the conversion of MCI to AD from the baseline MRI, FDG-PET, and CSF data. The results on both sets of experiments demonstrate that our proposed M3T learning scheme can achieve better performance on both regression and classification tasks than the conventional learning methods.