In this paper we provide a categorisation and implementation of digital ink features for behaviour characterisation. Based on four feature sets taken from literature, we provide a categorisation in different classes of syntactic and semantic features. We implemented a publicly available framework to calculate these features and show its deployment in the use case of analysing cognitive assessments performed using a digital pen.
The Rey Osterrieth Complex Figure (ROCF) is a widely used neuropsychological test for visual perception and long term visual memory. Many scoring systems are used to quantify the accuracy of the drawings; these are currently implemented by hand in a subjective manner. The paper gives details of the current progress of a novel technique to locate the scoring sections of the most common of these systems (the Osterrieth Scoring System), with the ultimate goal of automating the scoring system. High levels of distortion are possible, making this an extremely difficult task; however location and perceptual grading of the basic geometric features (triangles, rectangles and diamonds) have been most successful. All but one section in the test data was located (99.3 success) and 78 of the perceptual grades calculated were within 5 of grades generated by independent raters. Unary spatial metrics have been implemented to reduce the possible section candidates by an average of 75 without the loss of a single section. The conventional Clock Drawing Test (cCDT) is a rapid and inexpensive screening tool for detection of moderate and severe dementia. However, its usage is limited due to poor diagnostic accuracy especially in patients with mild cognitive impairment (MCI). The diagnostic value of a newly developed digital Clock Drawing Test (dCDT) was evaluated and compared with the cCDT in 20 patients with early dementia due to AD (eDAT), 30 patients with amnestic MCI (aMCI), and 20 cognitively healthy controls (HC). Parameters assessed by dCDT were time while transitioning the stylus from one stroke to the next above the surface (i.e. time-in-air), time the stylus produced a visible stroke (i.e. time-on-surface), and total-time during clock drawing. Receiver-operating characteristic (ROC) curves were calculated and logistic regression analyzes have been conducted for statistical analysis. Using dCDT, time-in-air was significantly increased in eDAT (70965.8 ms) compared to aMCI (54073.7 ms; p = .027) and HC (32315.6 ms; p < .001). In addition, time-in-air was significantly longer in patients with aMCI compared to HC (p = .003), even in the aMCI group with normal cCDT score (54141.8 ms; p < .001). Time-in-air using dCDT allowed discrimination of patients with aMCI from HCs with a sensitivity of 81.3 and a specificity of 72.2 while cCDT scoring revealed a sensitivity of 62.5 and a specificity of 83.3 . Most interestingly, time-in-air allowed even discrimination of aMCI patients with normal cCDT scores (80 from all aMCI patients) from HCs with a clinically relevant sensitivity of 80.8 and a specificity of 77.8 . A combination of dCDT variables and cCDT scores did not improve the discrimination of patients with aMCI from healthy controls. In conclusion, assessment of time-in-air using dCDT yielded a higher diagnostic accuracy for discrimination of aMCI patients from HCs than the use of cCDT even in those aMCI patients with normal cCDT scores. Modern digitizing devices offer the opportunity to measure subtle changes of visuo-constructive demands and executive functions that may be used as a fast and easy to perform screening instrument for the early detection of cognitive impairment in primary care.  The Clock Drawing Test (CDT) is a rapid, inexpensive, and popular neuropsychological screening tool for cognitive conditions. The Digital Clock Drawing Test (dCDT) uses novel software to analyze data from a digitizing ballpoint pen that reports its position with considerable spatial and temporal precision, making possible the analysis of both the drawing process and final product. We developed methodology to analyze pen stroke data from these drawings, and computed a large collection of features which were then analyzed with a variety of machine learning techniques. The resulting scoring systems were designed to be more accurate than the systems currently used by clinicians, but just as interpretable and easy to use. The systems also allow us to quantify the tradeoff between accuracy and interpretability. We created automated versions of the CDT scoring systems currently used by clinicians, allowing us to benchmark our models, which indicated that our machine learning models substantially outperformed the existing scoring systems. Peripheral vision is strongly limited by crowding, the deleterious influence of neighboring stimuli on target perception. Many quantitative aspects of this phenomenon have been characterized, but the specific nature of the perceptual degradation remains elusive. We utilized a drawing technique to probe the phenomenology of peripheral vision, using the Rey–Osterrieth Complex Figure, a standard neuropsychological clinical instrument. The figure was presented at 12° or 6° in the right visual field, with eye tracking to ensure that the figure was only presented when observers maintained stable fixation. Participants were asked to draw the figure with free viewing, capturing its peripheral appearance. A foveal condition was used to measure copying performance in direct view. To assess the drawings, two raters used standard scoring systems that evaluated feature positions, spatial distortions, and omission errors. Feature scores tended to decrease with increasing eccentricity, both within and between conditions... The digital clock drawing test is a fielded application that provides a major advance over existing neuropsychological testing technology. It captures and analyzes high precision information about both outcome and process, opening up the possibility of detecting subtle cognitive impairment even when test results appear superficially normal. We describe the design and development of the test, document the role of AI in its capabilities, and report on its use over the past seven years. We outline its potential implications for earlier detection and treatment of neurological disorders. We set the work in the larger context of the THink project, which is exploring multiple approaches to determining cognitive status through the detection and analysis of subtle behaviors.