Prostate Cancer Detection using Deep Convolutional Neural Networks

Abstract: Prostate cancer is one of the most common forms of cancer and the third leading cause of cancer death in North America. As an integrated part of computer-aided detection (CAD) tools, diffusion-weighted magnetic resonance imaging (DWI) has been intensively studied for accurate detection of prostate cancer. With deep convolutional neural networks (CNNs) significant success in computer vision tasks such as object detection and segmentation, different CNNs architectures are increasingly investigated in medical imaging research community as promising solutions for designing more accurate CAD tools for cancer detection. In this work, we developed and implemented an automated CNNs-based pipeline for detection of clinically significant prostate cancer (PCa) for a given axial DWI image and for each patient. DWI images of 427 patients were used as the dataset, which contained 175 patients with PCa and 252 healthy patients. To measure the performance of the proposed pipeline, a test set of 108 (out of 427) patients were set aside and not used in the training phase. The proposed pipeline achieved area under the receiver operating characteristic curve (AUC) of 0.87 (95% Confidence Interval (CI): 0.84-0.90) and 0.84 (95% CI: 0.76-0.91) at slice level and patient level, respectively.

\section*{Introduction}

Prostate cancer is the most common form of cancer among males in the United States. In 2017, it was the third leading cause of death from cancer in men in the United States, with around 161,360 new cases which represented 19$\%$ of all new cancer cases and 26,730 deaths, which represented 8$\%$ of all cancer deaths~\cite{Prostate_Cancer}. Despite the fact that prostate cancer is the most common form of cancer, if detected in the early stages, the survival rates are high due to slow progression of the disease~\cite{Prostate_Cancer}. Therefore, effective monitoring and early detection are the key for improved patients’ survival. \par

Currently, accepted clinical methods to diagnose clinically significant prostate cancer (PCa) are a combination of the prostate-specific antigen (PSA) test, digital rectal exam (DRE), trans-rectal ultrasound (TRUS), and magnetic resonance imaging (MRI). However, PSA screening leads to over-diagnosis, which leads to unnecessary expensive and painful needle biopsies and potential over-treatment~\cite{overdig}. Multiparametric MRI which relies heavily on diffusion-weighted imaging (DWI) has been increasingly becoming the standard of care for prostate cancer diagnosis in radiology setting where the area under the receiver operating characteristic curve (ROC) varies from 0.69 to 0.81 for radiologists detecting PCa~\cite{sonn2017prostate}. A standardized approach to image interpretation called Pi-Rads v2~\cite{Hassanzadeh2017} has been developed for radiologists, however, there remain issues with inter-observer variability in the use of the Pi-Rads scheme~\cite{rosenkrantz2016interobserver}. \par

Machine learning (ML) is a branch of artificial intelligence (AI) that is based on the idea of the system learning a pattern from a large scale database by using probabilistic and statistical tools and making decisions or predictions on the new data~\cite{ML1,ML2,ML3}. In medical imaging field, computer-aided detection and diagnosis (CAD), which is a combination of imaging feature engineering and ML classification, has shown potential in assisting radiologists for accurate diagnosis, decreasing the diagnosis time and the cost of diagnosis. Traditional feature engineering methods are based on extracting quantitative imaging features such as texture, shape, volume, intensity, and various statistics features from imaging data followed by a ML classifier such as Support Vector Machines (SVM), Adaboost, and Decision Trees~\cite{feature1,feature2,feature3,feature4,feature5,feature6}.\par

Deep learning methods have shown promising results in a variety of computer vision tasks such as segmentation, classification, and object-detection~\cite{CNNs,segmentation2,resnet}. These methods consist of convolutional layers that are able to extract different features from low-level local features to high-level global features from input images. A fully connected layer at the end of the convolutional neural layers converts convoluted features into the probabilities of certain labels~\cite{CNNs}. Different types of layers, such as batch normalization layer~\cite{bn}, which normalizes the input of a layer with a zero mean and a unit variant, and dropout layer~\cite{deep}, which is one of regularization techniques that ignores randomly selected nodes, have been shown to improve the performance of deep learning-based methods. Nevertheless, to achieve convincing performance, an optimal combinations and structures of the layers as well as precise fine-tuning of hyper-parameters are required~\cite{CNNs,resnet,vgg}. This remains as one of the main challenges of deep learning-based methods when applied to different fields such as medical imaging. \par

With CNNs' promising results in computer vision field~\cite{cnn,CNNs}, the medical imaging research community has shifted their interest toward deep learning-based methods for designing CAD tools for cancer detection. As a widely used approach, most of proposed algorithms require user-drawn regions of interest (ROI) to classify these user-annotated ROIs to PCa lesions and non PCa lesions. Tsehay \emph{et al.}~\cite{cad4} conducted a 3 $\times$ 3 pixel level analysis by 5 convolution layers deep VGGNet~\cite{vgg} inspired CNNs with 196 patients. They fine-tuned their classifier by cross-validation method within the training set with 144 patients and achieved area under ROC curve (AUC) of 0.90 AUC on a separated test set of 52 patients. The result was based on 3 $\times$ 3 windows of pixels extracted from  MRI slices of DWI, T2-weighted images (T2w), and b-value images of 2000s $mm^{-2}$.

Le \emph{et al.}~\cite{cad6} conducted two dimensional (2D) ROI classification with combination of fused multimodal Residual Network (ResNet)~\cite{resnet} and the traditional handcrafted feature extraction method. They augmented the training dataset and used the test set for fine-tuning and evaluating their classifier. They achieved ROI-level (lesion-level) AUC of 0.91. Liu \emph{et al.}~\cite{cad2} used VGGNet inspired 2D CNNs classifier to classify each sample corresponding to a 32 $\times$ 32 ROI (lesion) centered around biopsy location using a dataset, which was part of ProstateX challenge competition (``SPIE-AAPM-NCI Prostate MR Classification Challenge”)~\cite{armato2017prostatex}. They separated the dataset, 341 patients, into 3 sets, the training set with 199 patients for training, validation set with 30 patients for fine-tuning, and test set with 107 patients for evaluation, and applied data augmentation to all 3 sets. They used 4 different types of inputs which were generated with different combinations of DWI, apparent diffusion coefficient map (ADC), $K_{trans}$, and T2w for their study. They achieved AUC of 0.84 with the augmented test test.

Mehrtash \emph{et al.}~\cite{cad5} also used VGGNet inspired 9 convolution layers deep three dimensional (3D) CNNs classifier to classify 3D PCa lesions vs. non PCa lesions with 32 $\times$ 32 $\times$ 12 ROI using ADC, high b-value images, and $K_{trans}$ from dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) of ProstateX challenge dataset~\cite{armato2017prostatex}. They separated the data set with 341 patients into training set with 201 patients and test set with 140 patients, and achieved lesion-level performance of 0.80 AUC on their test set. They applied cross-validation method within the augmented training set during training. As it will be discussed in Discussion section, the proposed method in this paper is superior compared to these ROI-based solutions in terms of robustness and applicability in clinical usage since it foregos the need for manually or automatically generating ROIs.

Slice-level detection algorithms classify each MRI slice into with or without PCa tumors. Ishioka \emph{et al.}~\cite{cad3} performed the slice-level analysis with 316 patients by U-Net~\cite{unet} combined with ResNet. They created non-augmented training, validation, and test sets and achieved AUC of 0.79 on the test set, which included only 17 individual slices. The proposed algorithm in this paper performs slice-level detection as well using a much larger sample size with superior performance compared to that proposed by Ishioka \emph{et al.}~\cite{cad3}.

Patient-level algorithms classify patients into with and without PCa. It is generally a challenging task to merge ROI-based or slice-level results into patient-level results~\cite{cad2,cad3,cad4,cad5,cad6}. Wang \emph{et al.}~\cite{cad1} compared the performance of deep learning-based methods to non-deep learning-based methods on the classification of PCa MRI slices vs non PCa MRI slices with 172 patients. They evaluated their VGGNet inspired 7 layers (5 convolution layers and 2 inner product layers) CNNs classifier's performance based on cross-validation. First, they classified each slice of a given patient and then converted the slice-level results into patient-level results by a simple voting strategy and achieved the patient-level AUC of 0.84, positive prediction value (PPV) of 79\%, and negative prediction value (NPV) of 77\%. In this work, we achieved similar results with independent test set and larger sample size.

In this paper, we propose an automated pipeline for two levels of PCa classification: slice-level and patient-level. Our pipeline contains five individually trained CNNs architecture that is inspired by ResNet, a decision tree-based feature extractor, and a Random Forest classifier~\cite{breiman2001random,nguyen2013random}. For the robustness of the performance, we divided the dataset into three separate sets, the training, validation, and test sets, and ensured that the test set was never seen by the classifier during training and fine-tuning~\cite{ML1}. Our classifier's performance on the independent test set was superior and more robust compared to similar studies that proposed CAD tools for PCa detection using deep CNNs.

\section*{Results}
The AUC and ROC curve~\cite{roc} were used to evaluate the performance of the proposed pipeline. A ROC curve is a commonly used method to visualize the performance of a binary classifier by plotting true positive rates and false positive rates with different thresholds, and an AUC summarizes its performance with a single number. The great advantage of AUC is its validity in an unbalanced dataset. Since only a small number of DWI slices have PCa tumor (e.g., average of 1 to 3 slices per patient where the total number of slices are an average of 14), AUC is the best way to evaluate the performance of the pipeline. In addition, ROC curve allows us to pick desired specificity and/or sensitivity of the classifier through the threshold. This evaluation method is applied to slice-level and patient-level classifications using the test set with 108 patients (1,486 slices).

\subsection*{Slice-Level Performance}
Since the pipeline contains 5 individually trained CNNs, there are five different test results at slice level. Table~\ref{tab-CNNs} shows individual performances on the test set for each CNNs. Our best CNNs (CNN1) achieved the DWI slice-level AUC of 0.87 (95$\%$ Confidence Interval (CI): 0.84$-$0.90). Figure~\ref{fig-slice} shows the ROC curve of CNN1 performance.

\begin{table}[h!]
\centering
\begin{tabular}{|l|l|l|}
\hline
\rowcolor{Gray}
\multicolumn{1}{|c|}{Architecture} & Test AUC (95 \% CI)      \\ \hline
\textbf{CNN1}                          & \textbf{0.87} (0.84 - 0.90)  \\ \hline
CNN2                                   & 0.87 (0.84 - 0.90)        \\ \hline
CNN3                                   & 0.86 (0.83 - 0.89)          \\ \hline
CNN4                                   & 0.85 (0.82 - 0.88)          \\ \hline
CNN5                                    & 0.85 (0.82 - 0.88)         \\ \hline
\end{tabular}
\caption{Slice-level performances of 5 individually trained deep CNNs.}
\label{tab-CNNs}
\end{table}

\begin{figure}[h]
\centering
\includegraphics[width=95mm]{slice_result}
\caption{Slice-level ROC curve of the proposed ResNet inspired deep learning architecture (AUC: 0.87, CI: 0.84$-$0.90)}
\label{fig-slice}
\end{figure}

\subsection*{Patient-Level Performance}
The patient-level AUC by our Random Forest classifier with the features extracted through CNNs was 0.84 (95$\%$ CI: 0.76$-$0.91) (Figure~\ref{fig-patient}).

\begin{figure}[ht]
\centering
\includegraphics[width=95mm]{patient_result}
\caption{Patient-level ROC curve of the proposed pipeline: Random Forest classifier trained on the features extracted by the CNNs (AUC: 0.84, CI: 0.76$-$0.91)}
\label{fig-patient}
\end{figure}

\section*{Discussion}

In the literature, several PCa classification methods for MRI images have been developed to address the inherent challenges of CAD tools for cancer detection, which can be categorized into two classes: radiomics-driven feature-based methods~\cite{feature1,feature2,feature3,feature4,khalvati2015,khalvati2018} and deep learning-based methods~\cite{cad1,cad2,cad3,cad4,cad5,cad6}.\par

Radiomics-driven feature-based methods consist of two stages: extraction of hand crafted features and classification based on these features. These methods require a comprehensive set of radiomic features, which include first- and second-order statistical features, high-level features such as morphological features~\cite{feature1}, and voxel-level feature~\cite{feature2}. For the classification using radiomic features, several approaches have been proposed. Different machine learning classifiers~\cite{ML1} such as naive Bayesian classifier~\cite{feature1}, support vector machines~\cite{khalvati2015,khalvati2018}, and Random Forest classifier~\cite{feature2} have been used. However, it has been shown that deep learning methods are superior to radiomics-driven feature-based methods in classification of PCa~\cite{cad1}.\par

ROI is one of commonly used data structures in medical image analysis. Usually delineated by the user, ROIs are samples within medical images identified for a particular purpose~\cite{roi}, which often contain cancer tumors. ROI-based methods directly compare and only classify regions or bounding boxes that contains tumors over healthy tissues. ROI-based methods have been used in both radiomics-driven and deep CNNs-based methods for PCa CAD tool design. In deep CNNs-based methods, Liu \emph{et al}~\cite{cad2}, Tsehay \emph{et al}~\cite{cad4}, and Le \emph{et. al.}~\cite{cad6} used 2D ROIs of cancer tumors and Mehrtash \emph{et al}~\cite{cad5} and used 3D ROIs of cancer tumors as their data structures (eg. $32\times 32\times 12$ ROI).

ROI-based CAD algorithms have several limitations. First, ROI-based algorithms require a time consuming manually generated (by expert reader) or automatically generated segmentation of ROI as a part of the pipeline to generate ROI-based dataset. If it is a manually generated segmentation, the application for clinical use is limited because it ultimately relies on the clinician's review and expertise and is not fully automated. If, on the other hand, it is an automatically generated segmentation, the result of classification depends on the performance of the segmentation algorithm, and inaccuracies from ROI segmentation algorithm can lead to poor PCa detection performance. Moreover, most of ROI-based methods use sliding windows of pixels as data structures to feed the CNNs, which makes it a challenging task to achieve an acceptable performance on classification of PCa at patient level due to the fact that each patient's MRI data constitutes several hundreds of thousands of windows of pixels. Therefore, ROI-based methods~\cite{cad2,cad4,cad5} struggle to merge individual ROI-based results into patient-level classification and they usually rely on basic merging methods such as simple voting~\cite{cad1}, which makes it a challenging task to achieve acceptable performance at patient level.\par

In this work, instead of feeding ROIs into CNNs, we used automatically center cropped DWI images with the only user intervention being to indicate the first and last slice that contained prostate gland. This is advantageous because it does not require generation of ROIs by either hand or segmentation algorithms. Thus, the proposed pipeline performance is independent of ROI generation method. There are other studies in the literature that proposed slice-based analysis~\cite{cad1,cad3}, but our slice-level performance (AUC: 0.87) and the sample size of the test set (108 patients or 1,486 slices), were significantly superior to their performance and sample size. For example, Ishioka \emph{et al.}~\cite{cad3} proposed a slice-level algorithm using 316 patient data for training and validation. The test set was only 17 slices withe AUC of 0.79. Furthermore, we used the results generated by CNNs as features for classifying PCa at patient level, which was not the case with these previous works on slice-level algorithms.\par

Completely isolating test data from training and validation is crucial to measure true performance of a (deep) machine learning-based classifier. Cross-validation is a well-known method to evaluate the performance of the classifier~\cite{cad1}. However, it is only relevant for optimizing or fine-tuning the model because there is a possibility that cross-validation leads to a model that over-fits. Fine-tuning the classifier based on the performance of the test set (e.g., adopted in~\cite{cad6}) makes the test set not independent from the trained and optimized classifier, and hence, the performance achieved is optimistic and not realistic. The fine-tuning and optimization of the model must be done through a validation set, which is separate than both training and test sets as adopted in our work in this paper and those of~\cite{cad3,cad4,cad5}. Moreover, the test set should not be augmented (e.g., adopted in~\cite{cad2}) to keep the robustness of the results. Due to test data cross-contamination with training or validation sets via cross-validation or data augmentation, the performance of the proposed models in the literature is rather optimistic(e.g.,~\cite{cad2,cad6}).\par

In this work, we divided the entire dataset into three different sets, training, validation, and test set. In the slice-level analysis, the training set was used to train the model, and the validation set was used to fine-tune and optimize our CNNs architecture, and the test set was used to evaluate the performance of the CNNs. In the patient-level analysis, cross validation was used within the validation set to fine tune and optimize our decision tree based feature selector and Random Forest classifier, and tested on the test set. As a result, our classifier's results were more robust compared to studies that used cross-validation as a measure of the performance~\cite{cad1} and training deep learning classifier without the validation set~\cite{cad6}. For the studies that used independent test set~\cite{cad2,cad3,cad5}, our results are superior. For example, Liu \emph{et al}~\cite{cad2} conducted 2D ROI slice level analysis and achieved 0.84 AUC for ROC-based (centered around biopsy location) classification only compared to 0.87 AUC for our proposed pipeline for slice-level classification.

Turning ROI-level results or the slice-level results of MRI data into patient-level result has been a major challenge in PCa classification via deep learning~\cite{cad2,cad3,cad4,cad5,cad6}. This is due to the fact that the 3D MRI volume of each patient may have hundreds or thousands of ROIs. Wang \emph{et. al.}~\cite{cad1} converted their slice-level result into patient-level by averaging all of the slice-level probabilities for patient and thresholding the average probabilities to classify PCa at patient level. Although this method achieved patient-level performance similar to our proposed pipeline's results (AUC: 0.84), it is based on cross validation, which makes it an optimistic result. In contrast, the results presented in this paper is based on a test set which is completely separate than the training and validation sets. Moreover, our test data contained 108 patients, which is significantly larger than the dataset with 17 patients for each fold~\cite{cad1} .

\section*{Methods}
\subsection*{Data}

This retrospective study was approved by the Research Ethics Board (REB) of Sunnybrook Health Sciences Centre and all methods were carried out in accordance with relevant guidelines and regulations. The need for written informed consent was waived by the institutional REB. A cohort of 427 consecutive patients with a Pi-Rads score of 3 or higher who underwent biopsy were included. Out of 427 patients, 175 patients had clinically significant prostate cancer and 252 patients did not. A total of 5,832 2D slices of each DWI sequence (e.g., b0) which contained prostate gland were used as our dataset. We set the patient with Gleason score higher than or equal to 7 (International Society of Uropatholgists grade group (GG>=2) as the patient with a clinically significant prostate cancer and patient with Gleason score lower than or equal to 6 (GG=1) or with no cancer (GG=0) as the patient without a clinically significant prostate cancer. \par

\subsection*{MRI Acquisition}
The DWI data was acquired between January 2014 to July 2017 using a  Philips Achieva 3T whole body unit MR imaging scanner. The transverse plane of DWI sequences was obtained using a single-slot spin-echo echo-planar imaging sequence with four b values (0, 100, 400, and 1000s mm$^{-2}$), repetition time (TR) 5000 $\sim$ 7000ms, echo time (TE) 61ms, slice thickness 3mm, field of view (FOV) 240mm $\times$ 240mm and matrix of 140 $\times$ 140.

DWI is an MRI sequence which measures the sensitivity of tissue to Brownian motion and it has been found to be a promising imaging technique for PCa detection~\cite{biomarker}. The DWI image is usually generated with different b values (0, 100, 400, and 1000s mm$^{-2}$) which generates various signal intensities representing the amount of water diffusion in the tissue and can be used to estimate ADC and compute high b-value images (b1600)~\cite{highb} .

In order to use DWI images as input to our deep learning network, we resized all of the DWI slices into 144 $\times$ 144 pixels, and center cropped them with 66 $\times$ 66 pixels such that the prostate was covered. The CNNs were modified to feed DWI data with 6 channels (ADC, b0, b100, b400, b1000, and b1600) instead of images with 3 channels (red, green and blue.)

\subsection*{Training, Validation, and Test sets}
We separated 427 patients DWI images into three different sets, the training set with 271 patients (3,692 slices), the validation set with 48 patients (654 slices), and the test set with 108 patients (1486 slices) where the training/validation/test ratio was 64\%, 11\%, 25\%. The separation procedure of the dataset was as follows. First, we separated the dataset into two sets, the training/validation set as 75\% and the test set as 25\% to maintain a reasonable sample size for the test set. Second, we separated  the training/validation set into two sets with training set as 85\% of training/validation set and the validation set as 15\% of training/validation set (Table~\ref{tab-data}). The ratios between the PCa patients and non PCa patients are kept roughly similar through out data sets. \par

\begin{table}[h!]
\centering
\label{my-label}
\begin{tabular}{|c|c|c|c|c|}
\hline
\rowcolor{Gray}
\multicolumn{1}{|c|}{Data Set} & Patients with PCa & Patients without PCa & Slices with PCa tumors & Slices without PCa tumors      \\ \hline

\textbf{Training Set}              & 105 & 166 & 439 & 3,253\\ \hline
\textbf{Validation Set}         & 18 & 30 & 66 & 588\\ \hline
\textbf{Test Set}               & 52& 56& 226 & 1,260     \\ \hline
\end{tabular}
\caption{Number of patients and slices with and without PCa for training, validation, and test sets.}
\label{tab-data}
\end{table}

\subsection*{Data Preprocessing}
All of DWI images in the dataset were normalized using the following function.

\begin{equation}
X_{i\_normalized} = \frac{X_{i} - \mu}{std}
\end{equation} where $X_{i}$ is the pixels in an individual MRI slice, $\mu$ is the mean of the dataset, std is the standard deviation of the dataset, and $X_{i\_normalized}$ is the normalized individual MRI slice. Normalization was performed across the entire dataset.

\subsection*{Pipeline}
The proposed pipeline consists of three stages. In the first stage, each DWI slice is classified using five individually trained CNNs models. In the second stage, first-order statistical features (e.g., mean, standard deviation, median, etc.) are extracted from the probability sets of CNNs outputs, and important features are selected through a decision tree-based feature selector. In the last stage, a Random Forest classifier is used to classify patients into groups with and without PCa using these first order statistical features. The Random Forest classifier was trained and fine-tuned by the features extracted from the validation set with 10 fold cross-validation method. Figure~\ref{fig-pipeline} shows the block diagram of the proposed pipeline.

\begin{figure}[ht]
\centering
\includegraphics[width=180mm]{Pipeline}
\caption{The proposed pipeline block diagram. The inputs to each CNNs are 66 $\times$ 66 $\times$ 6 (ADC, b0, b100, b400, b1000, b1600) MRI slices. The output is the slice level and patient level results.}
\label{fig-pipeline}
\end{figure}

\subsubsection*{ResNet}

Since ResNet architecture has shown promising performance in multiple computer vision tasks~\cite{resnet}, we chose it as our base architecture for this research. Each Residual Block consists of convolutional layers~\cite{cnn} and identity shortcut connection~\cite{resnet} that skips those layers, and their outcomes are added at the end, as shown in Figure~\ref{fig-preact}-Left. When input and output dimensions are the same, the identity shortcuts, denoted by x, can be directly applied. The following formula shows the identity mapping process.

\begin{equation}
y = F(x, \left \{ W_i \right \}) + x
\end{equation} where $F(x, {W_i})$ is the output from convolutional layers and x is the input. When the dimension of input is not the same as that of the output (e.g., at the end of the Res. block), the linear projection $W_{s}$ changes the dimension of the input to be same as that of the output which is defined as:

\begin{equation}
Y = F(x, \left \{ W_i \right \}) + W_{s}x.
\end{equation}\par
To improve the performance of the architecture, we implemented a fully pre-activated residual network~\cite{Preact}.
In the original ResNet, batch normalization and ReLU activation layers were followed after the convolution layer, but in pre-activation ResNet, batch normalization and ReLU activation layers comes before the convolution layers. The advantage of this structure is that the gradient of a layer does not vanish even when the weights are arbitrarily small~\cite{Preact}. Instead of 2-layer deep ResNet block, we implemented a 3-layer deep "bottleneck" building block since it significantly reduces training time without sacrificing the performance~\cite{resnet} (Figure~\ref{fig-preact}-Right).

\begin{figure}[h!]
\centering
\includegraphics[scale=0.43]{preactivation}
\caption{ The structural difference between original residual network and fully pre-activated residual network.}
\label{fig-preact}
\end{figure}

\subsubsection*{CNNs Architectures and Training}
A ResNet with 41 layers deep was created for the slice-level classification. The architecture is composed of 2D convolutional layers with a $7 \times 7$ filter followed by a $3 \times 3$ Max pooling layer and residual blocks (Res Block). The depth of 41 layers were found to be optimal through hyper-parameter fine-tuning procedure using the validation set. Since the input images were small (66 $\times$ 66 pixels ) and the tumorous regions were even smaller (e.g., 4 $\times$ 3 pixels), additional ResNet blocks or deeper networks were needed. The first ResNet Block (ResNet Block1 in Table~\ref{fig-resnet}) is 3-layer bottleneck blocks with 2D CNN layers with filter sizes 64, 64 and 256 which is stacked 4 times. The second ResNet Block (ResNet Block2 in Table~\ref{fig-resnet}) is 3-layer bottleneck blocks with 2D CNN layers with filter sizes 128, 128, and 512 which is stacked 9 times. $2 \times 2$ 2D Average Pooling, Dropout layer, and 2D Fully connected Layer with 1000 nodes for two probabilistic outputs are followed by the end of Res Blocks. Table~\ref{fig-resnet} shows the overview of the proposed CNNs architecture.

Stochastic Gradient Decent~\cite{SGD} was used as the optimizer with the initial learning rate of 0.001, and it was reduced by a factor of 10 when the model stopped improving after iterations. The model was trained with the batch size set to 8. Dropout rate was set to 0.90. We used a weight decay of 0.000001 and a momentum of 0.90. Since the dataset is extremely unbalanced, binary cross entropy~\cite{bce} was used as the loss function. \par

\begin{table}[h!]
\centering
\label{my-label}
\begin{tabular}{|c|c|}
\hline
\textbf{Layer Name} & \textbf{Details about the layer}              \\ \hline
Conv layer          & 2D Convolutional Layer ($7 \times 7$, 64, stride 2)                                                                                \\ \hline
Max Pool            & 3x3 max pool, stride 2                                                                                \\ \hline
ResNet Block 1 & $ \left [ \begin{array}{cc} 1 \times 1, 64  \\
                    3 \times 3, 64\\
                    1 \times 1, 256
                    \end{array}\right] \times 4

$\\ \hline

ResNet Block 2 & $ \left [ \begin{array}{cc} 1 \times 1, 128  \\
                    3 \times 3, 128\\
                    1 \times 1, 512
                    \end{array}\right] \times 9

$\\ \hline
Ave Pool            & 2D Average Pooling ($7 \times 7$)                                                                                \\ \hline
FC                  & Fully Connected Layer (2-d, softmax)                                                                                \\ \hline
\end{tabular}
\caption{The Architecutre of the proposed CNNs}
\label{fig-resnet}
\end{table}

\subsubsection*{Stacked Generalization}
Due to the randomness in training CNNs (for instance, at the beginning of training CNNs, weights are set to arbitrary random numbers), each CNNs may be different despite identical set of hyper-parameters and input datasets. This means each CNNs may capture different features for the patient-level classification. Stacked generalization~\cite{wolpert1992stacked} is an ensemble technique that trains multiple classifiers with the same dataset and makes a final prediction using a combination of individual classifiers' predictions. Stacked generalization typically yields better classification performance compared to a single classifier~\cite{wolpert1992stacked}. We implemented a simple stacked generalization method to improve the patient-level performance by feeding all the slice-level probabilities generated by the five CNNs into a first-order statistical feature extractor to generate one set of features for each patient. In the proposed pipeline, the patient-level performance significantly improved (2-tailed P = 0.048) using 5 CNNs compared to a single CNNs (AUC: 0.84, CI: 0.76$-$0.91, vs. AUC: 0.71, CI: 0.61$-$0.81).

\subsubsection*{First Order Statistical Feature Extraction}
Each CNNs produced two probabilities associated with each class (PCa and non PCa) for each DWI slice. For each patient using the probability output of each CNNs, top five probabilities for each class of PCa and non PCa that were above 0.74 were selected. This was done to ensure weak probabilities at slice-level were not used for patient-level classification. The cutoff of 0.74 was selected by grid-search using the validation set. Next, from the remaining probabilities for each patient (both PCa and non PCa classes), the first order statistical features were extracted, and important features were selected by a decision trees-based feature selector. We extracted nine first-order features which are the mean, standard deviation, variance, median, sum, minimum (only from non PCa class), maximum (only from PCa class), skewness~\cite{kim2013statistical}, kurtosis~\cite{kim2013statistical}, and range from the minimum to maximum from each probability set.

This produced 90 features for each patient (9 features for PCa and 9 features for non PCa class for each CNNs). We selected 26 best features using the decision tree-based feature selector~\cite{saeys2007review}. The decision tree based-feature selector was fine-tuned and trained with 10 fold cross-validation method using the validation set (Figure~\ref{fig-feature extractor}).

\begin{figure}[h!]
\centering
\includegraphics[width=180mm]{feature_extractor}
\caption{Block diagram of the proposed first-order statistical feature extractor. PCa Set: probabilistic output set from each CNNs which is associated with PCa class. Non PCa Set: probabilistic output set from each CNNs which is associated with non PCa class.}
\label{fig-feature extractor}
\end{figure}

Once first-order statistical features were extracted for each patient, a Random Forest classifier
~\cite{breiman2001random,nguyen2013random} was trained using the validation set and tested on the test set for patient-level classification.

\subsubsection*{Computational time}
The CNNs were trained using one Nvidia Titan X GPU, 8 cores Intel i7 CPU and 32 GB memory. It took 6 hours to train all five CNNs with up to 100 iterations, less than 10 seconds to train the Random Forest classifier, and less than 1 minute to test all 108 patients.

\section*{LIST OF ABBREVIATIONS}
CAD: computer aided detection \\
DWI: diffusion-weighted magnetic resonance imaging \\
CNNs: convolutional neural networks \\
PCa: clinically signification prostate cancer \\
ResNet: Residual Network \\
PSA: prostate-specific antigen \\
DRE: digital rectal exam \\
TRUS: trans-rectal ultrasound \\
MRI: magnetic resonance imaging \\
ROC: receiver operating characteristic curve\\
ML: machine learning \\
AI: artificial intelligent \\
SVM: support vector machine \\
AUC: area under the curve \\
PPV: positive prediction value \\
NPV: negative prediction value \\
2D: two dimensional \\
3D: three dimensional \\
CI: confidence interval \\
ROI: user drawn region of interest \\
TR: repetition time \\
TE: echo time \\
FOV: field of view

\section*{Ethics approval and consent to participate}

The Sunnybrook Health Sciences Centre Research Ethics Boards approved these retrospective single institution studies and waived the requirement for informed consent.

\section*{Author contributions}

SU, MAH, and FK contributed to the design and implementation of the concept. SU, IG, MAH, and FK contributed in collecting and reviewing the data. FK and MAH are co-senior authors for this manuscript. All authors contributed to the writing and reviewing of the paper. All authors read and approved the final manuscript.

\section*{Competing interests}
The authors declare no competing interests.

Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV); Quantitative Methods (q-bio.QM)


Definition and clinical validation of Pain Patient States from high-dimensional mobile data: application to a chronic pain cohort

Abstract: The technical capacity to monitor patients with a mobile device has drastically expanded, but data produced from this approach are often difficult to interpret. We present a solution to produce a meaningful representation of patient status from large, complex data streams, leveraging both a data-driven approach, and use clinical knowledge to validate results. Data were collected from a clinical trial enrolling chronic pain patients, and included questionnaires, voice recordings, actigraphy, and standard health assessments. The data were reduced using a clustering analysis. In an initial exploratory analysis with only questionnaire data, we found up to 3 stable cluster solutions that grouped symptoms on a positive to negative spectrum. Objective features (actigraphy, speech) expanded the cluster solution granularity. Using a 5 state solution with questionnaire and actigraphy data, we found significant correlations between cluster properties and assessments of disability and quality-of-life. The correlation coefficient values showed an ordinal distinction, confirming the cluster ranking on a negative to positive spectrum. This suggests we captured novel, distinct Pain Patient States with this approach, even when multiple clusters were equated on pain magnitude. Relative to using complex time courses of many variables, Pain Patient States holds promise as an interpretable, useful, and actionable metric for a clinician or caregiver to simplify and provide timely delivery of care.

\section{Introduction}

Recent advances in digital medicine have provided the opportunity to collect large sets of clinical data to evaluate and predict critical medical outcomes. For instance, mobile-based applications, accelerometers, and biosensors are now ubiquitous in phones and watches, enabling one to longitudinally track variables like mobility and speech, and facilitate patient symptom self-report. Importantly, these features may associate with clinical meaning. Large-scale studies have shown that data from mobile applications tracking daily activity may predict outcomes relevant to health and illness, such as in geriatric care and diabetes \cite{Ekelund2009}, \cite{Smirnova2020}. Further, language can assess affective, psycholinguistic, physiological, and cognitive features can predict physiological and pharmacological \cite{Agurto2020}, psychiatric \cite{Corcoran2018a}, and cognitive disease states \cite{Eyigoz2020}. These types of findings have demonstrated the promise of digital health profiles in understanding patient experience and predicting important clinical outcomes.

Despite these advances, the size and complexity of the clinical data generated by mobile applications is nontrivial to interpret and apply for several reasons. First, digital healthcare data can exist in multiple formats, creating the need to fuse vast amount of diverse information \cite{Yu2019}. Second, there is a need for methods that can obtain clear data representations. These methods should provide interpretation that are manageable in size, yet can maintain the characteristics of the raw information, allowing for patients and healthcare professionals to interpret and use the output \cite{Ambigavathi2018}. Attempts to reduce and understand such data in a biological context have commonly used data-driven methods, especially those using machine learning algorithms. This approach offers the advantage of being able to handle large, multidimensional data sets through the ability to recognize patterns or joint representations that are otherwise difficult to identify using standard statistical approaches, providing knowledge discovery about a particular topic that can span time, location, and scales \cite{Dinov2016}. In particular, clustering analysis offers the ability to collapse across otherwise incomprehensible multidimensional data and observe how features co-occur. In the case of spectrum illnesses that incorporate a range and variety of symptoms, decomposition can be helpful, with some outputs having an advantage in outcomes prediction \cite{Reinen2018}. But not all results allow for interpretation, and it is particularly susceptible to problems in small, unvalidated datasets which may result in overfitting and thus results that are not replicable or generalizable. Further, while results from unsupervised approaches may reveal meaningful clinical patterns, few methods exist to formally assign labels, rank, or identify qualitative aspects from the results of data-driven approaches through independent validation.

A prime illustration of this problem is in chronic pain, a disease affecting a substantial percentage of the population \cite{Zelaya2019} that significantly impacts general function including employment, mental health, and social interaction. This heterogeneous condition interacts with well-characterized facets of health, including mood, sleep, psychosocial function, medication use, and mobility. However, the current practice for most pain studies is to evaluate outcomes based on pain magnitude alone, which does not consider all of the variance shown to predict treatment success, quality-of-life (QoL), or other measurements of physical, psychological, and social well-being \cite{Gatchel2007}. But, using all of these features as outcome variables is nontrivial to compute, conceptualize, and interpret. Few standard approaches have been developed that incorporate both the computational methodology required to complete such a task, and the ability to provide a clinically interpretable summary of the output. To date, machine learning has been used to predict pain outcomes, identify clinical subgroups \cite{Mullin2021}, extract knowledge, and detect structure in biological and clinical features \cite{Lotsch2018}. In chronic pain, while artificial intelligence (AI) has been applied to improve diagnoses, fewer studies apply it to the treatment and management of pain patients \cite{Jenssen2021}, and analyses that use longitudinal data or clinical validation are extremely limited.

Given the quickly expanding capacity of digital health and learning algorithms to inform treatment outcomes in complex illnesses, there is a benefit to developing an approach to validate health states from multidimensional data. While it is known that various chronic pain symptoms can co-occur, it remains currently unknown whether symptom profiles may be successfully organized into distinct health states. Here, we propose a method by which we aim to identify clusters from high-dimensional, longitudinal data in chronic pain patients, and label them as Pain Patient States that may be operationalized for clinical application and decision making \cite{Reinen2020}\cite{Anitescu2022}. To this end, we examined data from chronic pain patients in three subsets of data: 1) with questionnaires only; 2) with questionnaires plus voice data; and 3) with questionnaires plus actigraphy data. The dimensionality of each dataset was reduced into stable clusters using standard unsupervised clustering algorithms. Next, we quantitatively evaluated the clusters based on relationships to established health metrics, using standard assessments as clinical benchmarks in chronic pain to compare the data-driven results. A clear ordinal rank of states emerged, allowing us to assign unique qualitative labels even in clusters that were nearly identical in pain magnitude, so that they may be used as clinically-informed states. This system serves as an example of organizing diverse types of large datasets and anchoring them to known metrics as to evaluate treatment or assess function. Here, these formerly convoluted data patterns may now act to contextualize signal, rank results, track longitudinal health changes, and monitor meaningful medical outcomes.

\section{Methods}

\subsection{Participants and Data Collection}

Participants were recruited from pain clinics in on-going, longitudinal, multi-center, clinical studies (Clinicaltrials.gov ID: NCT01719055) aimed to understand chronic lower back and leg pain patients who are candidates for spinal cord stimulator (SCS) treatment (Boston Scientific, Valencia, CA). Participants were recruited and enrolled in the NAVITAS and/or ENVISION studies at multiple United States clinical sites if they intended to receive or had already received an SCS trial or implant, were at least 18 years old, and had been diagnosed with intractable chronic neuropathic pain. Additionally, subjects may have been previously enrolled in the RELIEF study (Clinicaltrials.gov ID: NCT01719055). Data were included in this analysis from each study a subject was enrolled in. Health-related questionnaires were administered via an at-home, custom-designed clinical study version of a digital health ecosystem (Boston Scientific, Valencia, CA) for up to 36 months. The questions chosen included pain-related subjective ratings, symptoms hypothesized to contribute to variability in pain ratings, as well as symptoms hypothesized to be impacted by pain, specifically pain magnitude, mood, sleep, alertness, medication use, and activity. Following enrollment, data were collected in separate in-clinic and at-home data streams. Mobile data analyzed here included voice recordings, as well as daily, self-reported symptom monitoring, with the option to respond more frequently if participants wished. In addition, subjects were asked to wear a smartwatch to assess mobility using accelerometer data (Galaxy Watch S2, Samsung USA, Menlo Park, CA with custom watch application, Boston Scientific, Valencia, CA). In-clinic assessments were collected at the baseline (enrollment) visit, and at 1-month, 3-month, 12-month, and optionally 24-month and 36-month visits following enrollment. In the present analysis, we used in-clinic assessments to evaluate QoL \cite{Group1990} and disability measured by the Oswestry Disability Index, or ODI \cite{Fairbank2000} questionnaires.

\subsection{Voice data processing}

Voice recordings were collected from weekly recordings based on prompts aimed to understand the participant's experience with pain. Speech features for psycholinguistic, sentiment \cite{Pennebaker2015}, and acoustic characteristics \cite{Eyben2010}, \cite{DeJong2009} were extracted from the audio files using in-house and standard code. Age and sex were regressed from all features. Next, to reduce dimensionality of these features, a principal components analysis (PCA) was used ($var \geq 2\%$) to identify the decomposed components. These components were later included in a clustering analysis alongside the 6 features derived from the questionnaires.

\subsection{Actigraphy data processing}

Effective mobility was derived from the watch-based actigraphy data. It is a novel metric of physical function and activity meant to reflect the duration and type of activity a person experiences beyond steps or activities of daily life. Rates of activity were calculated into categories for each participant throughout the day. These categories ranged from Zone 0 (e.g., resting, using a mobile device while seated) to Zone 4 (e.g., intense or repetitive motion or vigorous exercise) and were used along with the questionnaire data in the clustering analysis.

\subsection{Data and Clustering Analysis}

For each participant, all available data was downloaded and selected based on days for which all subjective features from questionnaires (e.g., overall/leg/back pain, mood, sleep hours, sleep quality alertness, medication use for opioid/over-the- counter/non-opioid pain medication, activity interference due to pain, and activities of daily life), as well as actigraphy and voice data (where applicable) were present. Patients were included in the analysis regardless of time point in the study (e.g., baseline/enrollment, SCS trial period, follow-up), in the interest of observing a spectrum of pain-related variability and experience. However, the criteria for removing samples from the analysis consisted of: 1) any day missing a single data point, 2) any individual having fewer than 10 total complete data points, and when applicable 3) individuals who wore the smartwatch for less than 10 days. All question value responses were normalized prior to cluster analysis to equate the different subjective feature values across the individual question, and data distributions were inspected for abnormalities. Next, each question categorized to assess pain, sleep, and medication use were averaged to produce single composite scores for each modality; for activity, a difference score was taken between the two questions, in which we include a penalty that account with pain interferes with any overall activity. If any participant had answered more than one question on a certain day, the average of those responses was used to represent the daily value for that category. Participants were assessed for their average responses over time in order to determine the extent to which some participants responded more frequently than others, and the analysis was rerun without outliers to further ensure cluster stability.

Cluster definitions were calculated using a k-means clustering algorithm with Euclidean distance exploring up to cluster solutions for k = 10. Optimal k was determined using multiple methods including sum of squares distances and silhouette values, agglomerative analysis, and consensus clustering. To ensure clusters were similar across subsamples of participants exhibiting variability in number of responses included in the analysis, we repeated the analyses in varying samples of participants in which highly contributing participants (those with higher daily average responses) were excluded. Next, we employed an analogous approach to examine cluster solutions over the course of time. Generally, we expected the clusters to remain similar over time with some slight changes (e.g., higher pain prior to therapy) that would be evident in the cluster. With this in mind, cluster solution results were then visually inspected in order to ensure similarities in qualitative characteristics and are discussed in the results section.

\begin{figure}[htbp]
\centerline{\includegraphics{IEEE_DH_Figures_Final.001_resized.eps}}
\caption{Conceptual data and methods overview. (A) Data were collected from a multi-center clinical trial recruiting participant with chronic low back and leg pain seeking spinal cord stimulator (SCS) treatment. Both in-clinic and at-home data collection were used to record 1) questionnaire-based daily reports of pain, mood, activity, medication, alertness, sleep; 2) standard assessments of QoL (EQ5D) and disability (ODI); 3) voice responses to open-ended questions about their pain; and 4) actigraphy from a smartwatch. (B) Data from questionnaires, voice, and actigraphy were subjected to a k-means clustering analysis and the (C) resulting cluster representation was examined across features. To validate these clusters, (D) centroid distance to each cluster was compared to the clinical scores for disability and QoL allowing for (E) an interpretation and label to be assigned to each cluster. }
\label{fig}
\end{figure}

\section{Results}

\subsection{Sample demographics and data chronology}

In the primary analysis including questionnaires only, 121 individuals with 11,763 samples of data were used (40.5\% male, mean age 59.4 years old, 17.6 years since pain onset). In the analysis examining the addition of actigraphy data to the questionnaire data, 116 individuals with 11,286 samples of data were used (39.7\% male, mean age 59.3 years old, 17.8 years since pain onset). For the analysis including voice, 2,080 samples were included.

\subsection{Clustering results and characteristics for questionnaire data only}

Cluster definition was examined for the questionnaire-only data for k = 2 to 10. Sum of squares distances and silhouette analyses indicated that a cluster solution of k = 2 or 3 was stable. Agglomerative hierarchical clustering was repeated to validate k with cross-methodological clustering, which also converged on a solution of k = 2 or 3. Given the relative stability of smaller cluster solutions, we first examined a simple and stable solution of k = 2. Feature characteristics of the cluster solution for k = 2 were examined by inspecting mean values for each feature in each cluster (Figure 2A). Results indicated a clear negative-to-positive grouping of health features, such that the questionnaire responses of one cluster appeared to represent a superior health state represented by better mood, sleep, alertness and activity, and lower ratings of pain and medication use. The other cluster appeared to represent an inferior health state, characterized by higher pain and medication use, with lower ratings for alertness, mood, sleep, and activity. This analysis was repeated to exclude the  high-responder group in order to ensure that the clusters were not being driven by the high-responders. Results indicated that the clusters were very similar both in all participants, and without the high-responders. Finally, the cluster solutions were re-examined over the course of time, such that the analysis was repeated in the baseline period prior to SCS activation, during the first 6 months of treatment, and the subsequent 6 months of treatment. Results indicated that the cluster solution was very similar over time, with some indication of higher pain prior to treatment. An examination of a 3-cluster solution revealed a third, intermediate cluster that represented a health state similar to or in between the two states represented in the two-cluster solution (Figure 2B). This cluster showed relatively high ratings of alertness, mood, and sleep, but with intermediate values for pain, activity and medication use. A repeated analysis excluding high-responders also showed an intermediate cluster, with values for each feature with a magnitude between the previous two clusters.

\begin{figure}[htbp]
\centerline{\includegraphics{IEEE_DH_Figures_Final.002_resized.eps}}
\caption{Cluster analysis for questionnaire data reveals negative and positive symptom groups. (A) A two cluster (k = 2) solution resulting from the k-means analysis of the questionnaire data revealed two clusters of symptoms that stratified on a negative-to-positive spectrum of pain-related health, in which one cluster revealed a better health state of better mood, sleep, more reported activity and alertness, less medication usage, and lower pain. Conversely, the other cluster depicted a worse health based on the feature means. (B) A three cluster (k=3) solution also revealed a spectrum of positive to negative symptom groupings, including superior and inferior states similar to k=2, with an additional intermediate state showing moderate pain and medication use but with high mood, sleep, activity, and alertness scores.}
\label{fig}
\end{figure}

\subsection{Decomposing and clustering questionnaire and voice data}

Prior to clustering, the results of the principal components analysis (PCA) of the voice features were inspected. The results showed that 7 components were present, and characterized features such as voiced and unvoiced energy in a speech signal, negative sentiment, emotional content, and acoustic voice properties (Table 1). We repeated the clustering analysis with each of these 7 components included along with the 6 questionnaire components. With the addition of the 7 components, solutions for k of 2 or 5 were possible. For the k = 2 solution, results showed that in particular, component 4, which was characterized by high loadings of negative sentiment and acoustic features associated with emotion, tracked well with the inferior health cluster (Figure 3A). Further, while not all components showed the same discrimination between states as did component 4, there was evidence that the addition of the voice data expanded the granularity of the state solutions. This was illustrated by the comparison of a cluster solution with only questionnaires in which pain was stratified across 3 levels in all states (Figure 3B). When the cluster solution included both voice and questionnaires, pain across states expanded to 5 levels (Figure 3C).

\begin{table}[htbp]
\caption{Decomposition of voice features into 7 components}
\centerline{\includegraphics{IEEE_DH_Figures_Final.005_resized.eps}}
\label{fig}
\end{table}

\begin{figure}[htbp]
\centerline{\includegraphics{original_voice_cluster_fig_resized.eps}}
\caption{Adding voice features to cluster analysis improves pain granularity in state solutions. (A) Clustering analysis was run including 6 questionnaire components and 7 voice components for a 2-state solution, indicating that voice features denoting negative sentiment were associated with the poorer health cluster. (B) A 5-state cluster solution without voice features reveals three levels of pain magnitude across clusters, while the (C) addition of voice to a 5-state cluster solution adds further granularity to pain magnitude across clusters.}
\label{fig}
\end{figure}

\subsection{Clustering results and characteristics for questionnaire and actigraphy data}
Actigraphy data downloaded from the watch were parsed into mobility Zones 0 - 4 of effective mobility. Inspection of results indicated that these zones indeed provided granularity that added description beyond number of steps or self-reported ADLs (Figure 4). The clustering analysis included the 6 categories derived from the questionnaires along with the effective mobility. An analysis for optimal k showed that state solutions of up to 5 clusters was possible. These clusters appeared to range from a "best" state that included low pain and medication use, and high reports of mood, sleep, alertness, and effective mobility, to an inferior state that is associated with high levels of pain and medication use, and low reports of activity, mood, sleep, alertness, and effective mobility (Figure 5).

\begin{figure}[htbp]
\centerline{\includegraphics{IEEE_DH_Figures_effmob_resized.eps}}
\caption{Description of effective mobility zones. Mobility data was parsed into zones of “effective mobility” based on rates of activity calculated at regular time window intervals throughout the day. When compared to step counts and self-reported activities of daily life (ADLs), effective mobility showed additional computational granularity of participant mobility.}
\label{fig}
\end{figure}

\begin{figure}[htbp]
\centerline{\includegraphics{IEEE_DH_Figures_Final_effmob_results.001.eps}}
\caption{Adding mobility features contributes to cluster dimensionality. (A) A cluster solution including effective mobility identified 5 stable clusters for which the addition of effective mobility may contribute to additional clusters relative to the questionnaire-only solutions, still ranging from a negative-to-positive spectrum and including a best and a worst state. (B) States from the 5-cluster solution show further granularity as it pertains to patient experience beyond the 2- and 3-state model.}
\label{fig}
\end{figure}

\subsection{Cluster validation and state classification}
For the validation analysis, we obtained pairs of metrics comprised of 1) distances from the cluster centroids on a given day; and 2) responses to standard assessments (disability, or ODI, and QoL, or EQ5D measurements focusing on Pain, Activities, and VAS Health). These two metrics were collected within one week of each other; any pairs with collection dates outside of the week window were dropped from analysis. We first calculated correlations between centroid distances of each cluster in the two-state solution, and found that the  correlations were statistically significant and consistent in terms of direction and magnitude for the two states, indicating a clear best and inferior state (in cluster 1 values were: disability/ODI, r = 0.42, EQ5D Pain, r = 0.47, EQ5D Activities r = 0.37, EQ5D VAS Health r = $-0.32$; all p-values $<$0.001, for cluster 2 values were: disability/ODI, r = $-0.41$, EQ5D Pain, r = $-0.43$, EQ5D Activities r = $-0.38$, EQ5D VAS Health r = 0.28; all p-values $<$ 0.001). This indicated that larger centroid distances were associated with higher values for the outcomes. Critically, while most of the validation metric outcomes represented negative health values with increasing severity including disability, EQ5D-Pain, EQ5D-Activities, etc., the EQ5D measure of VAS Health represents health on a positive scale, and as expected showed an inverse relationship to the findings above. Given that each cluster was associated with consistent directionality across all of the standard assessments, we were able to infer that each of the clusters represented distinct health states, aligned with what we would have expected to find in patients across time.

\subsection{Cluster validation with voice data}

A similar analysis was repeated using the k = 2 cluster solution that included voice data. Results indicated that generally the directionality of the correlations was consistent relative to prior analyses. However, for several validation metrics, the magnitude of the r values increased with the addition of voice features (for disability/ODI, r = 0.47, EQ5D VAS Health r = $-0.49$). In particular, assessments that may take into account negative affect showed an increase in the correlation across these metrics. Notably, because voice data is collected less frequently, there was a decrease in sample size relative to the prior analysis. That said, permutation tests were used to compare across the two approaches and to ensure that there were no meaningful differences due to sample size. In all instances, permutation tests confirmed the significance of prior findings at p $<$ 0.05.

\subsection{Cluster validation with actigraphy data}
Next, we aimed to determine whether correlations between centroids from a more highly dimensional state solution compared to the standard assessments could provide further ordinal information about the states. To do this, we ran a similar analysis using the 5-state solution that was obtained with the cluster solution including effective mobility. Here, we found that the correlations across the 5 states also provided evidence for a consistent ranking of those states from best to worst (Table 2).

\begin{table}[htbp]
\caption{Cluster characteristics including effective mobility}
\centerline{\includegraphics{IEEE_DH_Figures_Final.004_resized.eps}}
\label{fig}
\end{table}

\subsection{Comparison of state timecourse to health events}
In an exploratory analysis, we examined the relationship between state expression change over time relative to known health events. Here (see Figure 6), we first show that states represent a more interpretable visualization of health changes across time relative to examining the timecourse of all variables at once. Second, several exemplar patients show expected changes in states before and after implantation of the SCS device, a procedure that involves surgery and probable eventual pain relief.

\begin{figure}[htbp]
\centerline{\includegraphics{IEEE_DH_Figures_Final_patientexamples.eps}}
\caption{Examples of patient experiences show that states track with meaningful clinical events. Top time course for each patient denotes state assignment, whereas lower time course shows changes in multiple variables. Bar graphs show the dwell time change before and after a notable event, which here involves the implantation of a SCS device hypothesized to bring about eventual pain relief and improvement in QoL. (Here, data in the time courses included overall, leg, and back pain, sleep hours and quality, number of activities, pain interference, medication usage for opioid, over-the-counter, and non-opioid pain medications, alertness, mood, and effective mobility. States are ranked as $A>B>C>D>E$, as shown in Table 2.)}
\label{fig}
\end{figure}

\section{Discussion}

\subsection{High-dimensional health data can be decomposed meaningfully}

Using a unique set of longitudinal questionnaire, mobility, and speech data, we have developed a novel method to decompose, group, and validate large amounts of chronic pain digital health data. This study marks one of the only approaches to create clinically usable pain-related categories from complex questionnaire, mobility, and speech data across time. This approach demonstrates that high dimensional, longitudinal health data from chronic pain patients may be decomposed into clusters and used to classify patients according to a holistic status named Pain Patient States. These states have an ordinal ranking based on clinically-validated standard health assessments. Specifically, we demonstrated that in chronic pain, we can take multiple streams of information including sleep hours and quality, mood, pain magnitude at multiple sites, alertness, multiple types of medication use, ADLs, actigraphy, and speech in order to represent 3-5 Pain Patient States over the course of time. The stable solutions that emerged from this method suggest the discovery of distinct clinical states with non-obvious properties that may serve as new knowledge that informs biological mechanisms and clinical care. In addition to the identification of these Patient Pain States, this improves upon prior assessments and clinical trials that only use pain magnitude as an outcome evaluation by considering a much more comprehensive picture of patient experience in a way that is clinically interpretable. This approach leverages both data- and clinically-driven analyses by first using powerful learning algorithms, and then comparing the output to standard clinical metrics. Consequently, we are able to transform what was previously multiple, complex time courses for hundreds of patients into 3-5 states that are clinically contextualized, straightforward, and meaningful.

\subsection{The decomposition can be externally validated and ranked}
We found that the resulting clusters from our analysis stratified on a negative-to-positive spectrum of health in chronic pain, and that these clusters were reliable across subsets of individuals and over time. Importantly, these states provide valuable, novel information per se, representing new findings that may define patient experience. Nevertheless, because they were derived from a purely data-driven analysis, we chose to compare cluster characteristics to independent standard assessments of disability and QoL. We found not only that good and bad clusters associate with better and worse disability and QoL, but that more granular state solutions had a clear ordinal rank which contextualized the data-driven output (Table 2). Further, in a 5-state solution (see figure 5A), only 2 levels of discriminable pain were present for 4 states. This adds clear dimensionality beyond what pain alone may indicate about a patient's well-being. Thus, we were able to assess 5 ordinal steps of health based on multidimensional aspects, providing evidence that we can offer a more full picture of patient experience yet preserve interpretability, making these states meaningful and actionable clinical information. This can improve precision in outcomes assessments, especially as it pertains to pain research and clinical trials.

\subsection{Objective data adds granularity to state solutions}
In particular, raw objective metrics such as actigraphy and speech features are too complex to use without some dimension reduction. However, actigraphy and speech offer insight into patient experience both because they reflect a novel behavioral measure and because they involve limited self-assessment, which is known to be susceptible to psychological biases. Here we showed that we were able to quantify and select features from these objective measures in a preprocessing step, and then incorporate them into a clustering analysis. We found that one benefit of this approach is that these types of features indeed add dimensionality to a state solution, and the preprocessing in this case allowed for the derived features to add some biological interpretation. Additionally, we identified speech features that capture negative sentiment, possibly augmenting the ability for the states to detect disability versus wellness as indicated by higher correlation values between those states and the independent assessments.

\subsection{Conclusions}
Ultimately, this analysis combined AI and clinical knowledge to successfully reduce complex mobile data into useful health states that reflect important clinical time points and changes in patient experience (Figure 6). While all approaches should be tested and verified broadly across additional populations and data sets, this approach lays a solid foundation by which complex datastreams may be reduced into and authenticated as useful wellness information. We were able to show that we could successfully use this method in patients undergoing treatment for chronic pain, with results yielding new, distinct representations of patient experience. These findings imply it is possible to expand this approach to other illnesses associated with heterogeneous sets of symptoms. Finally, while we were able to compare our findings to known metrics, the health states provide deep insights in and of themselves that could aid a clinician in medical decision making and patient care. Given the growing use of digital health solutions, this approach to define Pain Patient States holds great promise in harnessing AI-driven solutions to aid in the care of large groups of chronic pain patients.

\section*{Acknowledgment}
The NAVITAS and ENVISION Studies Physician Author Group includes  Richard Rauck (The Center for Clinical Research), Eric Loudermilk (PCPMG Clinical Research Unit), Julio Paez (South Lake Pain Institute), Louis Bojrab (Forest Health Medical Center), John Noles (River Cities Interventional Pain), Todd Turley (Hope Research Institute), Mohab Ibrahim (Banner University Medical Center), Amol Patwardhan (Banner University Medical Center), James Scowcroft (KC Pain Centers), Rene Przkora (University of Florida), Nathan Miller (Coastal Pain and Spinal Diagnostics), and Gassan Chaiban (Ochsner Clinic Foundation).

The Boston Scientific Research Scientists Consortium includes Dat Huynh (Boston Scientific, Data Research and Engineering), Kristen Lechleiter (Clinical Research, Boston Scientific), Brad Hershey (Data Research and Engineering, Boston Scientific), Rex Woon (Data Research and Engineering, Boston Scientific), and Matt McDonald (Boston Scientific, Data Research and Engineering).

We wish to acknowledge work by Erhan Bilal (IBM, Digital Health) for his work on consensus clustering.

\bibliographystyle{IEEEtran}

Applications (stat.AP); Quantitative Methods (q-bio.QM)


Decoding Complex Imagery Hand Gestures

Abstract: Brain computer interfaces (BCIs) offer individuals suffering from major disabilities an alternative method to interact with their environment. Sensorimotor rhythm (SMRs) based BCIs can successfully perform control tasks; however, the traditional SMR paradigms intuitively disconnect the control and real task, making them non-ideal for complex control scenarios. In this study, we design a new, intuitively connected motor imagery (MI) paradigm using hierarchical common spatial patterns (HCSP) and context information to effectively predict intended hand grasps from electroencephalogram (EEG) data. Experiments with 5 participants yielded an aggregate classification accuracy--intended grasp prediction probability--of 64.5\% for 8 different hand gestures, more than 5 times the chance level.

\section{INTRODUCTION}

Brain Computer Interfaces have shown promises in providing individuals with alternative interaction methods via brain activities.
Major electroencephalogram (EEG) based BCI systems can be categorized into three different categories based on the type of brain activity they utilize, event-related potential (ERP) based, steady state visually evoked potential (SSVEP) based, and motor imagery (MI) based.

ERP and SSVEP are more popular in BCI systems where the short response time and the availability of several options are of concern. Controlling a wheelchair~\cite{nezamfar2013brain} and typing with characters~\cite{nezamfar2016flashtype} are two examples that rely on the above features, respectively. In control and navigation applications SSVEP-based BCI systems are widely used due to their fast response.
SSVEP systems are fast, but they require the visual stimulation and vision capabilities~\cite{nezamfar2015stimuli} not suitable for individuals with severely impaired vision.

MI, the process of imagining a physical movement without execution, can be categorized as imagining moving different limbs (simple MI) and imagining different movements of the same limb (compound MI)\cite{yi2013eeg}. MI tasks desynchronize the stable, resting alpha/mu ($7-12$ Hz) and beta ($12-30$ Hz) rhythms, that can be measured through EEG potentials on the scalp\cite{yi2013eeg,pfurtscheller1999event}. The spatial distribution of brain activity measured by EEG signals in response to these imaginations depend on the complexity of the MI task; simple MI (imagination of left hand vs. right leg) produces activity in different spatial locations while compound MI (open palm vs. closed fist) produces activities in similar spatial cortical areas~\cite{vuvckovic2012two}. The high spatial correlation of compound MI coupled with the volume conduction effect, the spatial distortion of EEG signals on the scalp caused by travelling through tissue and fluid, make different MI tasks difficult to distinguish.

One particular application of BCIs is the control of prosthetics or exoskeletons for individuals with motor control loss.  From the BCI perspective, a brain controlled hand requires rapidly (real-time) and accurately classified MI that maps directly to the motion of the target limb. Simple MI is not sufficiently intuitive, and compound MIs are difficult to differentiate. Hence classifying EEG potentials suitable to control a robotic limb is a challenge.

Several studies have classified gestures with moderate success, but these gestures are not sufficiently intuitive to be used in a BCI prosthetic. These studies investigated classifying simple MI by simply distinguishing imagination of right and left hand movements with $80\%-93\%$ accuracy\cite{vuvckovic2012two,pfurtscheller1997eeg,hamedi2014neural}. Others have completed binary classifications on different same-limb movements, such as distinguishing between wrist flexion, extension, supination, pronation, and finger movement\cite{vuvckovic2012two,vuckovic2008delta,mohamed2011single} resulting in accuracies between $58-82\%$. However, the higher accuracies are only achieved when wrist extension has been one of the binary options; without wrist extension classification accuracies tend to run closer to the lower side of the reported range. Binary classification does not provide enough degrees of freedom to control a prosthetic hand, and the imaginary movements used for classification are not necessarily intuitive hand motions. A recent study~\cite{edelman2016eeg} classified four hand states (wrist flexion, extension, supination, and pronation) using source imaging analysis~\cite{edelman2014discriminating} to improve signals' separability. Even with this combination, the imagined  wrist motions do not map directly or intuitively enough to natural hand movements.
In this paper, we propose classification of four grasps on each hand totaling to 8 options. The grasps mimic four of the required grasps to complete everyday tasks. The goal is to improve classification accuracy and time to control a prosthetic hand, utilizing the natural choice of grasps.

\section{Method}

\subsection{Experiment design}
Classifier parameters are estimated using data collected during a supervised calibration session. The classifier is meant to discriminate EEG signals corresponding to imagination of four gestures shown in Fig.~1.a. Prior to data collection, participants get familiarized with gestures through executing each gesture two times, for 5 seconds. During the calibration session, participants are presented with the \emph{target} gesture for 2 seconds during which they are asked to focus on that particular gesture. Then, upon the presentation of the word \emph{start} on the display participants imagine the target gesture for 5 seconds (Fig.~1.b). Here, each MI task for a gesture is called a \emph{trial}.
Inter trial interval in the calibration task is arbitrary and is decided by the user based on their required time to rest. Each participant performs 20 trials of each gesture, summing to a total of 160 MI trials.

\begin{figure}
    \centering
     \begin{subfigure}[]
         \centering
         \includegraphics[width=0.8\columnwidth]{fig/Gestures.jpg}

     \end{subfigure}
          \vfill
            \begin{subfigure}[]
             \centering
             \resizebox{.8\columnwidth}{!}{
             \begin{tikzpicture}
                 \draw[ultra thick,->] (0,0) -- (8,0);
                 \draw[dashed] (7,0.35) -- (7.5,0.350);
                 \foreach \x in {0,2}
                 \draw (\x cm,1pt) -- (\x cm,-1pt) node[anchor=north]{$\x$};
                 \draw (6 cm,1pt) -- (6 cm,-1pt) node[anchor=north]{$7$};
                 \draw[thick] (0,0.1) rectangle (2,0.6);
                 \node[text width=2cm] at (1.5,0.32) {Prepare};
                 \node[text width=4cm] at (4.5,0.32) {Motor Imagination};
                 \node[text width=1cm] at (6.65,0.32) {Rest};
                 \draw[thick] (2,0.1) rectangle (6,0.6);
                 \draw[thick] (6,0.1) -- (7,0.1);
                 \draw[thick] (6,0.6) -- (7,0.6);
                 \node[text width=2cm] at (7.75,-0.3){Time (s)};
             \end{tikzpicture}
             }
             \caption{ a) gestures for each hand. b) The experimental paradigm}

        \end{subfigure}
    \label{fig:4gestures}
\end{figure}

\subsection{EEG feature extraction}
Prior to feature extraction, EEG signals are filtered with a $3-30$~Hz FIR band-pass filter to retain the alpha/mu and beta frequency bands, as they show the most activity in response to imagination and execution of hand gestures~\cite{pfurtscheller1997eeg}. Assuming the band-passed EEG signal follows a Gaussian distribution within the time window, and source activity constellations between different class pairs are independent, we can design a spatial filter (spatial transform) that maximizes the signal's variance for one class and minimizes for the other. We used the common spatial pattern~(CSP) algorithm~\cite{ramoser2000optimal} to calculate the spatial transforms. CSP algorithm is used to find the transformation matrix between class pairs.
CSP finds the best coordinates between binary classes.
In the multiclass case, we propose a Hierarchical Common Spatial Pattern~(HCSP) to utilize a series of binary transformations found by CSP.

 In HCSP, classes with common properties are placed in one category. For example, first level contains two categories, motions on the right hand and motions on the left hand. Similarly, extension or flexion of fingers and abduction or adduction of thumbs are considered different categories in the second and the third level respectively. Fig~\ref{fig:Hierarchy} outlines the HCSP for classification of different hand gestures. The goal of each classifier is to classify the categories not classes. By cascading the category classifiers, all 8 classes can be detected.
 \begin{figure}
   \centering
   \resizebox{.8\columnwidth}{!}{
\begin{tikzpicture}[sibling distance=9em,
  branch/.style = {shape=rectangle, rounded corners,
    draw, align=center,
    top color=white, bottom color=blue!20}]]
  \node[branch] {Decision}
    child { node[branch] {Left Hand}
        child { node[branch] {Fingers Extension}
            child { node[branch] {Thumb Abduction}}
            child { node[branch] {Thumb Adduction}}}
        child { node[branch] {Fingers Flexion}}}
    child { node[branch] {Right Hand}};

\node[text width=1cm,font=\bf] at (2,-2.1) {$\ddots$};
\node[text width=1cm,font=\bf] at (0.4,-3.6) {$\ddots$};
  \node[text width=1cm] at (-1.5,-0.75) {$L^1$:};
    \node[text width=1cm] at (-3.5,-2.25) {$L^2$:};
    \node[text width=1cm] at (-5.5,-3.75) {$L^3$:};
\end{tikzpicture}
}
        \caption{Hierarchical classification model}%
        \vspace{-.5cm}
        \label{fig:Hierarchy}%
\end{figure}

 The normalized spatial covariance matrix is estimated as
\begin{equation}
C_j = \frac{1}{N_j}\sum_{i=1}^{N_j}\frac{E_i^{(j)}{E_i^{(j)}}'}{trace(E_i^{(j)}{E_i^{(j)}}')}
\end{equation}
where $E_i^{(j)}$ is an $m\times t$ matrix representing filtered windowed EEG evidence corresponding to the $i^{th}$ trial  of the $j^{th}$ class. Additionally, $m$ is the number of channels, $t$ is the number of samples in a time window, $'$ is the transpose operator, $trace(.)$ is the sum of the diagonal elements of the matrix, and $N_j$ is the number of trials for the $j^{th}$ class.

In order to find the spatial transformation matrix at each level,$V_L$, the optimization problem in equation~\ref{eqe:CSP} should be solved.
\begin{equation}
    V_L^* = \underset{V_L}{\operatorname{argmax}}\frac{V_L^T \sigma_{L_{l=-1}} V_L}{{V_L}' (\sigma_{L_{l=-1}}+\sigma_{L_{l=+1}}) V_L}
    \label{eqe:CSP}
\end{equation}
where $\sigma_{L_{l=-1}}$ and $\sigma_{L_{l=+1}}$ are the average covariance matrices - where $\pm1$ represent one of the categories - in level $L$ and are calculated by
\begin{equation}
    \sigma_{L_{l}} = \frac{\sum_{j\in category({L_{l}})}N_j C_j}{\sum_{j\in category({L_{l}})}N_j}
    \label{eqe:sigma}
\end{equation}
$\sum_{j\in category({L_{l}})}N_j$ is the number of trials for each category at each level.

Solving the optimization problem represented in equation~\ref{eqe:CSP} is identical to solving generalized eigenvalue problem
\begin{equation}
    V_{L}'\sigma_{L_{l=-1}}V_L = D \wedge V_{L}'(\sigma_{L_{l=-1}}+\sigma_{L_{l=+1}})V_L = I
    \label{eqe:GEP}
\end{equation}
where $D$ is a diagonal matrix.

 According to each spatial filter $V_L$, computed for level $L$, the projected data $P_{L_l}$ corresponding to the $i^{th}$ trial of the $j^{th}$ class, $E_i^{(j)}$, can be calculated as
\begin{equation}
    P_{L_l} = {V_L}' E^{(j\in category(L_l))}
    \label{eqe:projected data}
\end{equation}
In equation~\ref{eqe:GEP} the $k$ smallest and largest eigenvalues in $D$ correspond to $k$ leftmost/rightmost columns in spatial filter $V$ respectively. These values yield to the smallest variance in category $L_{l=-1}$ and simultaneously largest variance in category $L_{l=+1}$ and vice versa. To extract the features, the first and the last $k$ rows of $P_{L_l}$ are considered.

Using $P_{L_l}^{(K)}$ ($K=1 \hdots 2k)$ which maximizes the difference of variances between two categories, $f_{L_l}^{(K)}$, features for each trial of the corresponding category at each level are calculated as
\begin{equation}
    f_{L_l}^{(K)} = log \left (\frac{var(P_{L_l}^{(K)})}{\sum_{q=1}^{2k}var(P_{L_l}^{(q)})}\right )
    \label{eqe:feature}
\end{equation}
where, $i$ is the trial index in each the category.
\subsection{EEG Likelihood Probability Extraction}
\label{sec:method_classification}
For proper estimation of the classifier performance a leave-one-out cross validation approach is used. At each level, the corresponding feature vector, $f_{L_{l_i}}^{(K)}$, is calculated for each trial. The classifier is trained in two steps. First, Fisher LDA algorithm is used to extract fisher scores by
\begin{equation}
F_{L_{l_i}}^{(K)} = \left (w_L^{(K)}\right )' f_{L_{l_i}}^{(K)}
\end{equation}
where
\begin{equation}
    w_L^{(K)} \propto \left (\Sigma_{L_{l={-1}}}^{(K)} + \Sigma_{L_{l={+1}}}^{(K)}\right )^{-1} \left (\mu_{L_{l={+1}}}^{(K)}-\mu_{L_{l={-1}}}^{(K)}\right )
\end{equation}
and $\Sigma_{L_{l={-1}}}^{(K)}, \mu_{L_{l={-1}}}^{(K)}$ and $\Sigma_{L_{l={+1}}}^{(K)}, \mu_{L_{l={+1}}}^{(K)}$ are the category's covariance and mean.

Then, the likelihood probability densities are calculated for the fisher scores in each category $P(F_{L_{l}}^{(K)}|L=l)$.
Multi-variate Gaussian model is considered at each level for density estimation due to the nature of EEG signals.
Classification accuracy is defined as the probability of the correct answer.\par

Different integer window lengths from 1 to 5 seconds are considered for feature extraction.
\subsection{Graphical model}
\label{sec:graphical_model}
 The graphical model in Fig~\ref{fig:GraphicalModel} is proposed to probabilistically merge the category classifiers and employ the inter-level information to calculate the posterior probability of each gesture.
    \begin{figure}%
    \centering
        \resizebox{.4\columnwidth}{!}{

\begin{tikzpicture}[thick,scale=0.9, every node/.style={transform shape}]

\tikzstyle{a} = [rectangle, draw, minimum height=10em, minimum width=25em]

 \node [level1] (hand) {\Huge $L_e^1$};
 \node [level1, left = 1cm of hand] (fingers) {\Huge $L_e^2$};
 \node [level1, left = 1cm of fingers] (thumb) {\Huge $L_e^3$};

 \node [desire, above = 1.5cm of hand] (Decision) {\Huge $s_e$};

\node [response, below = 1.5cm of  hand] (EEGHand) {\Huge $\epsilon_{e_i}^1$};
\node [response, below = 1.5cm of  fingers] (EEGFingers) {\Huge $\epsilon_{e_i}^2$};
\node [response, below = 1.5cm of  thumb] (EEGThumb) {\Huge $\epsilon_{e_i}^3$};

\draw [arrow] (hand) -- (fingers) ;
\draw [arrow,dashed] (Decision) -- (hand);
\draw [arrow,dashed] (Decision) -- (fingers);
\draw [arrow,dashed] (Decision) -- (thumb);
\draw [arrow] (fingers) -- (thumb);

\draw [arrow] (hand) -- (EEGHand);
\draw [arrow] (fingers) -- (EEGFingers);
\draw [arrow] (thumb) -- (EEGThumb);
\node[a,fit =(EEGHand) (EEGFingers) (EEGThumb) ,label={[label distance=0, text=red]300:\Huge $i=1:t$}] (container) {};
\end{tikzpicture}
}

        \caption{System graphical model in epoch $e$. The final state $s_e$ is composed of the state in each level $L_{e}^{M}$ and thus deterministically related (dashed lines). States are calculated from the observed random variables and extracted EEG features of each level $\epsilon_{e_i}^M$. States and random variables are probabilistically related (solid lines). }%
        \vspace{-.5cm}
        \label{fig:GraphicalModel}%
    \end{figure}
This model represents the generative model of the collected EEG data in epoch $e$ and the time required for the model to make a decision ($1-5$ s).
The goal of this graphical model is to estimate the next state $s_e$ with incorporation of inter-level prior information.
The inter-level information is the prior probability of a gesture at a level given the state at the higher level.
This information is gathered based on the combination of gestures used over time for different tasks.
For instance, the probability of the thumb being closed would be increased if closed fingers is detected at the higher level.
Specifically in this graphical model:$s_e$ is the decision between all classes at epoch $e$, $L_e^j$ represents the decision from the level j classifier, and $\epsilon_{e_i}^j$ is the extracted CSP feature from EEG evidence at time $i$ for decision $L_e^j$.

Using the graphical model above, the posterior probability of each state given the observed random variables ($\{\epsilon_{e_i}^1\}_{i=1}^{t},\{\epsilon_{e_i}^2\}_{i=1}^{t},\{\epsilon_{e_i}^3\}_{i=1}^{t}$) can be calculated to construct the PMF over all gestures, required to make a decision.  The posterior probabilities are calculated as follows:
\begin{equation}
\begin{split}
    P(s_e&\ |\{\epsilon_{e_i}^1\}_{i=1}^{t},\{\epsilon_{e_i}^2\}_{i=1}^{t},\{\epsilon_{e_i}^3\}_{i=1}^{t}) \\
    &\ = P(L_e^1,L_e^2,L_e^3|\{\epsilon_{e_i}^1\}_{i=1}^{t},\{\epsilon_{e_i}^2\}_{i=1}^{t},\{\epsilon_{e_i}^3\}_{i=1}^{t}) \\
    &\ \propto P(\{\epsilon_{e_i}^1\}_{i=1}^{t},\{\epsilon_{e_i}^2\}_{i=1}^{t},\{\epsilon_{e_i}^3\}_{i=1}^{t}|L_e^1,L_e^2,L_e^3)\\
    &\ \times P(L_e^1,L_e^2,L_e^3) \\
    &\ \propto P(\{\epsilon_{e_i}^1\}_{i=1}^{t}|L_e^1)P(\{\epsilon_{e_i}^2\}_{i=1}^{t}|L_e^2)P(\{\epsilon_{e_i}^3\}_{i=1}^{t}|L_e^3) \\
    &\ P(L_e^3|L_e^2)P(L_e^2|L_e^1)P(L_e^1)
\end{split}
\end{equation}

In this set of equations: $P(\{\epsilon_{e_i}^M\}_{i=1}^{t}|L_e^M)$, represents EEG observation likelihood for a particular gesture at level $M$, estimated using the calibration data; $P(L_e^M|L_e^{M-1})$, is the probability of each command at level $M$ given the inter-level information; and $P(L_e^1)$, represents the prior probability of each hand being target.
\subsection{Decision Criteria}\label{sysDec}
One decision is made in every epoch. Decisions are made using a Maximum-a-Posteriori (MAP) estimate.
	\begin{equation}
		\hat{s}_e=\arg\max_{s_e}{P(s_e|\{\epsilon_{e_i}^1\}_{i=1}^{t},\{\epsilon_{e_i}^2\}_{i=1}^{t},\{\epsilon_{e_i}^3\}_{i=1}^{t})}
		\label{eq:decision}
	\end{equation}
At each time interval $t$, if the posterior probability of $\hat{s}_e$ is greater than the system \emph{Confidence Threshold}, it will be chosen as the final decision of the epoch.
If after $N$ time interval (here $N=5$) the posterior probability is not greater than the system \emph{Confidence Threshold}, the maximum posterior probability of $\hat{s}_e$ across all time intervals will be chosen as the final decision of the epoch.
Analysis and results
\section{Analysis and Results}
Five healthy participants (3 males and 2 females) in the range of 20-30 years old consented and participated in a data collection session following an approved protocol by Northeastern University's IRB office. Participants were not under the influence of any chemicals, such as caffeine.

Before starting the experiment, different gestures were described to the participants; participants executed these gestures to get more familiar with the motions to be imagined during the imagination task. Participants were asked to imagine the entire progress of the gestures, not only the final position. Between each trial (2 seconds preparation, 5 second gesture imagination), participants rested until they felt comfortable to continue. On average, the experiment consisting of 160 trials, 4 different gestures on each hand took less than 45 minuets.

All possible number of features ($K$) and window lengths ($t\geqslant3s$) combinations were evaluated offline to find the parameter values leading to the highest classification accuracy.
Figure~\ref{fig:accuracy8} shows the total system accuracy as a function of the window length and number of features.
As expected, more EEG evidence (larger $t$) results in higher accuracy.
The average maximum accuracy of 64.5\% (71\% the highest) is achieved using $K=6$ and the window length of $t=5$ seconds using uniform inter-level prior probability.
\begin{figure}
    \centering
    \includegraphics[width=0.9\columnwidth]{fig/All_acc.jpg}
    \caption{Average system accuracy using different number of features (x-axis) and different time-window lengths of EEG evidence.}
    \label{fig:accuracy8}
\end{figure}
The average confusion matrix is shown in figure~\ref{table:8 classes}.
The confusion matrix shows the system accurately classifies between left and right hand (level 1), hence the two approximately zero blocks on the top-right and bottom-left of the matrix (Green squares).
The highest confusion is between classes that differ in thumb position (level 3), as shown by the highlighted red blocks.
\begin{figure}
    \centering
    \includegraphics[width=0.9\columnwidth]{fig/Confusion_MI.jpg}
    \caption{Average confusion matrix for 5 participants with using $K=6$ features and the window length of $t=5$ seconds}
    \label{table:8 classes}
\end{figure}
The confusion matrix represented in figure~\ref{table:8 classes} shows gestures with differences in details are not as separable as the more general levels like left and right hand.

\section{DISCUSSION}
With the goal of utilizing intuitive and natural gestures towards building motor imagery based classifiers.
The natural gestures are more complex they reflect on the same region and require more sophisticated classifiers as opposed to the tasks performed or imagined on the similar limbs on the different sides.
We proposed a hierarchical model combining binary classifiers on different levels taking advantage of the similarities in different gestures.
Binary CSP is used to extract features at each level.
The results from each classifier is probabilistically merged, and inter-level probabilities are employed to achieve a multiclass gesture classifier. In a study with 5 healthy participants our method resulted in an average accuracy of $64.5\%$ among 8 complex hand gestures, more than 5 times the chance level.

\addtolength{\textheight}{-12cm}

\bibliographystyle{IEEEtran}

Decoding Complex Imagery Hand Gestures