Our studies were supported by the University of Washington (including the Paul G. Allen School of Computer Science and Engineering; Department of Electrical and Computer Engineering; Population Health; Addictions, Drug and Alcohol Institute; and the Center for Research and Education on Accessible Technology and Experiences); the National Science Foundation (EDA-2009977, CHS-2016365, CHS-1941537, IIS1816687 and IIS7974751), the National Institute on Disability, Independent Living and Rehabilitation Research (90DPGE0003-01), Samsung Research America, and Google.




Checklist

For all authors...

Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Did you describe the limitations of your work?
See Sec.

Did you discuss any potential negative societal impacts of your work?
See Sec.

Have you read the ethics review guidelines and ensured that your paper conforms to them?
See Sec.


If you are including theoretical results...

Did you state the full set of assumptions of all theoretical results?

Did you include complete proofs of all theoretical results?


If you ran experiments (e.g. for benchmarks)...

Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
See Sec.

Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
See Sec.REF  and S. REF

Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
See Tab. REF

Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
See S. REF


If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

If your work uses existing assets, did you cite the creators?

Did you mention the license of the assets?

Did you include any new assets either in the supplemental material or as a URL?

Did you discuss whether and how consent was obtained from people whose data you're using/curating?

Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?


If you used crowdsourcing or conducted research with human subjects...

Did you include the full text of instructions given to participants and screenshots, if applicable?
See S. REF

Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
See S. REF

Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Our compensation is not billed hourly, but for the whole study. See Sec. REF



Additional Study Details
Study Documents
We provide a few important documents used in our data collection studies. Please find these files in the supplementary folder:


University IRB Approval Letter: The letter from University IRB to approve our studies.

Consent Form: The form to be signed to participants before joining the study.

Compensation Structure: Participants will earn up to $245 based on their participation compliance.

Participant instruction (iOS version, and Android version): Slide decks to guide participants through the app installation and Fitbit setup during the on-boarding.


Study Demographics

<TABLE>

Study Hardware and Setup
<FIGURE> 
 Our smartphone data collection app is compatible with both iOS and Android platforms. Therefore, we did not have limits on participants' devices.
Before each year's study, we tested our app on multiple smartphone brands to ensure its compatibility, robustness, and data collection quality.
However, problems such as smartphone battery drain, software crashes, and data uploading error are inevitable during the study. Thus, we developed a study dashboard to monitor the condition of data collection during the study, and our study team would reach out to help participants solve software or hardware when necessary.

Figure REF  presents a screenshot of the app. The interface is consistent on both platforms. Users can click 1) the “Save” button to manually trigger data uploading, 2) the “Open Survey” button to manually enter the survey if that's within designated time windows, (note that participants usually received EMAs through notifications), and 3) the “Refresh Fitbit Token” for Fitbit data access update.


As for wearables, we used two models of Fitbit (Flex2 for Year 1&2 and Inspire2 for Year 3&4). Both models support reliable physical activity and sleep behavior tracking, but not others (e.g., heart rate tracking).
Our internal team also tested and compared the two Fitbit models' tracking accuracy and did not observe significant difference.


Study Intrinsic Bias
We discuss some potential intrinsic bias in our datasets. For example:

Recruitment Bias: Only a portion of students who received our emails or social media posts would participate in our study, which could only represent a subset of the general population.

Gender Group Bias: Our studies intentionally over-sample females, which could involve bias towards the female group.

Generation Group Bias: Our studies intentionally over-sample immigrants and first-generation participants, which could involve bias against other generation groups.

Racial Group Bias: Asian and White are two dominant racial groups in our studies, while other racial groups are less represented. This could introduce racial bias.

Health Group Bias: Some health conditions would impact participants' compliance. For example, participants with severe depressive symptoms may stop responding to surveys or even charging their phones, which would introduce bias into the missing data rate.

Device Bias: Although our data collection app is compatible with both iOS and Android platforms, the differences between OS systems and smartphone models may introduce bias into the dataset.

We look forward to future exploration of these different aspects of intrinsic bias.

Additional Correlation Analysis

In addition to identifying features that have a consistent correlation with the depression label across all years' datasets (see Figure REF ), we are also interested in the features that have opposite correlation directions between pre-COVID and post-COVID periods.
We followed a similar procedure as Sec. REF  to find features that have a consistent and significant correlation direction within two years (DS1&2, or DS3&4) but an opposite direction between pre- and post- COVID datasets.
Figure REF  shows one representative feature from each data type.

<FIGURE>
There are some interesting findings, especially when compared against Figure REF .
For example, Figure REF  indicates that generally more frequent and longer smartphone usage is positively correlated with depression labels. However, in the morning time, this finding only holds before COVID. After the outbreak of COVID, frequent usage of a smartphone becomes negatively correlated with depression. This may be explained by the fact that the smartphone becomes a necessary tool for all kinds of daily routines when people are locked at home, which could overturn the correlation direction as participants with depression may tend to lose interest in general activities [1]}.
We look forward to more analysis and insights from future researchers.


Survey Details
We list out the survey names and short descriptions used in our study. Please find specific question items in the supplementary folder.
c|m0.45|c|c|c
Description of Survey Scales

1c|Scale Name & Abbreviation  1c|Short Description  1c|Scoring Range  1c|Year  1c[c]Collection
Time
UCLA [2]}
Short-form UCLA
Loneliness Scale  A 10-item scale measuring one's subjective feelings of loneliness as well as social isolation. Items 2, 6, 10, 11, 13, 14, 16, 18, 19, and 20 of the original scale are included in the short form. Higher values indicate more subjective loneliness.  10 - 40  35*1,2,3,4  35*pre, post




Social Fit [3]}
Sense of Social
and Academic Fit Scale  A 17-item scale measuring the sense of social and academic fit of students at the institution where this study was conducted. Higher values indicate higher feelings of belongings.  17 - 119




2-Way SSS [4]}
2-Way Social
Support Scale  A 21-item scale measuring social supports from four aspects (a) giving emotional support, (b) giving instrumental support, (c) receiving emotional support, and (d) receiving instrumental support. Higher values indicate more social support.  [c](a) 0 - 25
(b) 0 - 25
(c) 0 - 35
(d) 0 - 20




PSS [5]}
Perceived Stress Scale  A 14-item scale used to assess stress levels during the last month. Note that Year 1 used the 10-item version. Higher values indicate more perceived stress.  [c]0 - 56 (Year 2,3,4)
0 - 40 (Year 1)




ERQ [6]}
Emotion Regulation
Questionnaire  A 10-item scale assessing individual differences in the habitual use of two emotion regulation strategies: (a) cognitive reappraisal and (b) expressive suppression. Higher scores indicate more habitual use of reappraisal/suppression.  [c](a) 1 - 7
(b) 1 - 7




BRS [7]}
Brief Resilience Scale  A 6-item scale assessing the ability to bounce back or recover from stress. Higher scores indicate more resilient from stress.  1 - 5




CHIPS [8]}
Cohen-Hoberman Inventory
of Physical Symptoms  A 33-item scale measuring the perceived burden from physical symptoms, and resulting psychological effect during the past 2 weeks. Higher values indicate more perceived burden from physical symptoms.  0 - 132




STAI [9]}, [10]}
State-Trait Anxiety
Inventory for Adults  A 20-item scale measuring State-Trait anxiety. Year 1 used the State version, while other years used the Trait version. Higher values indicate higher anxiety.  20 - 80




CES-D [11]}, [12]}
Center for Epidemiologic
Studies Depression Scale
Cole version  A 10-item scale measuring current level of depressive symptomatology, with emphasis on the affective component, depressed mood. Year 2 used the 9-item version. Higher scores indicate more depressive symptoms.  [c]0 - 30 (Year 1,3,4)
0 - 27 (Year 2)




BDI2 [13]}
Beck Depression
Inventory-II  A 21-item detect depressive symptoms. Higher values indicate more depressive symptoms. 0-13: minimal to none, 14-19: mild, 20-28: moderate and 26-63: severe.  [c]0 - 63




MAAS [14]}
Mindful Attention
Awareness Scale  A 15-item scale assessing a core characteristic of mindfulness. Year 1 used a 7-item version, while other years used the full version. Higher values indicate higher mindfulness.  1 - 6
BFI10 [15]}
The Big-Five Inventory-10  A 10-item scale measuring the Big Five personality traits Extroversion, Agreeableness, Conscientiousness, Emotional Stability, and Openness. The higher the score, the greater the tendency of the corresponding personality.  1 - 5  1,2,3,4  pre
Brief-COPE [16]}
Brief Coping Orientation
to Problems Experienced  A 28-item scale measuring (a) adaptive and (b) maladaptive ways to cope with a stressful life event. Higher values indicate more effective/ineffective ways to cope with a stressful life event.  [c](a): 0 - 3
(b): 0 - 3  18*2,3,4  18*pre, post




GQ [17]}
Gratitude Questionnaire  A 6-item scale assessing individual differences in the proneness to experience gratitude in daily life. Higher scores indicate a greater tendency to experience gratitude.  6 - 42




FSPWB [18]}
Flourishing Scale
& Psychological
Well-Being Scale  An 8-item scale measuring the psychological well-being. Higher scores indicate a person with “more psychological resources and mental strengths”.  8 - 56




EDS [19]}, [20]}
Everyday Discrimination
Scale A 9-item scale assessing everyday discrimination. Higher values indicate more frequent experience of discrimination.  0 - 45




CEDH [21]}, [20]}
Chronic Work Discrimination
and Harassment  A 12-item scale assessing experiences of discrimination in educational settings. Higher values indicate more frequent experience of discrimination in the work environment.  0 - 60




B-YAACQ [23]}
The Brief Young Adult
Alcohol Consequences
Questionnaire (optional)  A 24-item scale measuring the alcohol problem severity continuum in college students. Higher values indicates more severe alcohol problems.  0 - 24
PHQ-4 [24]}, [25]}
Patient Health
Questionnaire 4  A 4-item scale assessing (a) mental health, (b) anxiety, and (c) depression. Higher values indicate higher risk of mental health, anxiety, and depression.  [c](a): 0 - 12
(b): 0 - 6
(c): 0 - 6  7*2,3,4  7*Weekly
EMA




PSS-4 [26]}, [5]}
Perceived Stress Scale 4 A 4-item scale assessing stress levels during the last month. Higher values indicates more perceived stress.  0 - 16




PANAS [28]}, [29]}
Positive and Negative
Affect Schedule  A 10-item scale measuring the level of (a) positive and (b) negative affects. Higher values indicates larger extent.  [c](a): 0 - 20
(b): 0 - 20



Sensor Feature Details
The following tables list out specific features based on RAPIDS [30]}.
All features are extracted with multiple time_segments: morning (6 am - 12 pm), afternoon (12 pm - 6 pm), evening (6 pm - 12 am), night (12 am - 6 am), allday, 7-day history, 14-day history, weekday, and weekend (the last two are calculated once a week).
Moreover, all numeric features have two extra versions: 1) normalized (subtracted by each participant's median and divided by the 5-95 quantile range); 2) discretized (low/medium/high split by 33/66 quantile of each participant's feature value).
We employ a specific naming format of all features:

[feature_type]:[feature_name][_norm or NULL]:[time_segment]

<TABLE><TABLE><TABLE>

PS.1. It is worth noting that the missing rate of call-related features are high. This is mainly because most these features are event-based. If a participant did not receive a phone call at a day, that day will have empty call features.

PS.2. One limitation of our physical activity and sleep feature data comes from a Fitbit issue: If the data on the wearable device is not synced with the smartphone over a few days, it would trigger some internal space-saving strategy to discard low-level details and only contain high-level summary data, leading to information loss and affecting feature correctness. This would be reflected by the missing features (e.g., small or missing countepisodeactivebout), which is not common in our datasets.

Additional Model & Benchmark Information
We provide a more detailed description of benchmark-related processing. Many texts are taken from [31]} with courtesy.
Depression Detection Ground Truth Processing
Due to some design iteration, we did not include PHQ-4 in DS1, but only PANAS.
Although PANAS contains questions related to depressive symptoms (e.g., “distressed”), it does not have a comparable theoretical foundation for depression detection like PHQ-4 or BDI-II.
Therefore, to maximize the compatibility of the datasets, we trained a small ML model on DS2 that has both PANAS and PHQ-4 scores to generate reliable ground truth labels.
Specifically, we used a decision tree (depth=2) to take PNANS scores on two affect questions (“depressed” and “nervous”) as the input and predict PHQ-4 score-based depression binary label. Our model achieved 74.5% and 76.3% for accuracy and F1-score on a 5-fold cross-validation on DS2.
The rule from the decision tree is simple: the user would be labeled as having no depression when the distress score is less than 2, and the nervous score is less than 3 (on a 1-5 Likert Scale).
We then applied this rule to DS1 to generate depression labels.

Behavior Modeling Algorithm Implementation Details
Please refer to our GLOBEM codebease for the specific implementations and hyperparameter tuning.
Depression Detection Algorithms

Canzian et al. [32]}
Features: Location trajectory features directly computed from the past two-week time window.
Model: A support vector machine (SVM).

Saeb et al. [33]}
Features: Location and screen features aggregated with daily average of the past two weeks.
Model: A logistic regression model with elastic regularization.

Farhan et al. [34]}
Features: Location and physical activity features from the past two-week window.
Model: An SVM.

Wahle et al. [35]}
Features: Several feature types (activity, location, WiFi, screen, and call) over the past two weeks. Both daily aggregation (i.e., mean, sum, variance) and direct computation of the features of the two weeks are used. WiFi features are excluded to ensure the compatibility with our datasets.
Model: SVM and Random Forest.

Lu et al. [36]}
Features: Location, activity, and sleep features computed from the past two weeks.
Model: Multi-task learning combining linear regression & logistic regression. One model for iOS and one for Android are built to deal with device platform differences,

Wang et al. [37]}
Features: Location, screen, activity, sleep, and audio features aggregated by calculating daily average and slope of the past two weeks. Audio features are excluded as they are not collected.
Model: A lasso-regularized logistic regression model.

Xu et al.-I (Interpretable) [38]}
Features: Location, screen, activity, and sleep features in multiple epochs of a day (morning, afternoon, evening, night). Association rule mining is applied to mine out interpretable behavior rules that capture differences between participants with depression and without depression. Then, the rules are used to filter and aggregate features of multiple days.
Model: An Adaboost model.

Xu et al.-P (Personalized) [39]}
Features: A similar set of basic features as [38]}. With each feature as a time sequence, a user behavior relevance matrix is computed using the square of Pearson correlation to capture users with strong positive or negative correlation.
Model: aAtraditional collaborative-filtering-based model to select features and obtain an intermediate prediction using each feature, and combine the results of all features via majority voting.

Chikersal et al. [41]}
Features: A similar set of basic features as [38]}. Aggregations (breakpoint and slope) across multiple time ranges (daily and biweekly) are calculated, followed by a nested randomized logistic regression for feature selection.
Model: Separate gradient boosting and logistic regression models using data from every sensor, and combine the prediction with another Adaboost model to generate the final prediction.

Each algorithm will lead to one model. All these models' hyperparameters are tuned via grid search with the same range as mentioned in each prior work.

Domain Generalization Algorithms
The data format of all deep-learning based algorithm is the same: a subset of important daily features in the most recent traditional depression detection algorithms [41]}, [39]}, with the past-four-week feature matrix as the input.

It is worth noting that we picked these deep learning techniques to cover the major approaches of domain generalization [45]}, including 1) data manipulation (Mixup), 2) representation learning (IRM, DANN, CSD), and 3) learning strategy (MLDG, MASF, Siamese, Reorder).


ERM (Empirical Risk Minimization) [46]}
The basic model training techniques without particular design for domain generalization. ERM shows a competitive performance in previous CV generalization tasks [47]}, [45]}. Multiple architectures with ERM are implemented: a) ERM-1D-CNN: one-dimensional CNN that treats the data as a time-series of length 28; b) ERM-2D-CNN: two-dimensional CNN that treats the data as an one-channel image; c) ERM-LSTM: another architecture to model time-series data; d) ERM-Transformer: a transformer-based architecture for modeling sequence data.

Mixup (ERM-Mixup) [49]}
A data augmentation technique that performs linear interpolation between two instances with a weight sampled from a Beta distribution.
1D-CNN is used as the architecture as it is robust to feature positions in the feature matrix. Same for the rest algorithms.

IRM (Invariant Risk Minimization) [50]}
A representation learning paradigm to estimate invariant correlations across multiple distributions and learn a data representation such that the optimal classifier can match all training distributions.

DANN (Domain-Adversarial Neural Network) [51]}
Another representation learning technique that adversarially trains the generator and discriminator. The discriminator is trained to distinguish different domains, while the generator is trained to fool the discriminator to learn domain-invariant feature representations. Two setups are tested, one treating each dataset as a domain (DANN-D (Dataset as Domain)), and one treating each person as a domain (DANN-P (Person as Domain)).

CSD (Common Specific Decomposition) [52]}
A feature disentanglement-based representation learning technique from the multi-component analysis perspective, which extracts the domain-shared and domain-specific features using separate network parameters. Similar to DANN, it can support CSD-D and CSD-P.

MLDG (Meta-Learning for Domain Generalization) [53]}
One of the first methods using meta-learning strategy for domain generalization. MLDG splits the data of the training domains into meta-train and meta-test to simulate the domain shift to learn general features. It supports MLDG-D, and MLDG-P.

MASF (Model-Agnostic Learning of Semantic Features) [54]}
A learning strategy that combines meta-learning and feature disentanglement. After simulating domain shift by domain split, MASF further regularizes the semantic structure of the feature space by introducing a global loss (to preserve relationships between classes) and a local loss (to promote domain-independent class clustering). It supports MASF-D, and MASF-P.

Siamese Network [55]}
A metric-learning based strategy to find a better pair-wise distance metric. It aims to decrease the distance between positive pairs and increase the distance between negative pairs.

Reorder [31]}
A recently proposed method to leverage the continuity of behavior trajectory [31]}.
It designed a pretext task which shuffles the temporal order of the feature matrix. Then a model is trained to reconstruct the original sequence, jointly optimized with the main classification task over different domains, as shown in Fig.REF . By capturing the continuity of daily behaviors, the model could learn to extract representations that are generalizable across individuals.
Overall, the model can be trained via the following objective function:
\(\operatornamewithlimits{argmin}_{\theta _f,\theta _c,\theta _r} \sum _{i=1}^{S} \left(\sum _{j=1}^{N_i} \mathcal {L}_c(h(x_j^i|\theta _f,\theta _c), y_j^i) + \sum _{j=1}^{\beta N_i} \alpha \mathcal {L}_r(h(z_j^i|\theta _f,\theta _r), p_j^i)\right)\) 
where both \(\mathcal {L}_c\)  and \(\mathcal {L}_r\)  are cross-entropy losses. \(S\)  is the total number of training domains, and \(N_i\)  is the size of a domain \(i\) . \(\alpha \)  is used to control the weight of the reordering task while \(\beta \)  is used to control the size of reordering data.
\(x\)  is the input matrix, \(y\)  is the classification label, \(z\)  is the feature matrix \(x\)  after the reordering, and \(p\)  is the permutation index (from 1 to 200 among the 200 pre-determined permutation set).
\(x_j^i, y_j^i, z_j^i, p_j^i\)  are specific instances in each domain \(i\)  with index \(j\) .
We picked the number of segmentation as \(n=10\)  \((\lceil 28/3 \rceil )\)  since \(28!\)  or \(14!\)  (\(28/2\) ) is too computationally expensive.

<FIGURE>One algorithm could lead to one or multiple models.
Models from No.2 to No.9 all use the same 1D-CNN as the backbone. We use a simple architecture based on a small-range tuning using ERM-1D-CNN.
It has 3 1D-convolution layers (size 8, stride 3, ReLU activation), each followed by a batch normalization layer, a max-pooling layer, as well as a dropout layer (rate 0.25).
We tested with different layer sizes (8, 16, 32) and depth (3,5,7), and observed similar results, thus we chose size as 8 and depth as 3 to save computing cost.
A fully connected layer (size 16) was attached after flattening the third convolution layer's output to convert it into a vector of length 16. The following layers are customized for each model.
Other architectures are also simple: ERM-2D-CNN used three 2D-convolution layers with the same size, stride, and activation function as 1D-CNN; ERM-LSTM used two bi-directional layers with the hidden size as 20; ERM-Transformer used two transformer blocks, each with 4 self-attention heads (size 4) and a 1D-convolutional feed forward layer (size 16).
For all models, we adopted a common training setup. Specifically, we used Adam as the optimizer and adopted a cosine annealing schedule, with an initial learning rate of 0.001, an annealing decay of 0.95, and a step size of 100.

Training Resources
Since all deep learning models are small, we only used CPU for the model training.
We leveraged a university computing cluster (300 CPUs) with the SLURM Workload Manager. The whole training was completed within 48 hours.

Additional Generalization Results
<TABLE><TABLE><TABLE><TABLE>
Dataset Statements & Documents
Our multi-year data collection study closely followed a sister study in Carnegie Mellon University (CMU). We acknowledge all efforts from CMU Study Team to provide important starting and reference materials.
We state that we bear all responsibility in case of direct violation of participants' privacy right.
Author Contribution Statement
We clarify every author's contribution to the datasets and the paper. Basic contributions like paper proof-reading are default and omitted. Leading conceptualization and effort are bolded.
[leftmargin=10pt]
Xuhai Xu
Data Collection: Led technical parts of data collection in 2019 through 2021. Developed and maintained data collection applications from 2019 to 2021. Assisted with data collection from 2019 to 2021; Assisted database maintenance of all years' datasets.
Analysis and Benchmark: Led curation of dataset, analysis, visualization, benchmarking, and data validation. Main developer of benchmark platform GLOBEM.
Paper Writing & Supplementary Materials: Led paper writing, organization, and design of data sharing process.

Han Zhang
Data Collection: Developed and maintained data codebook and data cleaning (all years). Assisted with the data collection from 2020 to 2021; quality assurance for data collection applications from 2019 to 2021.
Analysis and Benchmark: Led curation of dataset and visualization. Assisted with analysis, benchmarking, and data validation.
Paper Writing & Supplementary Materials: Led curation of dataset details and data sharing agreement in supplementary materials. Assisted with paper writing.

Yasaman Sefidgar
Data Collection: Led design of infrastructure, pipeline and study codebase and codebooks impacting all years of data cleaning and processing; Led planning for 2019 data collection. Led technical parts of data collection in 2018 and 2019. Also maintained database and study servers for 2018 and 2019; assisted with 2018 and 2019 data collection; and assisted with 2020 planning for data collection.
Analysis and Benchmark: Not involved.
Paper Writing & Supplementary Materials: Provided helpful comments.

Yiyi Ren
Data Collection: Led transition of infrastructure for sensor data cleaning to RAPIDS. Assisted with data collection study from 2019 - 2021; developed the study codebase, codebook and mobile applications that impact all years; maintained database and study servers from 2019 to 2021.
Analysis and Benchmark: Not involved.
Paper Writing & Supplementary Materials: Not involved

Xin Liu
Data Collection: Not involved.
Analysis and Benchmark: Provided assistive effort with computing resources support, quality assurance, analysis, visualization, data validation, and GLOBEM development.
Paper Writing & Supplementary Materials: Editing and framing.

Woosuk Seo
Data Collection: Led 2018 data collection planning and data collection.
Analysis and Benchmark: Not involved.
Paper Writing & Supplementary Materials: Not involved

Jennifer Brown
Data Collection: Led 2020 data and 2021 data collection planning and data collection. Assisted with codebook from 2019 to 2021.

Kevin Kuehn
Data Collection: Led 2019 data collection planning and data collection.
Analysis and Benchmark: Not involved.
Paper Writing & Supplementary Materials: Not involved

Mike Merrill
Data Collection: Not involved.
Analysis and Benchmark: Assisted with data analysis, visualization, and benchmarking.
Paper Writing & Supplementary Materials: Assisted with paper writing and study documentation in supplementary materials.

Paula Nurius
Data Collection: Supervised study material design and high-level planning for 2018-2021. Provided resources for study
Analysis and Benchmark: Not involved.
Paper Writing & Supplementary Materials: Supervised data-sharing agreement.

Shwetak Patel
Data Collection: Not involved.
Analysis and Benchmark: Supervised data analysis. Provided computing resources.
Paper Writing & Supplementary Materials: Editing.

Tim Althoff
Data Collection: Not involved.
Analysis and Benchmark: Supervised data analysis, visualization, and benchmark results.
Paper Writing & Supplementary Materials: Editing.

Margaret E. Morris
Data Collection: Supervised study material design and high-level planning for 2019-2021. Provided resources for study.
Analysis and Benchmark: Not involved.
Paper Writing & Supplementary Materials: Supervised data-sharing agreement. Editing.

Eve Riskin
Data Collection: Supervised study material design and high-level planning for 2018-2021. Provided resources for study.
Analysis and Benchmark: Not involved.
Paper Writing & Supplementary Materials: Supervised data-sharing agreement. Editing.

Jennifer Mankoff
Data Collection: Supervised study material design and high-level planning for 2018-2021. Provided resources for study.
Analysis and Benchmark: Supervised data analysis and benchmark results.
Paper Writing & Supplementary Materials: Supervised data-sharing agreement and paper writing.

Anind K. Dey
Data Collection: Supervised study material design and high-level planning for 2018-2021. Provided resources for study.
Analysis and Benchmark: Supervised data analysis and benchmark results.
Paper Writing & Supplementary Materials: Supervised data-sharing agreement and paper writing.


Data Hosting, Licensing, and Maintenance Plan
Due to the sensitive nature of the dataset, we release our feature-level data with open credentialed access. Therefore, we plan to leverage the PhysioNet platform for data hosting and licensing, and maintenance.
Host: The PhysioNet platform with Credentialed Access.
License: PhysioNet Credentialed Health Data License 1.5.0
Long-term Preservation: PhysioNet is a well-known platform for freely-available health research data, software, challenges, and tutorials.
It is a reliable platform for long-term hosting and preservation of our datasets.
We will provide in-time maintenance for error correction through the platform.
We will also actively maintain our benchmark platform GLOBEM.
Dataset Meta-Data
We provide the meta-data below. Many texts are taken from the main paper. We adopt the meta-data format from PhysioNet as we will leverage it for the data release.
Title:
GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization
Abstract:
We present the first multi-year mobile sensing datasets.
Our multi-year data collection studies span four years (10 weeks each year, from 2018 to 2021). The four datasets contain data collected from 705 person-years (497 unique participants) with diverse racial, ability, and immigrant backgrounds.
Each year, participants would install a mobile app on their phones and wear a fitness tracker. The app and wearable device passively track multiple sensor streams in the background 24\(\times \) 7, including location, phone usage, calls, Bluetooth, physical activity, and sleep behavior.
In addition, participants completed weekly short surveys and two comprehensive surveys on health behaviors and symptoms, social well-being, emotional states, mental health, and other metrics.
Our dataset analysis indicates that our datasets capture a wide range of daily human routines, and reveal insights between daily behaviors and important well-being metrics (e.g., depression status).
We envision our multi-year datasets can support the ML community in developing generalizable longitudinal behavior modeling algorithms.
Background:
Among various longitudinal sensor streams, smartphones and wearables are arguably one of the most widely available data sources [7].
The advances in mobile technology provide an unprecedented opportunity to capture multiple aspects of daily human behaviors, by collecting continuous sensor streams from these devices [10,11], together with metrics about health and well-being through self-report or clinical diagnosis as modeling targets.
It poses unique challenges compared to traditional time-series classification tasks [6].
First, the data covers a much longer time period, usually across multiple months or years.
Second, the nature of longitudinal collection often results in a high data missing rate.
Third, the prediction target label is sparse, especially for mental well-being metrics.
Longitudinal human behavior modeling is an important multidisciplinary area spanning machine learning, psychology, human-computer interaction, and ubiquitous computing.
Researchers have demonstrated the potential of using longitudinal mobile sensing data for behavior modeling in many applications, e.g., detecting physical health issues [9], monitoring mental health status [11], measuring job performance [8], and tracing education outcomes [12].
Most existing research employed off-the-shelf ML algorithms and evaluated them on their private datasets. However, testing a model with new contexts and users is imperative to ensure its practical deployability.
To the best of our knowledge, there has been no investigation of the cross-dataset generalizability of these longitudinal behavior models, nor an open testbed to evaluate and compare various modeling algorithms.
To address this gap, we present the first multi-year passive mobile sensing datasets to help the ML community explore generalizable longitudinal behavior models.
Methods & Technical Implementation:
Our data collection studies were conducted at a Carnegie-classified R-1 university in the United State with an IRB review and approval.
We recruited undergraduates via emails from 2018 to 2021. After the first year, previous-year participants were invited to join again.
The study was conducted during Spring quarter for 10 weeks each year, so the impact of seasonal effects was controlled.
Based on their compliance, participants received up to $245 in compensation every quarter.
<FIGURE>The four datasets (DS1 to DS4) have 155, 218, 137, and 195 participants (705 person-years overall, and 497 unique people).
Our datasets have a high representation of females (58.9%), immigrants (24.2%), first-generations (38.2%), and disability (9.1%), and have a wide coverage of races, with Asian (53.9%) and White (31.9%) being dominant (e.g., Hispanic/Latino 7.4%, Black/African American 3.3%).
Part 1: Survey Data
We collected survey data at multiple stages of the study. We delivered extensive surveys before the start and at the end of the study (pre/post surveys) and short weekly Ecological Momentary Assessment (EMA) surveys during the study to collect in-the-moment self-report data. All surveys consist of well-established and validated questionnaires to ensure data quality.
Our pre/post surveys include a number of questionnaires to cover various aspects of life, including 1) personality (BFI-10, The Big-Five Inventory-10), 2) physical health (CHIPS, Cohen-Hoberman Inventory of Physical Symptoms), 3) mental well-being (e.g., BDI-II, Beck Depression Inventory-II; ERQ, Emotion Regulation Questionnaire),
and 4) social well-being (e.g., Sense of Social and Academic Fit Scale; EDS, Everyday Discrimination Scale).
Our EMA surveys focus on capturing participants' recent sense of their mental health, including PHQ-4, Patient Health Questionnaire 4; PSS-4, Perceived Stress Scale 4; and PANAS, Positive and Negative Affect Schedule.
We use the depression detection task as a starting point for behavior modeling.
We employ BDI-II (post) and PHQ-4 (EMA) as the ground truth. Both are screening tools for further inquiry of clinical depression diagnosis.
We focus on a binary classification problem to distinguish whether participants' scores indicate at least mild depressive symptoms through the scales (i.e., PHQ-4 \(>\)  2, BDI-II \(>\)  13).
The average number of depression labels is 11.6\(\pm \) 2.6 per person. The percentage of participants with at least mild depression is 39.8\(\pm \) 2.7% for BDI-II and 46.2\(\pm \) 2.5% for PHQ-4.
Due to some design iteration, we did not include PHQ-4 in DS1, but only PANAS.
Although PANAS contains questions related to depressive symptoms (e.g., “distressed”), it does not have a comparable theoretical foundation for depression detection like PHQ-4 or BDI-II.
Therefore, to maximize the compatibility of the datasets, we trained a small ML model on DS2 that has both PANAS and PHQ-4 scores to generate reliable ground truth labels.
Specifically, we used a decision tree (depth=2) to take PNANS scores on two affect questions (“depressed” and “nervous”) as the input and predict PHQ-4 score-based depression binary label. Our model achieved 74.5% and 76.3% for accuracy and F1-score on a 5-fold cross-validation on DS2.
The rule from the decision tree is simple: the user would be labeled as having no depression when the distress score is less than 2, and the nervous score is less than 3 (on a 1-5 Likert Scale).
We then applied this rule to DS1 to generate depression labels.
Part 2: Sensor Data
We developed a mobile app using the AWARE Framework [5] that continuously collects location, phone usage (screen status), Bluetooth scans, and call logs. The app is compatible with both the iOS and Android platforms. Participants installed the app on smartphones and left it running in the background.
In addition, we provided wearable Fitbits to collect their physical activities and sleep behaviors. The mobile app and wearable passively collected sensor data 24\(\times \) 7 during the study.
The average number of days per person per year is 77.5\(\pm \) 8.9 among the four datasets.
Content description:
We release four datasets, named INS-W_1, INS-W_2, INS-W_3, and INS-W_4.
A dataset has three folders. We provided an overview description below. Please refer to our GitHub README page for more details.
[leftmargin=10pt]
SurveyData: a list of files containing participants' survey responses, including pre/post long surveys and weekly short EMA surveys.

FeatureData: behavior feature vectors from all data types, using RAPIDS [2] as the feature extraction tool.

ParticipantInfoData: some additional information about participants, e.g.,, device platform (iOS or Android).

Specifically, the folder structure of a dataset folder is shown as follows:
pic dir tree,
where level=0directory,
,
[root of a dataset folder
[SurveyData
[dep_weekly.csv, file]
[dep_endterm.csv, file]
[pre.csv, file]
[post.csv, file]
[ema.csv, file]
]
[FeatureData
[rapids.csv, file]
[location.csv, file]
[screen.csv, file]
[call.csv, file]
[bluetooth.csv, file]
[steps.csv, file]
[sleep.csv, file]
[wifi.csv, file]
]
[ParticipantsInfoData
[platform.csv, file]
]
]
The SurveyData folder contains five files, all indexed by pid and date:
[leftmargin=10pt]
dep_weekly.csv: The specific file for depression labels (column dep) combining post and EMA surveys.

dep_endterm.csv: The specific file for depression labels (column dep) only in post surveys. Some prior depression detection tasks focus on end-of-term depression prediction.
These two files are created for depression as it is the benchmark task. We envision future work can be extended to other modeling targets as well.

pre.csv: The file contains all questionnaires that participants filled in right before the start of the data collection study (thus pre-study).

post.csv: The file contains all questionnaires that participants filled in right after the end of the data collection study (thus post-study).

ema.csv: The file contains all EMA surveys that participants filled in during the study. Some EMAs were delivered on Wednesdays, while some were delivered on Sundays.

PS: Due to the design iteration, some questionnaires are not available in all studies. Moreover, some questionnaires have different versions across years. We clarify them using column names. For example, INS-W_2 only has CESD_9items_POST, while others have CESD_10items_POST. CESD_9items_POST is also calculated in other datasets to make the modeling target comparable across datasets.
The FeatureData folder contains seven files, all indexed by pid and date.
[leftmargin=10pt]
rapids.csv: The complete feature file that contains all features.

location.csv: The feature file that contains all location features.

screen.csv: The feature file that contains all phone usage features.

call.csv: The feature file that contains all call features.

bluetooth.csv: The feature file that contains all Bluetooth features.

steps.csv: The feature file that contains all physical activity features.

sleep.csv: The feature file that contains all sleep features.

wifi.csv: The feature file that contains all WiFi features. Note that this feature type is not used by any existing algorithms and often has a high data missing rate.

Please note that all features are extracted with multiple time_segments
[leftmargin=10pt]
morning (6 am - 12 pm, calculated daily)

afternoon (12 pm - 6 pm, calculated daily)

evening (6 pm - 12 am, calculated daily)

night (12 am - 6 am, calculated daily)

allday (24 hrs from 12 am to 11:59 pm, calculated daily)

7-day history (calculated daily)

14-day history (calculated daily)

weekdays (calculated once per week on Friday)

weekend (calculated once per week on Sunday)

For all features with numeric values, we also provide two more versions:
[leftmargin=10pt]
normalized: subtracted by each participant's median and divided by the 5-95 quantile range

discretized: low/medium/high split by 33/66 quantile of each participant's feature value

The ParticipantInfoData folder contains files with additional information.
[leftmargin=10pt]
platform.csv: The file contains each participant's major smartphone platform (iOS or Android), indexed by pid

demographics.csv: Due to privacy concerns, demographic data are only available for special requests. Please reach out to us directly with a clear research plan with demographic data.

Usage notes:
We provide a behavior modeling benchmark platform GLOBEM [1].
The platform is designed to support researchers in using, developing, and evaluating different longitudinal behavior modeling methods.
Researchers who use the datasets must agree to the following terms.
Commercial use
The database will not be used for non-academic research purposes. Non-academic purposes include but are not limited to:
[leftmargin=10pt]
proving the efficiency of commercial systems

training or testing of commercial systems

using screenshots of subjects from the dataset in advertisements

selling data from the dataset

creating military applications

developing governmental systems used in public spaces

Distribution
The database will not be re-distributed, published, copied, or further disseminated in any way or form whatsoever, whether for profit or not. This includes further distributing, copying or disseminating to a different facility or organizational unit in the requesting university, organization, or company, with the exception of using small portions of data for the exclusive purpose of clarifying academic publications or presentations.
Privacy 
Although the database has been anonymized, we cannot eliminate all potential risks of privacy information leakage. The PI of any research group access to the dataset, is responsible for continuing to safeguard this database, taking whatever steps are appropriate to protect participants’ privacy and data confidentiality. The specific actions required to safeguard the data may change over time.
Misuse
If at any point, the administrators of the datasets at the University of Washington have concerns or reasonable suspicions that the researcher has violated these usage note, the researcher will be notified. Concerns about misuse may be shared with PhysioNet and other related entities.
Our datasets have led to multiple publications:
[leftmargin=10pt]
Sefidgar YS, Seo W, Kuehn KS, Althoff T, Browning A, Riskin E, Nurius PS, Dey AK, Mankoff J. Passively-sensed behavioral correlates of discrimination events in college students. Proceedings of the ACM on human-computer interaction. 2019 Nov 7;3(CSCW):1-29.

Zhang H, Nurius P, Sefidgar Y, Morris M, Balasubramanian S, Brown J, Dey AK, Kuehn K, Riskin E, Xu X, Mankoff J. How does COVID-19 impact students with disabilities/health concerns?. arXiv preprint arXiv:2005.05438. 2020 May 11.

Xu X, Chikersal P, Dutcher JM, Sefidgar YS, Seo W, Tumminia MJ, Villalba DK, Cohen S, Creswell KG, Creswell JD, Doryab A. Leveraging Collaborative-Filtering for Personalized Behavior Modeling: A Case Study of Depression Detection among College Students. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2021 Mar 29;5(1):1-27.

Nurius PS, Sefidgar YS, Kuehn KS, Jung J, Zhang H, Figueira O, Riskin EA, Dey AK, Mankoff JC. Distress among undergraduates: Marginality, stressors and resilience resources. Journal of American college health. 2021 May 30:1-9.

Morris ME, Kuehn KS, Brown J, Nurius PS, Zhang H, Sefidgar YS, Xu X, Riskin EA, Dey AK, Consolvo S, Mankoff JC. College from home during COVID-19: A mixed-methods study of heterogeneous experiences. PloS one. 2021 Jun 28;16(6):e0251580.

Sefidgar YS, Nurius PS, Baughan A, Elkin LA, Dey AK, Riskin E, Mankoff J, Morris ME. Examining Needs and Opportunities for Supporting Students Who Experience Discrimination. arXiv preprint arXiv:2111.13266. 2021 Nov 25.

Xu X, Mankoff J, Dey AK. Understanding practices and needs of researchers in human state modeling by passive mobile sensing. CCF Transactions on Pervasive Computing and Interaction. 2021 Dec;3(4):344-66.

There are a few known limitations in these datasets:
[leftmargin=10pt]
Limited study population: Only a portion of students who received our emails or social media posts would participate in our study, which could only represent a subset of the general population.

High data missing rate: Missing data is inevitable due to various reasons, such as low battery, data transfer loss, and sensor permission withdrawal. Survey data can also be missing sometimes due to the lack of compliance.

Ethics:
Our datasets aim at aiding research efforts in the area of developing, testing, and evaluating machine learning algorithms to better understand college students’ (and potentially more general population) daily behaviors, health, and well-being from continuous sensor streams and self-reports.
These findings may support public interest in how to improve student experiences and drive policy around adverse events students and others may experience.
Privacy is the major ethical concern of our data collection studies. We strictly follow the IRB rules to anonymize participants' data.
Anyone outside our core data collection group cannot access direct individually-identifiable information. We also eliminated the data for users who stopped their participation at any time during the study.
Since some sensitive sensor data (e.g., location) can disclose identities, we only release feature-level data under credentialing to protect against privacy leakage.
Data collected from human subjects:
The study protocol was approved by relevant Institutional Review Boards (IRBs). Human participants signed a consent form before participating in the study.
Clinical trial data: N/A
Data collected from animals: N/A
Acknowledgments:
Our multi-year data collection study closely followed a sister study at Carnegie Mellon University (CMU). We acknowledge all efforts from CMU Study Team to provide important starting and reference materials [4].
Moreover, our studies were greatly inspired by StudentLife researchers from Dartmouth College [3].
Our studies were supported by the University of Washington (including the Paul G. Allen School of Computer Science and Engineering; Department of Electrical and Computer Engineering; Population Health; Addictions, Drug and Alcohol Institute; and the Center for Research and Education on Accessible Technology and Experiences); the National Science Foundation (EDA-2009977, CHS-2016365, CHS-1941537, IIS1816687 and IIS7974751), the National Institute on Disability, Independent Living and Rehabilitation Research (90DPGE0003-01), Samsung Research America, and Google
Conflicts of interest:
The author(s) have no conflicts of interest to declare.
Version: 1.0.0
References & Weblinks:
[1] Benchmark Platform GLOBEM. github.com/UW-EXP/GLOBEM
[2] Rapids documentation. https://www.rapids.science/1.6/.
[3] Studentlife study https://studentlife.cs.dartmouth.edu/, 2014.
[4] A. Doryab, D. K. Villalba, P. Chikersal, J. M. Dutcher, M. Tumminia, X. Liu, S. Cohen, K. Creswell, J. Mankoff, J. D. Creswell, et al. Identifying behavioral phenotypes of loneliness and social isolation with passive sensing: statistical analysis, data mining and machine learning of smartphone and fitbit data. JMIR mHealth and uHealth, 7(7):e13209, 2019.
[5] D. Ferreira, V. Kostakos, and A. K. Dey. Aware: Mobile context instrumentation framework. Frontiers in ICT, 2:6, 2015.
[6] R. Godahewa, C. Bergmeir, G. I. Webb, R. J. Hyndman, and P. Montero-Manso. Monash time series forecasting archive. In Neural Information Processing Systems Track on Datasets and Benchmarks, 2021.
[7] N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. T. Campbell. A survey of mobile phone sensing. IEEE Communications Magazine, 48(9), 2010.
[8] S. M. Mattingly, J. M. Gregg, P. Audia, A. E. Bayraktaroglu, A. T. Campbell, N. V. Chawla, V. Das Swain, M. De Choudhury, S. K. D’Mello, A. K. Dey, et al. The tesserae project: Large-scale, longitudinal, in-situ, multimodal sensing of information workers. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1–8, 2019.
[9] J.-K. Min, A. Doryab, J. Wiese, S. Amini, J. Zimmerman, and J. I. Hong. Toss “n” turn: Smartphone as sleep and sleep quality detector. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, page 477–486, New York, NY, USA, 2014. Association for Computing Machinery.
[10] M. E. Morris, Q. Kathawala, T. K. Leen, E. E. Gorenstein, F. Guilak, W. DeLeeuw, and M. Labhard. Mobile therapy: case study evaluations of a cell phone application for emotional self-awareness. Journal of medical Internet research, 12(2):e10, 2010.
[11] R. Wang, F. Chen, Z. Chen, T. Li, G. Harari, S. Tignor, X. Zhou, D. Ben-Zeev, and A. T. Campbell. Studentlife: Assessing mental health, academic performance and behavioral trends of college students using smartphones. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pages 3–14. ACM, 2014.
[12] R. Wang, G. Harari, P. Hao, X. Zhou, and A. T. Campbell. Smartgpa: how smartphones can assess and predict academic performance of college students. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing, pages 295–306, 2015.

