An Early Warning System for Depression in Students

Use Cases
1

Can depression be predicted before it starts? Weather forecasters do not predict a storm by waiting for rain. They study the conditions that produce it, such as pressure, temperature, and wind, and how those conditions change together. The WARN-D study applies a similar idea to depression in students.

Researchers at Leiden University designed a two-year study to investigate whether patterns in students’ daily experiences and wearable sensor data could help identify early warning signs of depression. After an online baseline survey, participants reported on their daily lives on their phones four times a day for 85 days while wearing a smartwatch. Follow-up assessments were scheduled every three months for 21 months to track how their mental health developed. The goal is to develop a personalized early warning system that could help identify who may be at risk of depression and when. This remains a research goal, not an established prediction capability.

Building such a system requires a large, well-documented dataset. Data from WARN-D have also been used to investigate practical questions about running ecological momentary assessment (EMA) studies: how individual items perform, how reliable scales are, how often to collect measurements, and how to identify potentially careless responses.

A large, well-documented EMA study can answer questions about how to run EMA itself.

Study at a Glance

Study feature Detail
Population Students enrolled at higher-education institutions in the Netherlands, aged 18 and older
Design Four-cohort study over two years: baseline assessment, 85 days of daily monitoring, and follow-up assessments every three months for 21 months
Sample Approximately 1,750 students across four cohorts in later publications; the original recruitment target was approximately 2,000
Daily monitoring 85 days per participant
EMA Four surveys per day at semi-random times, plus a weekly survey on Sundays
Measurement points Up to 352 per participant
Languages Dutch or English, chosen by each participant
Wearable Garmin vívosmart 4 smartwatch during the monitoring period, with data collected through Fitrockr
Feedback Participants could access personalized reports of their own EMA data after the monitoring period
Compensation Up to €45 for the EMA monitoring phase, depending on completed surveys
Tools Qualtrics for the baseline survey, the Avicenna app (then Ethica) for the daily surveys, Garmin and Fitrockr for wearable data, and the team’s own FRED software for feedback reports
Funding European Research Council (Horizon 2020, Grant No. 949059)

How the Study Worked

Students signed up online, chose Dutch or English, and completed a screening questionnaire. Those who were eligible completed a 75-minute baseline survey in Qualtrics covering a broad range of risk and protective factors for depression. They then moved on to the daily monitoring stage, with follow-up assessments scheduled every three months for the rest of the two-year study.

Four Surveys a Day

During the monitoring stage, the Avicenna app (then Ethica) prompted participants four times a day at semi-random times. Every daily survey included a core set of questions about positive and negative affect, such as how sad, nervous, overwhelmed, cheerful, motivated, or relaxed participants felt at that moment. Participants rated their feelings on a scale from 1 to 7. The affect questions appeared in a randomized order at each prompt.

Some questions were asked only in the morning or evening. For example, the evening survey was scheduled to arrive between 9:04 and 9:34 p.m. and expired after 20 minutes. A longer survey every Sunday asked about the previous week, including depression and anxiety symptoms.

A Smartwatch Alongside the Surveys

Throughout the monitoring stage, participants also wore a Garmin vívosmart 4 smartwatch. The study collected wearable data, including measures related to sleep, activity, heart rate, and stress, so researchers could later compare selected wearable measures with the corresponding self-reports. The watch data were collected through Fitrockr.

Giving Participants Their Data Back

After the monitoring stage, participants could access personalized reports of their own EMA data. The research team developed an open-source tool for generating these reports, FRED (Feedback Reports on EMA Data).

FRED helps participants explore how their moods and experiences changed over the three months and, when enough data are available, how their feelings related to one another. Of the 95 participants who evaluated an early version of the reports, most rated them favorably. On a seven-point scale, the average rating for how well the reports described them was 5.2.

One Dataset, Many Lessons About EMA

Many EMA studies focus on what participants report about their experiences. WARN-D data have also been used to examine how well EMA measurements work. The following papers address questions researchers face when designing or analyzing their own EMA studies. Several also share code and tutorials to help others apply similar methods to their data.

How do individual EMA items perform?

A tutorial examined a wide range of WARN-D items from 599 participants using descriptive statistics and visualizations. It explored how responses were distributed, how they changed over time, how they varied with context, and how they related to traditional questionnaire measures of psychopathology.

How reliable are EMA scales?

A preregistered study estimated the internal consistency of WARN-D scales for positive and negative affect, measured four times a day, and for depression and anxiety, measured weekly. It compared six statistical approaches, including methods that estimate reliability separately for each person.

Do smartwatch and self-report measures capture the same thing?

A study of 781 students compared wearable sensor data with self-reports. Sleep measured by the watch was robustly associated with self-reported sleep. The association was weaker for tiredness, while the watch’s stress measure showed little overlap with self-reported stress for most individuals. These findings highlight the distinction between associations observed across a group and agreement between measures for individual participants.

How often should depressive symptoms be measured?

A study applied the Nyquist–Shannon theorem from signal processing to depressive symptom data from WARN-D and a second dataset from a clinical sample. The authors found that useful information could be captured at multiple sampling frequencies, including every other week, weekly, and daily. They suggest that weekly assessments may be sufficient for regular monitoring, while more frequent measurement may be useful when rapid symptom changes are expected, such as during treatment. These findings can inform sampling decisions, but they do not establish a universal frequency for all EMA studies.

How can careless responses be detected?

Careless responses are answers given without paying attention to the questions. A study applied 13 methods for identifying potentially careless responses to 206,318 observations from 1,194 participants. The methods produced substantially different results, both in how many responses they flagged and in which responses they identified. The authors caution against relying on a single indicator and provide tutorials to help researchers evaluate careless responding in their own data. They also note that these detection methods require further validation.

Research Team

Eiko I. Fried
Principal investigator of WARN-D
Department of Clinical Psychology, Leiden University

Ricarda K. K. Proppert
WARN-D team member; co-author of the study protocol
Department of Clinical Psychology, Leiden University

Carlotta L. Rieble
WARN-D team member; co-author of the study protocol
Department of Clinical Psychology, Leiden University

Björn S. Siepe
First author of the item performance tutorial and the smartwatch and self-report comparison
Psychological Methods Lab, University of Marburg

Leonie V. D. E. Vogelsmeier
First author of the careless responding study
Department of Methodology and Statistics, Tilburg University

Publications

Planning a Similar Study?

A protocol like WARN-D’s daily monitoring stage uses Surveys scheduled with a Time Triggering Logic that places prompts at random times within set windows, and an Expiry Time that closes each prompt after a few minutes. Flow Control can randomize question order, Localization lets participants answer in their own language, and responses are available as Raw Data for analysis.

Explore Avicenna’s features and integrations or talk to our team to see how you can build your study protocol.