Wearable sensors are placed on StressID participants to gather physiological responses, including an electrocardiogram (ECG), an electrodermal activity (EDA) sensor and a breathing sensor. This data is combined with synchronized audio and video recordings of the participants’ faces.
In the long-term, the results of StressID data analysis could be used in healthcare applications. Advanced stress detection is of great interest in the medical sector. “For example, autistic patients can be sensitive to stress during medical examinations, and it’s important to be able to detect this early on to anticipate and better manage the patient’s condition,” says the PhD student.
Hava Chaptoukaev received the NeurIPS 2023 Scholar Award at the 37th Conference on Neural Information Processing Systems, held in December 2023 in the United States. StressID is a database for identifying stress using unimodal and multimodal data. Its strength lies in its ability to collect all modalities (video, audio, biological signals) synchronously and in high quality. StressID also has the advantage of including recordings from a large number of participants.
Experimental data and protocol
The recordings from 65 subjects were analyzed for StressID, resulting in 39 hours of annotated data. The volunteer participants, 18 women and 47 men aged between 21 and 55, are students and professionals in the fields of science, technology, engineering and mathematics. A certain level of English is required for participants to complete the 11 tasks used to assess their stress and relaxation levels.
The tasks include public speaking, reading, color tests and counting. After analyzing the data, the exercise considered the most stressful was counting. The instructions read: “Start counting backwards from 1011 by subtracting 7, for 1 minute (as fast as you can)”.
According to Hava Chaptoukaev, “the tasks used are clinically established and the protocols already exist. We selected them so as to create as varied a list of exercises as possible. They are also approved by a psychologist, who provided guidance with the selection.”
Various triggers are used to identify stress. They are activated through a guided breathing task (to obtain a baseline of subjects’ state of stress), two emotional video clips, seven different interactive exercises, and a final relaxation task.
The StressID protocol fills the gaps in the many existing databases on emotion and its classification. Stress databases are rare, and those that do exist have the disadvantage of having a very limited number of participants. The data is available on request, as Maria Zualaga says: “to access all the data, there’s simply a verification process. You have to sign an agreement beforehand on its proper use, as it is intended for research purposes.”
Contributions provided by StressID
This new multimodal dataset focusing on stressful tasks is one of StressID’s main contributions. The project also features synchronized and annotated modalities, as well as participants’ self-assessments of their levels of relaxation, stress, agitation and valence, which in psychology expresses a positive or negative emotional state.
The experimental protocol provided is easy to reproduce, to record behavioral and physiological responses to various triggers via portable and comprehensive sensors. Instructions for using the dataset are also available. Available in open-source format are several basic models for recognizing stress using video, audio and physiological signals, as well as multimodal models combining these three inputs.
“Participants tell us whether they felt negative or positive sensations, but also how involved they were in the task. Few people felt angry or sad when carrying out our tests, but many found them amusing,” says Hava Chaptoukaev.
“StressID is the result of collaboration between several universities. It involves experts in data processing, video and audio,” says Maria Zuluaga. She adds that “the project will be extended to look more closely at the physiological data”, with the long-term ambition of predicting stress using non-invasive devices and sensors.
