Emotional Modelling to Enhance Learning with Games (AMELIA)

Funding period: 09/2023 - 12/2024

This research project (ID#: 101105874) deals with measuring and modeling emotions as nonlinear dynamical systems that manifest before, during, and after learners interact with game-based learning environments called Antidote COVID-19 and MediaWatch.

To collect emotions, a mixed-multimodal methods approach was utilized, capturing a range of data channels, including video recordings of facial expressions of emotions and posture, audio recordings of emote- and think-alouds, and computer-screen recordings of learner’s interactions with game-based mechanics, neuro-imaging data (NIRS), eye tracking, and physiological signals (EDA, PPG, respiration, ECG) during game-based learning (GBL).

Two studies with a within-subjects pre/post design took place in this project. One at Tampere University (TAU) in Finland and the other at the University of Graz in Austria.

• TAU: A sample of 81 participants’ multimodal data was collected during game-based learning with Antidote Covid-19 on an iPad.
• GRAZ: Data collection is still on going; A sample of 60 participants will be collected using a 2x3 within-subjects, pre/post design. Participants learn with MediaWatch with three different types of feedback and we will assess its relation to affective responses, cognition, and learning outcomes.

These data will be leveraged to study how multiple affective dimensions manifest during GBL, including expressive, affective, motivational, neurophysiological, and cognitive, to assess their relation to cognitive processes and learning outcomes. The findings of this work will provide insight into the role of affective processes on cognition and learning outcomes with game-based learning environments

• Data management plan aligned with FAIR principles and the Academy of Finland Data Management guidelines can be found here
• Project visibility here
• Data and Scripts can be found at this repository.

Highlighted Publications:

• Cloude, Huber, Wei, Esmanhoto, Dindar, Ninaus, & Kiili. (In press). A balance between stability and flexiblity: Adaptive patterns of self-regulated learning processes during game-based learning. British Journal of Educational Technology. [PDF IN PRODUCTION]
• Cloude, E.B., Dindar, M., Ninaus, M., Kiili, K. (2024). Synchrony Between Facial Expressions and Heart Rate Variability During Game-Based Learning: Insights from Cross-Wavelet Transformation. In: Ferreira Mello, R., Rummel, N., Jivet, I., Pishtari, G., Ruipérez Valiente, J.A. (eds) Technology Enhanced Learning for Inclusive and Equitable Quality Education. EC-TEL 2024. Lecture Notes in Computer Science, vol 15159. Springer. PDF LINK HERE