Towards Automated Tracking of Affect: Testing the Use of Continuous Self-Reports and Multimodal Metrics

Abstract

The affective states of students while studying outside of class hold rich information about how students are doing in a course and how instruction could be improved. In this paper, we test the co-occurrence of webcam-based nonverbal metrics with self-reported student affective states during an instructional video. Preliminary results suggest that low-level, selfreported affect can be a promising ground truth data for research, and that nonverbal metrics created from webcam streams can be associated with this self-reported affect.