Informing the Design of Teacher Awareness Tools through Causal Alignment Analysis

Abstract

Designers of teacher awareness tools (e.g., dashboards) must not only anticipate the effects analytics will have on awareness, but also how this enhanced awareness might affect teacher decision-making, and in turn, student learning. Yet teacher awareness tools are not commonly optimized to guide teacher behavior in ways that are productive for learning. In this paper, we introduce Causal Alignment Analysis (CAA), a framework for data-informed,iterative design of teacher awareness tools, which links the design of awareness tools to educational goals. We illustrate the usefulness of CAA with a case study, demonstrating the successful design alignment of an awareness tool with a causal path from teacher tool use to student learning. Over a sequence of four pilot studies conducted in both simulated and live classrooms, we demonstrate the iterative refinement of Lumilo, a real-time awareness tool, to draw teachers’ attention towards students who may benefit most from a teacher’s help.