Using Sentence Embeddings to Automatically Extract Cohesion and Alignment Metrics in Problem-Solving Tasks

Abstract

We introduce an automated approach that builds on sentence embeddings, a novel natural language processing technique that extracts meaning from sentences, to create two quantitative measures that serve as proxies of collaborative learning. Cohesion is extracted as adjacent utterance similarity and represents the amount of overlap between contiguous conversational turns. Alignment is extracted as a similarity between a focus utterance and a reference text and represents the degree to which a conversation utterance aligns with the task. These two dimensions divide the quality of conversation in four quadrants.