Please use this identifier to cite or link to this item: https://repository.isls.org/handle/1/11885
Title: Processes Matter: How ML/GAI Approaches Could Support Open Qualitative Coding of Online Discourse Datasets
Authors: Chen, John
Lotsos, Alexandros
Wang, Grace
Zhao, Lexie
Sherin, Bruce
Wilensky, Uri
Horn, Michael
Keywords: CSCL
Issue Date: 2025
Publisher: International Society of the Learning Sciences
Citation: Chen, J., Lotsos, A., Wang, G., Zhao, L., Sherin, B., Wilensky, U., & Horn, M. (2025). Processes Matter: How Ml/gai Approaches Could Support Open Qualitative Coding of Online Discourse Datasets. In Oshima, J., Chen, B., Vogel, F., & Järvelä, J. (Eds.), Proceedings of the 18th International Conference on Computer-Supported Collaborative Learning - CSCL 2025 (pp. 415-419). International Society of the Learning Sciences.
Series/Report no.: Proceedings of the 18th International Conference on Computer-Supported Collaborative Learning - CSCL 2025
Abstract: Open coding, a key inductive step in qualitative research, discovers and constructs concepts from human datasets. However, capturing extensive and nuanced aspects or “coding moments” can be challenging, especially with large discourse datasets. While some studies explore machine learning (ML)/Generative AI (GAI)'s potential for open coding, few evaluation studies exist. We compare open coding results by five recently published ML/GAI approaches and four human coders, using a dataset of online chat messages around a mobile learning software. Our systematic analysis reveals ML/GAI approaches' strengths and weaknesses, uncovering the complementary potential between humans and AI. Line-by-line AI approaches effectively identify content-based codes, while humans excel in interpreting conversational dynamics. We discussed how embedded analytical processes could shape the results of ML/GAI approaches. Instead of replacing humans in open coding, researchers should integrate AI with and according to their analytical processes, e.g., as parallel co-coders.
Description: Short Paper
URI: https://doi.org/10.22318/cscl2025.114185
https://repository.isls.org//handle/1/11885
Appears in Collections:ISLS Annual Meeting 2025

Files in This Item:
File SizeFormat 
CSCL2025_415-419.pdf313.92 kBAdobe PDFView/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.