Please use this identifier to cite or link to this item: https://repository.isls.org/handle/1/11774
Title: Differences in Novice and Expert Evaluations of Generative AI-Produced Definitions
Authors: Williams-Pierce, Caro
Keywords: Learning Sciences
Issue Date: 2025
Publisher: International Society of the Learning Sciences
Citation: Williams-Pierce, C. (2025). Differences in Novice and Expert Evaluations of Generative Ai-Produced Definitions. In Rajala, A., Cortez, A., Hofmann, R., Jornet, A., Lotz-Sisitka, H., & Markauskaite, L. (Eds.), Proceedings of the 19th International Conference of the Learning Sciences - ICLS 2025 (pp. 529-537). International Society of the Learning Sciences.
Series/Report no.: Proceedings of the 19th International Conference of the Learning Sciences - ICLS 2025
Abstract: Widely available generative AI products such as ChatGPT are transforming the ways in which students and professionals work and learn, but little is known about how novices and experts perceive, evaluate, and judge generative AI-produced materials. Established research from cognitive science suggests that novices and experts will attend differently to surface and deep structures of content-specific materials. This preliminary study focuses on examining the accuracy and confidence judgements and annotations from experts and novices in the field of Information Science, comparing their evaluations across three different conditions: a ChatGPT produced definition of ‘data’; a human crowd-sourced definition of ‘data’; and a ChatGPT produced definition of a nonexistent word, ‘techno-data.’ The paper concludes with next steps in this research study, as well as potential implications for learning, education, and expertise.
Description: Long Paper
URI: https://doi.org/10.22318/icls2025.620245
https://repository.isls.org//handle/1/11774
Appears in Collections:ISLS Annual Meeting 2025

Files in This Item:
File SizeFormat 
ICLS2025_529-537.pdf379.13 kBAdobe PDFView/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.