Please use this identifier to cite or link to this item:
https://repository.isls.org/handle/1/11774| Title: | Differences in Novice and Expert Evaluations of Generative AI-Produced Definitions |
| Authors: | Williams-Pierce, Caro |
| Keywords: | Learning Sciences |
| Issue Date: | 2025 |
| Publisher: | International Society of the Learning Sciences |
| Citation: | Williams-Pierce, C. (2025). Differences in Novice and Expert Evaluations of Generative Ai-Produced Definitions. In Rajala, A., Cortez, A., Hofmann, R., Jornet, A., Lotz-Sisitka, H., & Markauskaite, L. (Eds.), Proceedings of the 19th International Conference of the Learning Sciences - ICLS 2025 (pp. 529-537). International Society of the Learning Sciences. |
| Series/Report no.: | Proceedings of the 19th International Conference of the Learning Sciences - ICLS 2025 |
| Abstract: | Widely available generative AI products such as ChatGPT are transforming the ways in which students and professionals work and learn, but little is known about how novices and experts perceive, evaluate, and judge generative AI-produced materials. Established research from cognitive science suggests that novices and experts will attend differently to surface and deep structures of content-specific materials. This preliminary study focuses on examining the accuracy and confidence judgements and annotations from experts and novices in the field of Information Science, comparing their evaluations across three different conditions: a ChatGPT produced definition of ‘data’; a human crowd-sourced definition of ‘data’; and a ChatGPT produced definition of a nonexistent word, ‘techno-data.’ The paper concludes with next steps in this research study, as well as potential implications for learning, education, and expertise. |
| Description: | Long Paper |
| URI: | https://doi.org/10.22318/icls2025.620245 https://repository.isls.org//handle/1/11774 |
| Appears in Collections: | ISLS Annual Meeting 2025 |
Files in This Item:
| File | Size | Format | |
|---|---|---|---|
| ICLS2025_529-537.pdf | 379.13 kB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.