Please use this identifier to cite or link to this item: https://repository.isls.org/handle/1/10689
Title: Culturally and linguistically “Blind” or Biased? Challenges for AI Assessment of Models with Multiple Language Students
Authors: Li, Tingting
Miller, Emily Adah
He, Peng
Keywords: Learning Sciences
Issue Date: 2024
Publisher: International Society of the Learning Sciences
Citation: Li, T., Miller, E. A., & He, P. (2024). Culturally and linguistically “Blind” or Biased? Challenges for AI Assessment of Models with Multiple Language Students. In Lindgren, R., Asino, T. I., Kyza, E. A., Looi, C. K., Keifert, D. T., & Suárez, E. (Eds.), Proceedings of the 18th International Conference of the Learning Sciences - ICLS 2024 (pp. 1323-1326). International Society of the Learning Sciences.
Abstract: Investigating AI's role in educational assessments, this study compares AI- provided and teacher scores of hand-drawn scientific models by Multilingual Language Learners (MLLs) in elementary classrooms. Using Convolutional Neural Networks (CNN) for scoring, we aligned AI assessments with those of experienced teachers. The results show moderate agreement (Kappa = 0.326), with AI favoring mid-range scores, while teachers provided a broader score spectrum. This suggests AI's consistency may miss the interpretive nuances teachers offer. The study emphasizes careful AI integration to support the diverse assessments of MLLs, though it notes the limitations of a small sample size and the opaque AI scoring rationale. Our findings advocate for combining AI's analytical strengths with teacher expertise to enhance equitable, effective educational assessments.
Description: Short Paper
URI: https://doi.org/10.22318/icls2024.806499
https://repository.isls.org//handle/1/10689
Appears in Collections:ISLS Annual Meeting 2024

Files in This Item:
File SizeFormat 
ICLS2024_1323-1326.pdf492.53 kBAdobe PDFView/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.