Please use this identifier to cite or link to this item: https://repository.isls.org/handle/1/12974
Full metadata record
DC FieldValueLanguage
dc.contributor.authorKong, Lingchen
dc.date.accessioned2026-09-15T15:45:43Z-
dc.date.available2026-09-15T15:33:08Z
dc.date.available2026-09-15T15:45:43Z-
dc.date.issued2026
dc.identifier.citationKong, L. (2026). Item calibration in reading assessment with LLMs: Human alignment and the masking effect. In Litts, B. K., DeLiema, D., Lee, C., Krist, S. Mawasi, A., Martin, L., & Kumpulainen, K. (Eds.), Proceedings of the 20th International Conference of the Learning Sciences - ICLS 2026 (pp. 2543-2545). International Society of the Learning Sciences.en
dc.identifier.isbn979-8-9906980-7-9
dc.identifier.issn1819-0138
dc.identifier.urihttps://doi.org/10.22318/icls2026.617270
dc.identifier.urihttps://repository.isls.org/handle/1/12974-
dc.descriptionPOSTERen
dc.description.abstractThis study evaluates large language models (LLMs) for calibrating item difficulty in grade 8 reading comprehension using 30 NAEP items. LLM-calibrated difficulty estimates were compared with human benchmarks, and masking was used to manipulate input information. Results show no significant differences across human-derived difficulty levels, indicating misalignment with human performance. Masking systematically reduced accuracy, particularly for harder items as more input text was obscured. These findings raise concerns about the validity of LLM-based item calibration.en
dc.format.extent2543-2545en
dc.publisherInternational Society of the Learning Sciences
dc.relation.ispartofurn:ISBN:979-8-9906980-7-9
dc.relation.ispartofProceedings of the 20th International Conference of the Learning Sciences - ICLS 2026, pp. 2543-2545
dc.relation.ispartofseriesProceedings of the 20th International Conference of the Learning Sciences - ICLS 2026en
dc.subjectICLSen
dc.titleItem calibration in reading assessment with LLMs: Human alignment and the masking effecten
dc.title.alternativeItem Calibration in Reading Assessment with LLMs: Human Alignment and the Masking Effecten
dc.typeconferencePaper
dc.identifier.doi10.22318/icls2026.617270
dc.format.pagerange2543-2545en
dc.title.parentProceedings of the 20th International Conference of the Learning Sciences - ICLS 2026en
dc.bibliographicCitation.conferencenameProceedings of the 20th International Conference of the Learning Sciences - ICLS 2026en
dc.format.firstpage2543en
dc.format.lastpage2545en
Appears in Collections:ISLS Annual Meeting 2026

Files in This Item:
File SizeFormat 
ICLS2026_2543-2545.pdf250.65 kBAdobe PDFView/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.