Please use this identifier to cite or link to this item:
https://repository.isls.org/handle/1/12974Full metadata record
| DC Field | Value | Language |
|---|---|---|
| dc.contributor.author | Kong, Lingchen | |
| dc.date.accessioned | 2026-09-15T15:45:43Z | - |
| dc.date.available | 2026-09-15T15:33:08Z | |
| dc.date.available | 2026-09-15T15:45:43Z | - |
| dc.date.issued | 2026 | |
| dc.identifier.citation | Kong, L. (2026). Item calibration in reading assessment with LLMs: Human alignment and the masking effect. In Litts, B. K., DeLiema, D., Lee, C., Krist, S. Mawasi, A., Martin, L., & Kumpulainen, K. (Eds.), Proceedings of the 20th International Conference of the Learning Sciences - ICLS 2026 (pp. 2543-2545). International Society of the Learning Sciences. | en |
| dc.identifier.isbn | 979-8-9906980-7-9 | |
| dc.identifier.issn | 1819-0138 | |
| dc.identifier.uri | https://doi.org/10.22318/icls2026.617270 | |
| dc.identifier.uri | https://repository.isls.org/handle/1/12974 | - |
| dc.description | POSTER | en |
| dc.description.abstract | This study evaluates large language models (LLMs) for calibrating item difficulty in grade 8 reading comprehension using 30 NAEP items. LLM-calibrated difficulty estimates were compared with human benchmarks, and masking was used to manipulate input information. Results show no significant differences across human-derived difficulty levels, indicating misalignment with human performance. Masking systematically reduced accuracy, particularly for harder items as more input text was obscured. These findings raise concerns about the validity of LLM-based item calibration. | en |
| dc.format.extent | 2543-2545 | en |
| dc.publisher | International Society of the Learning Sciences | |
| dc.relation.ispartof | urn:ISBN:979-8-9906980-7-9 | |
| dc.relation.ispartof | Proceedings of the 20th International Conference of the Learning Sciences - ICLS 2026, pp. 2543-2545 | |
| dc.relation.ispartofseries | Proceedings of the 20th International Conference of the Learning Sciences - ICLS 2026 | en |
| dc.subject | ICLS | en |
| dc.title | Item calibration in reading assessment with LLMs: Human alignment and the masking effect | en |
| dc.title.alternative | Item Calibration in Reading Assessment with LLMs: Human Alignment and the Masking Effect | en |
| dc.type | conferencePaper | |
| dc.identifier.doi | 10.22318/icls2026.617270 | |
| dc.format.pagerange | 2543-2545 | en |
| dc.title.parent | Proceedings of the 20th International Conference of the Learning Sciences - ICLS 2026 | en |
| dc.bibliographicCitation.conferencename | Proceedings of the 20th International Conference of the Learning Sciences - ICLS 2026 | en |
| dc.format.firstpage | 2543 | en |
| dc.format.lastpage | 2545 | en |
| Appears in Collections: | ISLS Annual Meeting 2026 | |
Files in This Item:
| File | Size | Format | |
|---|---|---|---|
| ICLS2026_2543-2545.pdf | 250.65 kB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.