Please use this identifier to cite or link to this item: https://repository.isls.org/handle/1/11224
Title: Can We Trust Large Language Models for Video Analysis: An Exploration of Hallucination in Multimodal LLMs
Authors: Zhou, Yiqiu
Kang, Jina
Nguyen, Ha
Keywords: Learning Sciences
Issue Date: 2025
Publisher: International Society of the Learning Sciences
Citation: Zhou, Y., Kang, J., & Nguyen, H. (2025). Can we Trust Large Language Models for Video Analysis: An Exploration of Hallucination in Multimodal LLMs. In Rajala, A., Cortez, A., Hofmann, R., Jornet, A., Lotz-Sisitka, H., & Markauskaite, L. (Eds.), Proceedings of the 19th International Conference of the Learning Sciences - ICLS 2025 (pp. 110-118). International Society of the Learning Sciences.
Series/Report no.: Proceedings of the 19th International Conference of the Learning Sciences - ICLS 2025
Abstract: Multimodal Large Language Models (MLLMs) show promise for facilitating qualitative video analysis, potentially reducing the manual effort required for video selection and coding. However, MLLMs are subject to hallucination, where models generate incorrect or unfounded descriptions and interpretations. Current frameworks for evaluating MLLM hallucinations primarily focus on object-centric tasks, with limited focus on behaviors and social interactions common in education data. Applying MLLMs to analyzing 36 video clips of collaborative learning activities, we identify three distinct types of hallucinations: description hallucination (factual inconsistency), interpretation hallucination (misinterpretation of behaviors), and instruction hallucination (deviation from analytical focus). Despite MLLMs’ ability to process educational videos, they exhibit gender misidentification and inaccurate interpretation of student roles and behaviors. This exploratory work develops a domain-specific framework to understand and evaluate MLLM hallucinations, laying the groundwork for more reliable implementation in educational research and calling for further investigations
Description: Long Paper
URI: https://doi.org/10.22318/icls2025.704957
https://repository.isls.org//handle/1/11224
Appears in Collections:ISLS Annual Meeting 2025

Files in This Item:
File SizeFormat 
ICLS2025_110-118.pdf458.6 kBAdobe PDFView/Open


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.