CS Colloquium: Olga Kellert on From Classroom Speech to Evidence-Linked AI Analytics of Language-Learning Interaction

About

Abstract: What happens after a learner makes an error—and how can AI help researchers and educators examine the exchange that follows? This talk presents two recently submitted studies with my co-authors on transforming authentic multilingual classroom recordings into reliable, inspectable analyses of language-learning interaction.

The main focus is SLA-LENS, a framework that combines insights from second-language acquisition research with contextual large language model annotation and reproducible analytical reporting. Applied to 40 online Spanish classroom transcripts containing more than 21,000 sentence-level instances, it identifies learner errors, instructional feedback, and learners’ subsequent responses. A deterministic reporting layer summarizes participation, feedback, and interaction patterns, linking results to supporting transcript excerpts and timestamp. Comparisons of proprietary and open-weight models show that task-specific QLoRA adaptation improves Qwen3-8B’s exact annotation accuracy from 62.4% to 69.8% on an expert-reviewed transcript. Overall accuracy does not tell the whole story: models identify teachers’ feedback more reliably than learners’ responses, and some disagreements arise because an exchange has more than one plausible interpretation. We therefore examine errors that could affect the classroom analysis and link results to transcript examples, allowing researchers and educators to check the evidence and use it to inform teaching and curriculum design.

A complementary speech-recognition study examines the reliability of the transcripts underlying such analyses. In-domain adaptation substantially improves MOSS transcription, bringing it close to commercial Muse performance, while learner speech and multilingual turns remain challenging. Matching training data to classroom interaction proves more useful than adding larger, less closely matched datasets.

Together, these studies connect speech recognition, contextual language understanding, and transparent educational analytics. Both papers report recent work; links to preprints will be shared as soon as they are available.

Bio: Olga Kellert is an Associate Professor at Arizona State University and Director of its Digital Humanities Cluster. A linguist with a background in NLP, she has spent the past years developing interdisciplinary research connecting linguistics, NLP, and AI. Her work examines the representation of linguistic minorities in language technologies, including low-resource languages, dialects, and code-switching. Her research also extends to AI for health literacy, educational analytics, and tracing and evaluating AI outputs in data analysis. She collaborates with colleagues across the humanities, data science, computing, and public health.