Dissertation Defense: James LeDoux

About

Title: Reading Beyond Proficiency: Initial Construct Validity Evidence for the Higher Order Thinking-Reading Scale

Program: Curriculum and Instruction EdD

Committee Chair: Jeremy W. Ford

Committee: Jeremy W. Ford, Katherine Wright, Jim Fredricksen

Abstract: This study was prompted by a real classroom dilemma: Why do secondary students who have all achieved proficiency on large-scale testing produce significantly different constructed responses when reading the same fictional text? A quantitative study was launched to examine the construct validity of the Higher Order Thinking-Reading Scale (HOT-RS), a theoretically grounded rubric designed to measure higher-order thinking in secondary readers’ constructed responses to reading a fictional text. Selected-response reading assessments capture accuracy and recall but not the depth of a reader’s thinking, and no validated instrument existed to measure higher-order thinking directly from student writing. The HOT-RS was grounded in Afflerbach et al.’s (2015) framework for higher-order thinking in reading and developed following DeVellis’s (2017) scale development procedures. Participants were 49 secondary readers in grades 9 through 12, enrolled in an elective reading course at an urban public magnet high school in the northern Mountain West. Over four weeks of ordinary instruction, students produced written journal responses with minimal prompting to self-selected fiction. Three trained raters scored these responses using the HOT-RS, achieving good interrater reliability (average-measures intraclass correlation = .895 at the student level, .821 at the journal level). HOT-RS scores were then correlated with scores on the Reading Maturity Survey (RMS), a validated 60-item self-report measure of reading maturity, to gather initial evidence of construct validity. HOT-RS scores correlated positively and significantly with total RMS scores, r(47) = .35, p = .013, a medium effect. Of the six RMS subcomponents, HOT-RS scores correlated significantly with Reading Ability, r(47) = .51; Transformational Reading, r(47) = .37; and Reading Attitudes and Interests, r(47) = .33. Correlations with Reading Purposes, Sophistication of Reading Materials, and Higher-Order Literacy were positive but not significant. This pattern offered both convergent and discriminant validity evidence that HOT-RS scores related most strongly to what a reader demonstrates and least strongly to what a reader self-reports. Convergent and discriminant evidence together provide initial construct validity evidence for the HOT-RS and suggest that higher-order thinking in reading, as this instrument measures it, is meaningfully related to, but distinct from, reading maturity. The HOT-RS offers teachers a tool to measure higher-order thinking in reading as a means to understanding the thinking skills that readers possess, as well as those they lack. For researchers studying reading development, it offers an insight into the role that higher-order thinking plays in moving readers from proficiency towards that of expert reading.