Methods and Procedures: PIRLS 2021 Technical Report

Methods and Procedures:
PIRLS 2021 Technical Report

Matthias von Davier, Ina V.S. Mullis, Bethany Fishbein, and Pierre Foy, Editors


PIRLS 2021 Methods And Procedures Gear Icon

This technical report documents the methods and procedures used to develop, implement, and report the results of PIRLS 2021. PIRLS 2021 incorporated two major advances in international reading assessment at the fourth grade, including transitioning from paper to digital assessment and implementing a group adaptive assessment design. There were substantial challenges in the implementation, analysis, and reporting of PIRLS 2021 due to the COVID-19 pandemic. PIRLS 2021 data collection occurred over a two-year period, with schools worldwide facing many disruptions to normal instruction. Despite many obstacles, most countries met standards for high quality data. A multi-step, population-based linking approach to the achievement scaling enabled reporting comparable trends for paper and digital data on the PIRLS trend scale.

PIRLS 2021 was a collaborative effort involving hundreds of individuals around the world, including the TIMSS & PIRLS International Study Center, IEA Amsterdam, IEA Hamburg, Statistics Canada, and the National Research Coordinators and their teams in the participating countries.


   

Contents

Assessment Development
» Chapter 1: Developing the PIRLS 2021 Achievement Instruments » Chapter 2: Updating the PIRLS 2021 Instruments for Describing the Contexts for Student Learning » Chapter 3: Sample Design in PIRLS 2021
Data Collection Procedures
» Chapter 4: Survey Operations Procedures for PIRLS 2021 » Chapter 5: Systems and Instrument Verification for PIRLS 2021 » Chapter 6: Quality Assurance Program for PIRLS 2021 » Chapter 7: Creating the PIRLS 2021 International Database
Analysis and Reporting
» Chapter 8: Sample Implementation in PIRLS 2021 » Chapter 9: Reviewing the PIRLS 2021 Achievement Item Statistics » Chapter 10: PIRLS 2021 Achievement Scaling Methodology: Item Response Theory, Population Models, and Linking Across Modes » Chapter 11: Implementing the PIRLS 2021 Achievement Scaling Methodology » Chapter 12: Examining Country-Level Differences Between digitalPIRLS Data and Bridge Data » Chapter 13: Estimating Standard Errors in the PIRLS 2021 Results » Chapter 14: Using Scale Anchoring to Interpret the PIRLS 2021 Achievement Results » Chapter 15: Creating and Interpreting the PIRLS 2021 Context Questionnaire Scales