Michael O. Martin, Matthias von Davier, Pierre Foy and Ina V. S. Mullis
Overview
PIRLS 2021 is a comprehensive assessment of fourth grade students’ reading literacy achievement. Conducted on a regular five-year cycle, with each assessment linked to those that preceded it, PIRLS provides regular data on trends in students’ reading literacy on a common achievement scale. PIRLS 2021 begins the transition from paper-and-pencil to digital format, with about half the countries choosing to administer the digital format (digitalPIRLS) and half the paper format (paperPIRLS). To ensure comparability across formats, digitalPIRLS and paperPIRLS have the same content in terms of reading passages and questions, although digitalPIRLS takes advantage of some features and item types not available in paper and pencil mode. Countries choosing digitalPIRLS will also administer ePIRLS, an assessment of online reading to acquire and use information first conducted as part of PIRLS 2016. Similar to previous PIRLS assessments, PIRLS 2021 includes a series of contextual questionnaires to gather information about community, home, and school contexts for developing reading literacy.
Based on PIRLS experience in earlier assessment cycles, it is clear that achievement levels in reading comprehension vary widely both across and within countries, posing a challenge in matching the difficulty of the assessment to the reading ability of the students in every country. When an assessment is much too difficult or too easy for a population, little information is available to accurately measure performance. To address this challenge, PIRLS in the past has offered less difficult versions of its assessment materials that countries could choose to administer, beginning with prePIRLS in 2011 and following up with PIRLS Literacy in 2016. These efforts were successful in expanding PIRLS coverage of students at the lower end of the ability distribution but required separate (although linked) versions of PIRLS and did not address the need for more challenging material for higher achieving students.
PIRLS 2021 addresses the need for a broader range of assessment difficulty and better targeting of student ability by adopting a single unified assessment based on a new group adaptive assessment design (Click here for the rationale underlying group adaptive assessment designs). The new design is based on having three levels of passage difficulty—difficult, medium, and easy—that are combined into two levels of booklet difficulty. More difficult booklets are composed of two difficult passages or one medium and one difficult passage while less difficult booklets consist of an easy and a medium passage or two easy passages. Each country administers the entire assessment, but the balance of more difficult and less difficult booklets varies with the reading achievement level of the students in the country. For example, a country with higher average reading achievement (average score of 550 or above) could assign the more difficult booklets to 70% of its students and the less difficult booklets to 30% of its students, whereas a country with lower achievement levels (average score of 450 or below) could assign the more difficult booklets to 30% of its students and the less difficult booklets to 70%.
The group adaptive design improves the match between assessment difficulty and student ability in each country’s population by having a greater proportion of more difficult booklets in countries with relatively high achievement and a greater proportion of less difficult booklets in countries with relatively low achievement. Accordingly, the new design maximizes the information obtained from the assessment while minimally changing existing procedures and time requirements.
Although the group adaptive design was developed to provide a better match between assessment difficulty and student ability at the country level, it also is possible to apply the approach within a country, provided the country has clearly defined subpopulations that differ substantially in student achievement.