PIRLS 2021 International Results in Reading
Countries’ Reading
Achievement
The PIRLS 2021 data about students’ reading achievement provide an extremely valuable resource for continuing research about the impact of the COVID-19 pandemic on students’ learning. PIRLS 2021 is the only large scale international assessment that successfully collected data during education’s COVID-19 disruption. Further, the achievement data is accompanied by contextual information collected from several sources: principals’ reports about school conditions, students’ attitudes toward their reading instruction, and parents’ perceptions regarding the impact of the pandemic on their children’s learning (see later sections of the report on Home Environment Support; School Composition, Resources, and Climate; and Students’ Reading Attitudes and Behaviors).
The About PIRLS 2021 Section
The first section, About PIRLS 2021, provides a considerable amount of information about the PIRLS 2021 international assessment of reading comprehension at fourth grade (e.g., the content of the assessment, the 57 countries and 8 benchmarking entities participating in PIRLS 2021, the transition to digital assessment, and the numbers of students assessed).
Despite coinciding with school disruptions due to the COVID-19 pandemic, the PIRLS 2021 data collection successfully included nearly 400,000 students in 57 countries worldwide. Although the pandemic necessitated changes in school operations, frequently leading to school closures (see Exhibit 3 in About PIRLS 2021), the countries were able to make various adjustments in their data collection schedules, ranging from minor changes to essentially heroic efforts.
Exhibit 5 in About PIRLS 2021 shows the chronology of the PIRLS 2021 data collection from November 2020 through July 2022. Fortunately, modifying the schedules had minimal impact on the quality of the PIRLS 2021 data. The thorough and well documented adjudications of the PIRLS 2021 sampling procedures and data collection outcomes found that the PIRLS guidelines to ensure high quality were met for the most part, while the few exceptions were annotated appropriately (see Appendix A). The International Quality Assurance Program also monitored and documented the data collection activities (see Chapter 6 in Methods and Procedures: PIRLS 2021 Technical Report).
Impacts of Modifying the Assessment Schedule on Students’ Achievement
Consistently across assessment cycles, the PIRLS data collection procedures have included conducting the assessments at the end of the fourth grade school year so that as much of the curriculum has been covered as possible. In light of the data collection challenges in PIRLS 2021, 43 of the 57 countries managed to assess students at the end of the target school year, including conducting the assessment a year later than originally scheduled in a few countries (this is annotated in the exhibits with a bowtie after the countries’ names). So far, no discernible achievement differences have been identified that are associated with assessing fourth grade students one year later than initially scheduled.
However, in the other 14 countries (all in the Northern Hemisphere), the necessary modifications to the data collection schedule delayed assessing students in the fourth grade cohort until the beginning of the fifth grade. In reviewing the PIRLS 2021 achievement results, it appeared that some of these countries had an achievement advantage in PIRLS 2021 that also was manifested in the relatively larger trend increases between 2016 and 2021 (see report section on Trends in Reading Achievement).
Of course, the reasons for any achievement differences are unknown. However, although there was variation, the average age of students in the 14 countries that delayed assessment until the beginning of the fifth grade was half a year older on average than the average age of students assessed at the end of fourth grade.
Average Ages of Students Assessed in PIRLS 2021 and PIRLS 2016 by Data Collection Period
| PIRLS 2021 Data Collection Period | PIRLS 2021 | PIRLS 2016 | Difference |
|---|---|---|---|
| Assessed Fourth Grade Students at the End of the School Year | 10.2 | 10.2 | 0.0 |
| Delayed Assessment of Fourth Grade Cohort at the Beginning of Fifth Grade | 10.8 | 10.2 | 0.5 |
Because of rounding, some results may appear inconsistent.
Beyond finding that these students were comparatively older, unfortunately, without any information about the reading achievement of the students in the 14 countries at the end of the fourth grade or their activities over the summer months, the PIRLS 2021 data in and of itself cannot be used to disentangle the extent of the impact of the delayed assessment on students’ reading achievement. Researchers may be able to use within country data and local insights to study this issue in the future.
For now, however, throughout the report pink highlighting has been used to identify the results of the 14 countries where delayed assessment until the beginning of the fifth grade resulted in collecting data from a sample of comparatively older students.
