PIRLS 2021 International Results in Reading
Performance at International Benchmarks
To implement meaningful policy and curriculum reform, it is important to understand the differences in students’ reading competencies associated with higher or lower scores on the PIRLS reading achievement scale. For example, in terms of students’ reading comprehension skills and strategies, what does it mean for a country to have an average achievement of 513 or 426? Looking at additional data analyses can help a country determine if its students have gained the reading comprehension skills covered in the reading curriculum.
To provide an interpretation of the PIRLS 2021 average results summarized on the PIRLS achievement scale for reading comprehension in the fourth grade, this section of the report describes achievement at four points along the scale as International Benchmarks: Advanced International Benchmark (625), High International Benchmark (550), Intermediate International Benchmark (475), and Low International Benchmark (400). To develop the descriptions of the reading comprehension skills and strategies demonstrated by fourth grade students reaching each International Benchmark, the TIMSS & PIRLS International Study Center conducted a scale anchoring exercise together with the PIRLS 2021 Reading Development Group (RDG). With the PIRLS 2021 transition to a digital assessment, the scale anchoring was based on digital data. Further detail about the scale anchoring methodology is provided in Chapter 14 of Methods and Procedures: PIRLS 2021 Technical Report.
Descriptions of the PIRLS 2021 Texts
To reflect the PIRLS 2021 Reading Assessment Framework (Chapter 1 in PIRLS 2021 Assessment Frameworks), the International Benchmark descriptions were developed separately for the two overall reading purposes—Literary and Informational. The texts for the two purposes are described below because the benchmark descriptions consider the difficulty of the texts the students are asked to read in PIRLS. Students use somewhat similar reading comprehension skills and strategies with each higher benchmark, but the complexity and difficulty of the texts increases. Average text difficulty has been estimated by the average percent correct across items based on digital data.
Literary Texts
The PIRLS 2021 literary assessment included nine texts presented in the PIRLS 2021 digital format. The texts were complete short stories or episodes accompanied by supportive illustrations. The texts included contemporary and traditional stories with one or two main characters, a plot with one or two central events, and an overall theme or message. Taken as a whole, the literary texts included a range of styles designed to encourage students to engage with the events, settings, actions, consequences, characters, atmosphere, feelings, and ideas in the stories.
In accordance with the group adaptive assessment design, the texts represented three levels of difficulty—easy, medium, and difficult. The easy texts (76% correct on associated items, on average) were relatively accessible, approximately 500 words in length, with a clear linear structure, explicit meanings, and simply described characters. The language featured everyday vocabulary and straightforward sentence structures. The difficult texts (56% correct on associated items, on average) were relatively complex, approximately 850 words in length, with scope for exploring layers of meaning, such as plot twists, development of complicated ambivalent characters, and abstract ideas. They included a range of vocabulary, imagery, and figurative language. The medium texts (66% correct on associated items, on average) were of intermediate complexity, approximately 700 words in length, with a narrative structure and a clear message.
Informational Texts
The PIRLS 2021 assessment of informational reading included nine texts presented in the PIRLS 2021 digital format. The informational texts included a variety of continuous as well as non-continuous texts with charts and graphs. The texts had presentational features such as diagrams, maps, illustrations, photographs, or tables. The range of material covered scientific, biographical, and historical information and ideas. Texts were structured in a number of ways, including by logic, argument, chronology, and topic. Several included organizational features such as subheadings or text boxes.
In accordance with the group adaptive assessment design, the nine texts represented three levels of difficulty—easy, medium, and difficult. The easy texts (74% correct on associated items, on average) were approximately 500 words in length with a clear structure, explicit meanings, and straightforward sentence structures. The difficult texts (51% correct on associated items, on average) were approximately 850 words in length and conceptually more demanding, based on abstract or technical ideas and with a substantial number of embedded details, some complex sentences, and topic-specific vocabulary. The medium texts (60% correct on associated items, on average) were of intermediate complexity, approximately 700 words in length.
The PIRLS 2021 assessment also included five tasks specifically designed to assess online informational reading based on simulated websites about scientific and historical subject matter. Each task was structured as a class project or report, with an avatar teacher who introduced the questions and guided the students through the task. Each task involved students working across approximately three different websites with an average of 1,000 words of text per task and as many as 10 web pages. In addition to the text, the tasks included different kinds of visual information, such as photos, charts, and maps, as well as many navigational and dynamic features, such as animations, hyperlinks, tabs, and pop-up boxes. The five tasks were either difficult or medium (60% correct on associated items, on average).
Examples of the PIRLS 2021 Texts and Items
Video examples of one Literary text, The Empty Pot, and two Informational texts, The Amazing Octopus and Oceans (ePIRLS task), can be viewed below. The videos show each text in its entirety, together with its associated items.
Video Examples of the PIRLS 2021 Texts
The Empty Pot
Digital PIRLS Literary
Oceans
ePIRLS Online Informational
The Amazing Octopus
Digital PIRLS Informational
- The Empty Pot is a literary story with a message. The Emperor passes out seeds for a contest to see who will be the next Emperor. Jun is ridiculed when his seed does not grow and is ashamed to face the Emperor. However, Jun’s empty pot indicates his honesty because the seeds were boiled.
- The Amazing Octopus is an informational article about the characteristics of octopuses and some behaviors they display when they are in aquariums. Newly developed in PIRLS 2021, it has some animation to indicate the direction possible in future assessments.
- Oceans is an ePIRLS task consisting of three simulated websites about 1) the benefits of oceans, 2) ocean habitats, and 3) the problem of plastic pollution. Across the colorful web pages, there are a variety of navigational features, pop-ups, links, a video, and advertisements.
Descriptions of Reading Achievement at the PIRLS 2021 International Benchmarks
The PIRLS 2021 International Benchmarks build on each other, representing increasingly demanding reading comprehension skills and strategies with each higher benchmark—Low, Intermediate, High, and Advanced. The students who reached a benchmark also reached all previous benchmark(s). For example, students who reached the Advanced Benchmark also reached the Low, Intermediate, and High Benchmarks. Consistent with the two reading purposes that provide the foundation of the PIRLS 2021 Reading Assessment Framework, the scale anchoring analysis was conducted separately for the literary and informational texts and items. Within each reading purpose, the progression in reading comprehension processes can be seen from the description of the different benchmarks.
The interactive feature below presents the different PIRLS 2021 international benchmarks, and provides examples of items for each corresponding benchmark.
