PIRLS – Progress in International Reading Literacy Study international project – is a monitoring study, organized by the IEA (International Association for the Evaluation of Educational Achievement). The study compares the level and quality of reading and text understanding by elementary school students in different countries of the world and determines differences in national education systems. The research assesses two types of reading that are most frequently used by students in their school studies and outside school: reading to acquire reading literary experience and reading to acquire and use information. In 2006, the study involved 40 countries, including Russia.
PISA – Programme for International Student Assessment. International comparative study organized by the IEA (International Association for the Evaluation of Educational Achievement). The study assesses functioning literacy of 15-year old students in the areas of reading, mathematics and natural sciences. In 2006, the study involved 57 countries, including Russia.
Objectives of the research:
1) To follow the dynamics of reading literacy development among Russian schoolchildren in elementary and middle schools, determine possible reasons for the conflict between high results in PIRLS and poor results in PISA.
2) To determine a list of reasons for unsatisfactory results in the unified state examination: evaluation of the contribution of emotional-motivating characteristics of students, family characteristics, school characteristics etc.
3) To develop approaches and models of rendering assistance to the schools for which the problem of improving education quality is especially acute. Solving this problem would ensure equal access to high-quality basic education.
Recent studies have demonstrated that with the prevailing impact of family rather than school education resources on child’s educational achievements (which is especially true for Russia), schools are able to improve educational results in reading literacy through setting the right tasks and educational strategies. Still, most Russian schools do not take advantage of these opportunities adequately. This has been attributed to the attitudes of main participants of the educational process as well as to specifics of teaching techniques. Small or inadequate activity of the middle school results in the absence of progressive development of schoolchildren’s reading competence from the start to the end of the middle school, which is registered by the results of comparative international studies (PIRLS, TIMSS and PISA).
As part of the research, we have made:
I. Qualitative analysis and comparisons of the text types and questions used for testing understanding of texts used in the PIRLS and PISA studies. Based on the available tools of PIRLS and PISA studies, we have developed a unified test for students of the 4th, 6th and 8th grades that determines the level of their reading skills that are the focus of the PIRLS as well as PISA studies. In addition, we have conducted a quantitative analysis of task distribution in school textbooks in the most frequently used programs for elementary schools in terms of forming reading skills tested by PIRLS.
The results of the study demonstrated that in terms of reading skills measured using a method that combines the PIRLS and PISA tests:
1) the 6th grade does not differ from the 4th grade in terms of understanding information texts.
2) the 9th grade differs considerably from the 4th grade and the 6th grade, however accumulation of reading literacy during five years of education is modest: from 5% to 16% for some indicators (see charts 3 and 5).
We have demonstrated that two years of education in the Russian middle school (grades 5-6) do not have a significant effect on schoolchildren’ ability to understand information texts. The 9th grade differs substantially from the 4th grade, however accumulation of reading literacy over these five years is insignificant. The results make the problem of textbook quality and the ways to use them in education process especially acute.
II. Quantitative analysis of task distribution in textbooks. We have drawn the following conclusions.
1. The structure of tasks in most Russian textbooks devoted to the world around predominantly include tasks that check the skill of finding specific information in the form of direct reproduction.
2. Textbooks devoted to literary reading include tasks for all groups of skills.
3. Textbooks devoted to the world around are characterized by a high proportion of tasks that don’t have enough information in the text and don’t have references to additional sources of information, which can make the situation frustrating for schoolchildren.
4. Information provided in Russian manuals (both for the world around and literary reading) is distinguished by uniqueness of interpretation and absence of decision making across the text.
5. Textbooks in literary reading are mostly oriented towards stimulating readers to form and defend their own position. Textbooks devoted to the world around are primarily targeted at reproduction of the author’s position rather than creative assimilation of various ideas (pluralism).
III. Studying the results of State Unified Examination led to the following conclusions.
1. Higher results in mathematics are demonstrated during in the State Unified Examination by graduates from two-parent families, while the results in the Russian language among graduates from two-parent and single-parent (mother-led) families do not differ. The risk group in terms of SUE results include father-led single-parent families and no-parent families. Graduates that are only children post better results than graduates with siblings. No relation was registered between absence/presence of grandparents and SUE results. These regularities are confirmed by samples from two pilot regions. The sample from Pskov Region differs in terms of several parameters.
2. Higher SUE results in mathematics and the Russian language are demonstrated by graduates from full schools (middle school + high school) whose parents have a higher education. Moreover, in families where one parent has higher education, regardless of the education level of the other parent, SUE results are usually above average. Similar results obtained from the sample of night school graduates confirm this pattern. The group of risk in terms of SUE results consists of graduates from low-education level parents (including families when one of the parents has low level of education regardless of the education level of the other parent) (up to the 9th grade), which is especially evident in mathematics.
3. We note clusters of schools that demonstrate poor results based on similar problems in management, human resources, organization of the teaching process and cohorts. In the most disadvantaged groups, these problems are registered in a package.