Goal of research
The goal of the research is connected to the five fields of project development:
The development of the instrument for the assessment of reading comprehension and mathematical literacy skills in students of 4th grades (to complete the set of already developed tools for assessing children in the 1st and 3rd grades).
Transfer of the assessment system to the Russian-language platform. Development of the new system architecture convenient for all categories of users.
Calibration of item characteristics in the instruments for the 1st grade in order to verify the composition of the instrument.
Development of the methodology and procedures for vertical linking of the results obtained during the longitudinal project. The aim is to obtain comparable points for measurements in different years that are on the same scale. As a result, an empirical base of longitudinal research will be obtained.
Analyze the collected data on the base of the previously developed growth models methodology in order to determine the trajectories of the development of mathematical skills from measuring at the beginning of the first year through measuring at the end of the first year to measuring at the end of the second class.
In the field of psychometric works (Goals 1, 3, and 4), methods of the item response theory, in particular, the Rasch modeling methodology, have been used. In the field of statistical studies (Goal 5), the principles of multi-level regression analysis and growth models were used. To implement the work on the development of instruments (Goal 1), the methodological principles of test development were used (development of test items and test design). The Goal 2 (transfer of the assessment system to the Russian-language platform) was carried out based on the requirements for security, data protection and convenience for users.
A representative sample of students from 30 Kazan schools in the Republic of Tatarstan was used for psychometric research on the vertical linking of longitudinal scales (Goal 4) and research on the trajectories of the development of mathematical skills (Goal 5). The students were tested in three assessment cycles (October 2016, when the children first entered school and had studied for one month; May 2016, when the children finished the first grade; April 2017, when the children finished the second grade of the primary school). Sample stratification was based on the type of school (according to the name and status of the school as a general education school, gymnasium / lyceum or a school with in-depth study of subjects) and its location. One class, which was randomly selected among all the first classes of the school composed the sampling unit. In total, 1,289 children were surveyed in October 2016, 1,229 first-graders in May 2017 (the attrition of the sample is less than 5%), and 1,347 students in April 2018 (a larger number of children is associated with transfer of students during the school year between classes and schools). The final sample of the children who passed all stages of the evaluation included 1026 students (49% of them are girls).
Results of research
Two new computerized instruments were created for evaluating reading comprehension and mathematical literacy for fourth-graders while pursuing Goal 1. The requirements of the International Standard for the Assessment of Reading Comprehension and the requirements of the Federal State Educational Standards for Primary Education were taken into account to develop a tool for the assessment of reading comprehension. The items of the tool in mathematics are based on the Federal State Educational Standards for Primary Education and are not aligned with a specific kits of educational materials (textbooks etc).
Through the transfer of the diagnostic system to the Russian platform (Goal 2) a comprehensive system including a set of modules was created. Modules included private online accounts of schools, round-the-clock technical support during assessment periods, modules for cognitive assessment of children, non-cognitive assessment of children, questionnaires, information upload, automated generation of reports and feedback).
The results of the calibration tasks for the 1st class (Goal 3) were used to optimize the structure of the instrument scales (some tasks were moved to another place of the scale). However, most of the tools remained unchanged, since the analysis confirmed the optimal functioning of the CAT algorithm.
An empirical database for three cycles of assessing the same schoolchildren was obtained as a result of work on the vertical linking of scales (Goal 4). The results of all three cycles can be directly compared between years (that means that a single scale was obtained).
The data obtained in pursuing Goal 4 were used to build growth models for evaluating the trajectories of skills development in mathematics from the beginning of the school to the end of grade 2. The compensational pattern of mathematical skills development was confirmed. The gap between schoolchildren with different levels of math skills at the start of schooling is reduced by the end of the 2nd grade. The level of mother's education is positively associated with achievements in mathematics. However, the gap between schoolchildren with different levels of mother's education reduces by the end of the 2nd grade. The contribution of verbal skills in mathematics achievement also decreases at the end of the second grade. But the negative effects associated with inattention disorder do not diminish with time. These results suggest that elementary school education equalizes children, who are initially at different levels of mathematics training in a number of factors.
Level of pursuing, recommendations on pursuing or outcomes of the pursuing of the results
Methodological solutions obtained during the work in 2018 can be implemented to other projects and areas of research. The obtained empirical results can be used to improve the functioning of the primary education system.