• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site

Measurement of learning progress: methodological approaches to assessment and new opportunities for school

2019
Department: Centre for Psychometrics and Measurement in Education
The project has been carried out as part of the HSE Program of Fundamental Studies.

Goals

The goals of the scientific project “Measuring educational progress: the assessment methodology and new possibilities for schools” include a series of studies according to the three directions as follows:

  • Conducting the first comparative international study of students’ basic reading skills at the start of schooling and their progress during the first year of schooling;
  • Revealing the core intellectual factor of student’s preparedness to school education;
  • Study on the prevalence of Attention Deficit with Hyperactivity Disorder (ADHD) manifestations in primary school children and its relations with academic achievements in primary school.

Methods

The research was conducted using mixed methods of educational measurement, including Item Response Theory for scoring and advanced statistics methods for the secondary analysis, such as quantile regressions, multilevel regression and growth modelling with mixed effects.

For ADHD evaluation in primary school students the Russian version of adopted DSM-IV questionnaire was used in the framework of iPIPS project (international Performance Indictors in Primary School). The cognitive skills of children were assessed using a set of instruments for the following constructs: non-symbolic number sense, cognitive control, recoding of number information into verbal information and vice versa, short-term and operative memory. The iPIPS instruments were used to study the possibilities of international comparisons between reading results of children at the start of schooling in two different cultures. The instruments of iPIPS project were adapted into the Russian language and standardized according to the international standards during the previous research stages.

Empirical results

The empirical analysis of the research was based on the extensive data from the previous waves of assessments of primary school students, as well as on this year’s data. The empirical data set for the analysis on early reading skills was the results of 2019’s expert panel consisted of 12 experts in linguistics or teaching who are fluent in English and work in the relevant sphere. The analysis of possibilities for the international comparison of reading results was conducted on the data sets which include the testing results of the representative samples of students from Russia (Tatarstan republic, 4940 children) and Britain (Scotland, 6627 of children). The study of the cognitive factors of children’s development, including recoding number information skills, was based on the May, 2019’s data set resulted from 396 Russian first-graders (191 (48%) boys and 205 (52%) girls) who were tested with a series of cognitive tests. ADHD research was based on the three waves of assessment: from October 2016 (when the students entered the first year of schooling); 2) May 2017 (end of the first year); 3) May 2018 (end of the second year). The total sample consisted of 1026 students. Thus, the data contained the results from primary school students at the start of schooling, and end of the first and second school years, and covered basic reading skills, math skills, phonology skills, cognitive skills (number sense, memory etc.) and the results of ADHD manifestations assessments.

Results

For the purpose of research on the possibility of international comparative study of reading development between the Russian and the British students, the audit of the construct was conducted, and the model of reading skills levels was defined.  The model described the three different levels of reading items difficulties, working similarly for the Russian and English version of the instrument.

The model of reading levels was implemented for the comparison of items hierarchy in the two versions of the reading instrument, and for benchmarking (cut score setting). Using Rasch modelling approach, the benchmarks were obtained for students who participated in the testing in Russia and Scotland. Benchmarking allowed for the comparison of the groups of students who had similar reading skills levels in two countries.

For the purpose of estimation of the core intellectual factor related to the schooling preparedness, the test battery for computerized cognitive assessment of first-graders was developed. The battery was aimed at assessing the cognitive predictors of mathematical development. The analysis of the relations of recording number information skills  with mathematical achievements at the end of the first school year, demonstrated that the visual to verbal recording is a significant predictor of math achievements, while visual to verbal recoding is not. In addition to this it was demonstrated that high level of phonology can be a resource for correcting mathematical difficulties in children who experienced difficulties in both reading and mathematics.

Also, in 2019, the results of students at the end of the first year of schooling were generalized and the presence of ADHD symptoms was assessed on the sample of more than 2000 children from Krasnoyarsk region. Using the data from the three previous waves of assessment two sub-factors of ADHD symptoms (inattention and hyperactivity/impulsivity) were scrutinized. The analysis demonstrated the gender differences on ADHD manifestations, with boys showing more often symptoms of hyperactivity/impulsivity and girls showing more often inattention symptoms. However, girls’ educational dynamic had more negative effects from inattention symptoms than that of boys.

Implementation and recommendations on the implementation of the results

The results of the study based on the iPIPS instruments can be used in school practice as an element of independent diagnostics and progress evaluation in the regional level (Krasnoyarsk region, Tatar republic), as well as the inner school instrument for achievement monitoring. The results of each assessment are highlighted in analysis reports for teachers and school principals. The reports included the methodological recommendations of the use of assessment results in individualized work with students of different levels of cognitive and non-cognitive development. In 2019, students’ parents also received the feedback about their children’s results after the assessment. In addition to that, each teacher of the class assessed with the special cognitive test battery, received an extra report with their class aggregated results and brief recommendations on the possible methods of cognitive development of the students.

Publications:


Ilyushina N., Kuzmina Y., Kaiky D. the Deficit of Phonological Processing Associated with both Maths and Reading Difficulties Rather than Separate Maths or Reading Difficulties / Издательский дом НИУ ВШЭ. Series WP BRP "PSYCHOLOGY". 2019. No. 107.