• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site

An in-depth study of the results of an international assessment of education standards and the Unified State Exam in the context of economic indicators in education

Priority areas of development: state and public administration
2012
The project has been carried out as part of the HSE Program of Fundamental Studies.

This year our work has taken us in the following three directions of research:
1)          Analysis of how the character of schools and teachers influences student achievements in international comparative research. We look at reasons for the gap between PISA and TIMMS results
2)          The USE as a predictor of student performance at university.
3) Analysis of the effects of teacher credentials and schooling/teaching practices on students’ outcomes (measured by results of the Unified State Exam) for different categories of students.
 
Direction 1. Analysing the effect of school and teacher characteristics on student achievements in international comparative research. Sources for gap between PISA and TIMMS results
The focus of the study:
-      The effect of one extra year in school on the improvement of PISA results for Russian 15 year old students.
-      The reasons for the difference in TIMSS and PISA results 
Goal:
For part 1. The effect of one extra school year for Russia and some other countries.
¾  To explore how one extra year at school affects students’results in PISA.
For part 2. Sources for gap between PISA and TIMMS results (based on Russian results in PISA and TIMMS)
¾  To explain the difference between PISA and TIMSS results in Mathematics by exploring the latent structure of the test items.
¾  To conduct testing of 5000 students in 42 Russian regions on the sample of TIMSS-2011 using the methodology of international comparative study PISA-2012.
 Data:
TIMSS and PISA database, Unified State Exam scores.
Main results:
For part 1. The effect of one extra school year for Russia and some other countries.
It was revealed that the effectiveness of one extra year at school is not significant when the students of all educational programs are taken as a whole. When examined for different programs separately, increased effectiveness may be reported for vocational schools. The size of the effect varies in different countries.
For part 2. Reasons for the gap between PISA and TIMMS results (based on the Russian results in PISA and TIMMS)
A five-factor structure was revealed in TIMSS items as well as a six-factor structure in PISA. Further analysis showed only one skill in common for PISA and TIMSS test structures.
Application area: 
Educational policy, research in education.
 
Direction 2. The USE as a predictor of university student performance.
The focus of the study
The role of the Unified State Exam in the Russian university entrance procedure
Goal:
To evaluate the predictive power of USE results in various subjects.
Data:
We collected data from 16,500 students at 4 Russian universities. We obtained students’ Unified State Exam scores (in the subjects required for submission) and university grades (in all courses studied during the first two years). Approximately 50 different fields of study were covered.
 
Main results:
On average, the USE determines up to 50% of students’ academic performance during the first year of higher education. So we can conclude that it is one of the main predictors of educational success.
An attempt to implement weighting based on the regression results does not lead to a significant increase in predictive power.
For some of the school subjects, the USE scores very seldom have any significant relationship with achievement at university level.
Application area:
Educational policy, research in education, studying of the university entrance procedure.
 
Direction 3.Analysis of the effects of teacher credentials and schooling/teaching practices on student outcomes (measured using the Unified State Exam) for different categories of students.
The focus of the study:
Teacher credentials, schooling and teaching practices. In particular, the selection of students for streaming and in-class grouping of students by their abilities is of special interest
Goal:
To estimate the effect of teacher credentials and schooling/teaching practices on student outcomes.
Data:
¾             Student data representing one of the Russian regions was collected in June 2012. The data included a set of students’ and families' background characteristics and individual students’ Unified State Exam scores in Russian and Mathematics.
¾             Russian language and Mathematics teachers' data were collected at the same time.
 
Main results:
The negative effect of the most qualified (most experienced) Mathematics teachers compared to their marginally less qualified colleagues was confirmed. The largest gain from having a marginally less qualified math teacher is in evidence for students with higher family economic capital.
Being in an advanced stream, as well as frequent in-class grouping of students by ability, produce positive gains in Mathematics. At the same time, the most frequent work with a class as a whole produces negative maths scores, relative to Russian.
Students with higher family cultural levels and higher economic capital get more from being on an advanced math stream compared to students with lower family capital. For students with lower family economic capital, advanced Russian study produces negative gains. Students with higher family economic capital achieve more negative scores from more frequent work in class, as a whole.
 
Application area:
Educational policy, research in education.

Publications: