• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site

Impact of Education Quality Research on Policy Is Not as Significant as You Might Think

Impact of Education Quality Research on Policy Is Not as Significant as You Might Think

© Mikhail Dmitriev/ HSE

At a seminar held at HSE as part of the Days of the International Academy of Education in Moscow, Professor Gustavo E. Fishman (University of Arizona) likened international comparative studies of education quality to horse racing and discussed how these studies do not have as significant an impact on educational policy as is commonly believed.

In his opening remarks to the seminar, Andrey Zakharov, Head of HSE’s International Laboratory for Education Policy Analysis, noted that the results of comparative studies on the quality of education garner a lot of interest from politicians and the general public. Low results are a cause for concern. For example, in the 1990s, Germany and Poland dealt with so-called PISA-shock—their results in the comparative PISA study (an assessment of the functional literacy of 15-year-old students) were lower than expected. Japan experienced a similar situation when the results of another study—the TIMSS (an assessment of student preparedness in grades 4 and 8 in math and science)—were published. The question then arises: what we should change in the education system in order to improve it?

This question, albeit belatedly, has become a timely one for Russia, which has been participating in these studies since the 1990s, and whose results have now become a topic of active discussion. Most concerning among the results is Russian students’ less than satisfactory performance in PISA. However, this is not cause to conclude that these studies have had significant influence on Russian education policy. ‘Only some localized changes can be observed,’ Andrey Zakharov said.

In order to improve the quality of education in mathematics, statistics was added to the Russian school curriculum

Gustavo E. Fishman likened countries’ participation in international studies to horse racing, wherein contestants do not even know who they are competing with. It is believed that a country should be ahead of the curve, so a lot of time and money is spent on test preparation and assessment. At the same time, says the researcher, comparative studies were conceived in the 1960s without competition in mind, but for purposes of education quality assessment, and there was not as much interest in them. Now more than 140 countries participate in these studies which are growing in number, conducted by a wider variety of influential organizations (such as UNESCO, OECD and others), and yielding a larger amount of publications.

Gustavo E. Fishman

Professor at the University of Arizona

‘A new industry has appeared—an entire economic sector with professionals working in the area of international education quality studies. Even if you are not engaged in their realization and development, read about them, publish articles, and participate in the academic process.’

At the same time, the researcher believes, this entire industry exists on its own and has little impact on education policy. Comparative studies do not provide tools for improving the quality of education. But some people take dishonest measures to improve results in order to look better than their competitors.

In some countries, only the best students are selected to participate in the international testing

And this is not the only undesirable consequence.

An analysis of the educational policies of several countries participating in international studies showed that after the publication of their results, they implemented diametrically opposed changes. In one case, they centralized decision-making; in another, they decided to privatize schools; in a third, it was decided to increase state support for education and ensure that teachers meet high standards; and in yet another, it was decided to weaken state support of schools. ‘The studies were the same, but the resulting policies were different,’ said the researcher.

If a country performs poorly in an international study, the teachers are blamed, and the most common course of action is to fire the bad teachers.

In the media, teachers are usually criticized, Gustavo E. Fishman noted. This is at odds with the data we have from opinion polls about the role of teachers in people’s lives. In these surveys, only a small portion of respondents noted being dissatisfied with their teachers.

‘Bad’ teachers in schools


according to media data


according to survey respondents


Fishman’s main conclusion? Thanks to comparative studies of the quality of education, we have a wealth of interesting and informative data, the application of which remains an open question. Politicians use this data to justify their actions, but if countries demonstrating similar results in these studies implement opposite policies, can we trust their justifications? With every year, the assessment system becomes more perfected. But it does not help us come up with solutions to the problems that arise from it. Nor does it help us identify the problems, Fishman says.

See also:

HSE University: Moving Towards 2030 Goals

Okna Rosta, HSE University's bulletin, published a two-part interview with Rector Yaroslav Kuzminov about HSE’s Development Strategy this fall, and The HSE Look is glad to present it to our English-speaking audience as well.

The Effect of Digitalization on School Children Calculated for the First Time

The HSE Institute of Education together with Yandex completed Russia’s first large-scale experiment assessing the impact of online technology on primary school students’ performance in school. The researchers concluded that online completion of assignments help improve mathematical literacy, particularly among underachieving students.

Fighting Academic Failures: How to Prevent Underachievement at School

Children from undereducated, low-income families face a greater risk of poor academic performance. But schools are capable of decreasing these risks. Experts from the Center of Social and Economic School Development at the HSE Institute of Education have studiedinternational experience in addressing these challenges.

Russian Schools Are Changing Rapidly, But Not Always for the Better

According to a report by the Organization for Economic Cooperation and Development (OECD) entitled Measuring Innovation in Education 2019: What Has Changed in the Classroom?, Russia ranked among the top three countries where schools are changing most rapidly.

HSE’s Institute of Education Collaborates with Global School Laboratory

The Institute of Education at HSE has signed an agreement outlining collaboration with the online platform, GlobalLab - a community of teachers and students who work on joint research projects over the Internet.

HSE Institute of Education Now Able to Nominate Projects for Yidan Prize

The HSE Institute of Education has been authorised to nominate projects for the largest annual prize awarded in the field of education, the 2019 Yidan Prize. The size of the prize is nearly four times that of the Nobel Prize, with $3.87 million being awarded in two categories – education research and education development.

Ten Factors Ensuring Success in Educational Systems According to PISA Author

On April 14, 2017, Andreas Schleicher, Director for the Directorate of Education and Skills at OECD, spoke at the XVIII April Conference at Higher School of Economics (HSE). In 1999, he invented the Programme for International Student Assessment (PISA), one of the biggest international comparative studies of education quality. His honorary lecture was dedicated to global trends in the transformation of national education systems.

HSE Researchers Compare Performance in Mathematics for Children Starting School in Russia, Scotland and England

Researchers from the HSE Institute of Education have adapted and begun using an assessment tool for comparing the knowledge and skills of children starting school. Their first results were obtained from a sample of children starting school in Russia and the UK.


of Russian parents help their school-age children with their homework, or even do it with them.

It’s More Useful to Compare Students from Different Regions of a Single Country than from Different Countries

Professor Martin Carnoy of Stanford University and visiting professor at the Higher School of Economics, and Tatiana Khavenson, Research Fellow at the HSE Institute of Education, were among the authors of the report ‘An Analysis of the Impact of Education Policies on Student Achievement in the United States’, which was recently presented in Washington, DC. The key provisions of this report are of use when it comes to analyzing the situation in Russian education.