PISA Results Linked to More than Education System
HSE’s Institute of Education hosted a visit by University of Arizona Professor Emeritus David Berliner. In an interview with the Institute, Professor Berliner discussed problems of schooling in the U.S. and Russia, possible ways of evaluating the work of instructors, and also how the results of international educational research should be factored into decision-making.
The main problem is not the instructor, but the family
Most countries incorrectly interpret the results of the international educational studies PISA, TIMSS, and PIRLS, Professor Berliner believes. Overall, American schoolchildren demonstrate very low results on the Programme for International Student Assessment (PISA), which constantly pushes politicians to discuss the low quality of education, the substandard work of teachers and principals, the low quality of academic programmes, etc.
Analysis of PISA results show, however, that while most American schoolchildren really do perform poorly on the PISA, around 20% have very high results, and these are associated exclusively with socio-demographic factors. Children from middle-class families in the U.S. generally have very high results on international and national educational studies, while the scores of children from poorer families are rather abysmal. In addition, children from middle-class families are highly motivated and have all the conditions at home to do well in school. Around 24% of schoolchildren are below the poverty line in the U.S.; that is, a very substantial amount. On the other hand, in Finland, where students perform well on the PISA, only 4% of schoolchildren are below the poverty line.
But let us assume that in the U.S., there are simply not as many jobs that must be filled by people with high results in the PISA’s key components. How many of such jobs are in the Russian economy?
It appears that the results of international comparative studies are linked not only with the education system in a country, but also with the social situation as a whole. In the U.S. and Great Britain, stratification is very high and there are a fairly high number of poor people. This is why the results are average overall, while variance is extremely high. In countries of Northern Europe, there is less stratification and access to quality education is high, so overall results are high and variance is low. For that reason, when we talk about increasing the quality of education, it is necessary to think not only about the teacher, principals, and programmes of study, but also about the social policies of a country. For the U.S., the recipe for increasing PISA results lies in lowering poverty as a whole.
It is likely that in Russia the results of international comparative studies are also linked to social and economic traits of the country, and not just with education. This is why it is necessary to ask another important question: does a country’s economy currently need high PISA results? Finland is a small country; its entire population is significantly lower than even the number of American schoolchildren with the best results on the PISA. The Finnish economy could provide them all with jobs, it seems. But let us assume that in the U.S., there are simply not as many jobs that must be filled by people with high results in the PISA’s key components. How many of such jobs are in the Russian economy? This is why it is not necessary to chase after high PISA results like this; but this is exactly what is happening in both Russia and the U.S. now.
How to measure the quality of a teacher’s work
The government and business are inclined to believe that quantitative indicators can measure anything. In reality, this is not always possible, particularly when this concerns pedagogical performance, Professor Berliner believes. The idea exists in both the U.S. and Russia that the quality of a teacher’s work can be measured by academic performance or by a student’s subject knowledge. Actually, this is also not quite right.
Student performance and subject knowledge depend on an entire host of factors. Not least important are the socio-economic status and the cultural baggage of the parents. Children of wealthy and well-educated parents do better in school than children of poor and uneducated parents, and this is irrespective of location. A child's motivation to learn also depends on family. It turns out that in testing schoolchildren to measure the quality of teachers, we are actually assessing the family. And by using students’ academic performance to evaluate teachers’ pedagogical capabilities, we end up pushing teachers to give children undeservedly high marks.
It is now impossible to assess the quality of a teacher using quantitative indicators. Instead, Professor Berliner suggests using expert reviews. The experts would mostly include school principals. In both the U.S. and Russia, principals deal with a large amount of paperwork and bureaucratic tasks. This is wrong – principals must first and foremost assist teachers and assess their work. This can begin simply by attending classes regularly to see what a teacher does. It is also not a bad idea to video-record everything that happens in the classroom to see what goes on.
Aside from principals, outside experts must also assess teacher performance. They could consist of the most experienced and qualified practicing pedagogues. They should attend their colleagues’ classes and evaluate what happens there. Children’s subject knowledge does not always speak to a teacher’s abilities, and neither does the atmosphere in the classroom or the student-to-teacher ratio.
By visiting a colleague’s classroom, the expert teacher would also be able to help improve his or her work. For example, a useful practice is to conduct lessons at each other’s school. This helps better understand the challenges teachers face and also find ways to solve them.
According to Professor Berliner, the main problem for Russian teachers is the authoritarian and didactic style of work. This may be linked to the number of children in the classroom. In U.S. public schools, where there are around 30 children in a class, teachers behave much more didactically than in private schools, where class size is 10-15 children. A good approach can nevertheless be found for large classes. For example, teachers can be taught to break children up into smaller groups, each of which is given different tasks.
Ekaterina Rylko, for the HSE News Service
The HSE Institute of Education together with Yandex completed Russia’s first large-scale experiment assessing the impact of online technology on primary school students’ performance in school. The researchers concluded that online completion of assignments help improve mathematical literacy, particularly among underachieving students.
Children from undereducated, low-income families face a greater risk of poor academic performance. But schools are capable of decreasing these risks. Experts from the Center of Social and Economic School Development at the HSE Institute of Education have studiedinternational experience in addressing these challenges.
At a seminar held at HSE as part of the Days of the International Academy of Education in Moscow, Professor Gustavo E. Fishman (University of Arizona) likened international comparative studies of education quality to horse racing and discussed how these studies do not have as significant an impact on educational policy as is commonly believed.
According to a report by the Organization for Economic Cooperation and Development (OECD) entitled Measuring Innovation in Education 2019: What Has Changed in the Classroom?, Russia ranked among the top three countries where schools are changing most rapidly.
The Institute of Education at HSE has signed an agreement outlining collaboration with the online platform, GlobalLab - a community of teachers and students who work on joint research projects over the Internet.
On April 14, 2017, Andreas Schleicher, Director for the Directorate of Education and Skills at OECD, spoke at the XVIII April Conference at Higher School of Economics (HSE). In 1999, he invented the Programme for International Student Assessment (PISA), one of the biggest international comparative studies of education quality. His honorary lecture was dedicated to global trends in the transformation of national education systems.
HSE Researchers Compare Performance in Mathematics for Children Starting School in Russia, Scotland and England
Researchers from the HSE Institute of Education have adapted and begun using an assessment tool for comparing the knowledge and skills of children starting school. Their first results were obtained from a sample of children starting school in Russia and the UK.
It’s More Useful to Compare Students from Different Regions of a Single Country than from Different Countries
Professor Martin Carnoy of Stanford University and visiting professor at the Higher School of Economics, and Tatiana Khavenson, Research Fellow at the HSE Institute of Education, were among the authors of the report ‘An Analysis of the Impact of Education Policies on Student Achievement in the United States’, which was recently presented in Washington, DC. The key provisions of this report are of use when it comes to analyzing the situation in Russian education.