Statistics Explain Why Less-Competent People Tend to Overestimate Themselves
Why do people sometimes overestimate their knowledge—more often than not, those who are not well-versed in the subject at hand? Twenty years ago, two researchers studied this phenomenon and tried to explain it from a psychological point of view. This phenomenon is now known as the Dunning-Kruger effect after the names of the researchers. According to a report presented at a seminar of the HSE Department of Applied Economics, this effect has a purely statistical explanation.
In 1999, Justin Kruger and David Dunning described an experiment in which students were asked to take tests to assess their sense of humour, logical reasoning, and knowledge of English grammar, and suggest how well they did compared to the other respondents. Most participants significantly overestimated their abilities (on average, they thought they did better than 66% of their classmates), and the lower the test score was, the more the perceived score differed from the actual score. Only the strongest participants underestimated themselves. The authors of the article attributed this to the fact that people with weaker abilities not only performed worse on assignments, but also inadequately evaluated their knowledge.
Works criticising the conclusions of Dunning and Kruger began to appear several years after the publication of the original study. Many authors recognised the existence of the effect, but disagreed with the proposed explanation. One of the first critical articles attributed the Dunning-Kruger effect to people’s general tendency to rate their knowledge rather highly and to regression toward the mean. This regression was first described by Francis Galton in 1886: he observed that tall fathers tended to have shorter sons and short fathers tended to have taller ones. This is an example of the so-called tendency ‘toward the mean’.
Research Professor Anatoly Peresetsky, HSE Faculty of Economic Sciences, and his co-author Jan Magnus, University of Amsterdam, suggested a statistical rather than psychological explanation of the Dunning-Kruger effect. The researchers pointed out that the experiments in which this effect was studied allowed the participants to achieve scores within a strictly specified interval—for example from 0 to 10 or from 0 to 100 points. Two other papers previously noted this fact, but the idea was not developed any further.
Suppose that people can guess their score correctly within a limited margin of error (ie there is an equal probability that people will underestimate and overestimate themselves). As it is impossible to get less than 0 or more than 100 points, the two groups of participants are limited in their assumptions—those who did very badly in the test cannot say that they got -5 or -10 in their assignments, and those who answered everything correctly cannot say that they expect to get 110. It turns out that the average expected scores of the former group will be higher and the expected scores of the latter group will be lower than they might have been without the restrictions of the scale.
The authors tested this assumption on 665 HSE students who took a statistics exam in 2016–2019. In the middle of the exam, students were asked to write down what grade they expected to get (a small bonus was promised to those who guessed correctly). The results demonstrated the Dunning-Kruger effect.
To test whether it is possible to see the same deviations of expected estimates from the actual ones using only the laws of statistics, the authors have proposed a simple statistical model (the generalised tobit model, well known in economics). The model takes into account the limitation of prediction—the number from 0 to 100. The model-based pattern of the graph depends on three parameters: the extent to which participants can be wrong in their assumptions and two random variables that characterise the deviation of prediction scores from 0 and 100 (they are used to fulfil the requirement of bounded scores from above and below). The researchers have shown that with realistic values of these parameters, it is possible to create a graph that gives an almost perfect description of the actual data.
Anatoly Peresetsky, Research Professor at the Faculty of Economic Sciences
‘The Dunning-Kruger effect does exist, but it is a statistical artifact and concepts from psychology or cognitive science are not required to explain it. The question of why “psychological” explanations of this effect are still popular in psychology literature most likely deserves psychologists’ attention.’