HSE Launches Service for Psychometric Analysis of Online Courses
The website ‘Modern Digital Educational Environment in the Russian Federation’ has integrated an automated service for psychometric data analysis, which helps to improve the quality of educational content and the effectiveness of studying by analyzing the ‘digital footprints’ of students enroled in online courses.
The psychometric data analysis service was developed by HSE and assigned by the Russian Ministry of Education and Science as part of the ‘Modern Digital Educational Environment’ priority project aimed at improving the efficiency of online courses. The service is integrated as an online.edu.ru subsystem that makes it available to all developers of online courses registered on the site. Big data analytics will help developers to improve the content of online courses, optimize content for different groups of students, and make an unbiased assessment of test results.
‘Psychometrics - a special area of science - is developing rapidly in the world; it can significantly improve the quality and reliability of assessment procedures in online courses,’ says Yaroslav Kuzminov, HSE Rector. ‘Psychometric data processed with the help of artificial intelligence will make it possible to create many types of examination tasks. Computer technology will help to increase the efficiency and credibility of online learning, as well as to approve online courses as equal elements of the educational programme along with lectures in the classroom. The psychometric data analysis service developed by specialists from the Higher School of Economics is one of these technologies.’
Students of online courses leave a so-called ‘digital footprint’ - statistics on video views, passing tests (time spent on tests, number of attempts, ratio of correct/incorrect answers), and assessment of other students’ works (peer review). The data stored on online course platforms can now be analyzed automatically by using the new service.
The service analyzes an online course in four main areas: content complexity, student qualification, students' interaction with the content, and analysis of assessment tools. The analytical report received by the course developer includes recommendations for course modernization in order to improve its quality. Reports and recommendations help to optimize the difficulty and availability of course content for students, increase student involvement, match course complexity and student qualification, and improve assessment tools of the online course (test tasks and tasks for mutual evaluation).
‘During the service’s development, 270 online courses were reviewed and modified. More than 100 authors, developers and methodologists have been trained at HSE to work with analytics, and an electronic handbook containing methodological materials and recommendations for online course modernization has been published in the public domain for future users of the service. The improved courses will start in the autumn,’ says Evgenia Kulik, HSE Director for Online Education.
Once adjustments are made, an online course is automatically analyzed again for psychometric data and receives a ‘quality badge’ — a mark in the course passport on online.edu.ru. The priority project ‘Modern Digital Educational Environment in the Russian Federation’ focuses on quality assessment and comprehensive analysis of online courses, as well as increasing the efficiency of learning.
Comprehensive expertise helps to increase competition between platforms and developers, encourages authors to improve the quality of online courses, and provides educational organizations and students access to reliable information about online courses as part of virtual academic mobility; it also gives students and educational organizations confidence in online learning reliability.
In 2017, HSE won the tender to develop and implement methods and tools for psychometric analysis of online courses as part of the priority project ‘Modern Digital Educational Environment.’
Psychometrics developed as a field of study over a century ago and today its specialists are engaged in a wide range of tasks from personalized learning and analysis of users in social networks to medical analytics; it is the core of computational behavioural science. The Netherlands, Belgium, the UK and the USA are leaders in psychometric research. In Russia, psychometrics of online learning developed as a field of study only a few years ago.
The HSE Centre for Psychometrics in eLearning was established in June 2016 as part of the HSE eLearning Office. This is the first centre that specializes in psychometrics of e-learning in Russia.
‘Modern Digital Educational Environment in the Russian Federation’ is a priority project under the state programme ‘Development of Education’ for 2013-2020.
Evgenia Kulik
See also:
‘Up and Ahead’: Students in New Master's Programme to Study Psychometrics and Developmental Sciences
Enrolment is underway for the HSE Institute of Education’s new Master's programme, Science of Learning and Assessment, which was developed at the intersection of developmental science, advanced methods of neuroscience and psychometrics, and the theory and practice of testing and measurement. Students will learn to assess human development and adjust the learning process, relying on evidence-based approaches of neuroscience and current concepts of measuring skills, personality characteristics, competencies, and other complex constructs.
Russian Psychometrics at an International Level
Dmitry Abbakumov is the first researcher from a Russian university to present their work at the Psychometric Society conference, founded in 1935. He told the HSE new service about his study of online learning and why Russia has not played a major role on the international psychometrics scene – until now.
HSE Offers Psychometric Analysis of Online Courses
The HSE Centre for Psychometrics in eLearning is currently developing techniques, tools, and automated services for psychometric analysis of online courses as part of the project ‘Modern Digital Educational Environment in the Russian Federation’. In this respect, HSE was the winner of a competition held by the Russian Ministry of Education and will receive a grant for developing analytical approaches for e-learning.