- A
- A
- A
- ABC
- ABC
- ABC
- А
- А
- А
- А
- А
'This Was a Major Task for All University Services'
July marked the end of the first exam session conducted at HSE University entirely in remote format. Specialists of the eLearning Office and Digital Services told HSE University Life how they prepared for it and which problems they encountered.
'We realized that holding exams in this format was a new experience for the vast majority of departments'
Head, Proctoring Centre
For the Proctoring Centre, the term began much earlier than June: as early as April we began preparing instructors to serve as proctors, coordinating the format in which exams would be held and conducting exams and tests that had been planned before the term.
Many departments had planned their exams and tests in advance, making it possible to conduct a ‘test-run’ to prepare students for the main term.
Of course, we realized that holding exams in this format was a new experience for the vast majority of departments, that our colleagues would have many concerns and doubts, and that most of their questions would be due to a lack of awareness.
To allay concerns, Proctoring Centre staff conducted a series of webinars for the employees of every HSE campus and for individual faculties (by request) in which we explained in detail how proctoring works and the specifics of exam procedure.
How HSE University went through the lockdown
Of course, HSE students were still the main participants in the process. Not only were detailed text- and video-based instructions developed for them, but also comments in Q&A format on the most pressing questions, visual materials, and separate instructional videos were created so that students could fully understand the main points of the process.
It was very important for us to respond quickly to requests from examinees. Many eLearning Office staff members were themselves HSE students and understood how emotionally charged the atmosphere was in organizing that academic term.
In cooperation with the country’s other leading universities, HSE University selected the Examus platform for exams and signed a contract with the company for proctoring services. At the same time, we were working in a time of great uncertainty, and because many questions arose concerning technical and organizational issues, we acted in concert with our colleagues from both Digital Services and Examus.
In June, HSE asked all students to participate in a test-run of proctored exams to check the computers and see how well we were adapting to the new system.
HSE students primarily took exams in written and oral formats rather than as multiple-choice tests, and the proctoring instructions were formulated accordingly. In addition to term-end exams, we developed new sets of methodologies for conducting the official State Final Examination online. To optimize the business process, each division assigned several faculty members to be responsible for interacting with the Proctoring Centre staff.
'We were able to train 204 people, 195 of whom served as proctors during exams'
Deputy Head, Centre for Computational Educational Sciences
Analyst, Centre for Computational Educational Sciences
How can psychometrics help you in a pinch? My colleagues and I were responsible for training proctors. In May and early June, we managed to train 204 people, 195 of whom served as proctors during exams.
We constantly updated the materials on the eLearning Office site and structured the questions and answers. Of course, our experience with ‘the science of psychometrics’ also proved especially useful.
Every day from May 30 to June 25, 2020, HSE University synchronous proctors received individual training. In addition, participants completed a theoretical section consisting of text and video instructions.
'We provided a business process that made it possible to verify exam results easily and impartially'
Head, Centre for Computational Educational Sciences
The Centre for Computational Educational Sciences and the Centre for the Development of Educational Technologies established a procedure for checking the written portion of English exams issued by the university.
We provided a business process that made it possible to verify exam results easily and impartially. The exams consisted of two parts: a written section that instructors checked and a multiple-choice section that a computer checked automatically.
We anonymized the written tests, sorted them automatically according to educational program and distributed them to teachers for checking. We then linked the results automatically to students’ multiple-choice test scores.
Our colleague, Yevgeny Glazunov, developed a special script enabling us to eliminate, as much as possible, the effects of human error in such things as filling out forms — a significant problem in large-scale exams taken under a tight time deadline.
We are currently conducting an in-depth psychometric analysis of the results. This will help our colleagues at the Department of Foreign Languages improve teaching and exam quality.
From June 10 to July 3, HSE University used the Examus system to conduct 495 exams on its remote testing platform. June 15-18 marked the peak load on the information systems used to conduct the proctored exams.
On those days, proctored exams were administered to almost 17,000 students, or approximately one-half of all students who took exams during the entire period.
'I am very grateful to all the teachers and managers who were with the students at those moments and who supported them'
Manager, eLearning Office
Because a much greater number of students than usual were taking exams at the same time and the procedures for the examinees took on greater importance, we changed our approach to the support process.
We reviewed and adjusted the rules for handling requests by focusing on helping as many students as possible as quickly as possible. We served as first responders for questions related to the Examus system.
We expanded the amount of space available on the Help Desk. Now we can store the entire correspondence history of incoming requests — including mass requests with a large number of screenshots — and respond to inquiries more quickly and fully.
Unfortunately, we were subjected to an intense online attack of critical services during the exam session. However, everyone did an outstanding job of coping with the situation. I am very grateful to everyone who poured so much effort into correcting the situation — the teachers and managers who were with the students at those moments and who supported them.
'We understood how much responsibility we had when we responded to students’ urgent hotline requests'
Manager, eLearning Office
Manager, Proctoring Centre
Manager, Proctoring Centre
Support services were provided in shifts over those three weeks, with dispatchers, technical specialists and eLearning Office managers working almost 24 hours at a stretch.
We understood how much responsibility we had when we responded to students’ urgent hotline requests. Not only was it important for us to be on call and to answer questions in a timely manner, but we also provided emotional support to our students and teachers.
Due to the significant increase in inquiries to the support service, we hired and trained additional staff to work in the new environment. Throughout the exam period, we created and regularly updated working instructions for automating processes. To help students, a special FAQ section was created on the exam platform.
We also responded to questions from numerous foreign applicants to the university. This helped us create a database of English-language responses so that we could further automate responses to frequently asked questions.
During the examination period, our colleagues in Digital Services significantly improved the system’s ability to generate reports on exam results.
'Preparations for conducting the exams remotely began back in March'
Head, Office for Digitalization of Education
Preparations began back in March, when it became clear that exams would be conducted online. To this end, we used Moodle system elements and proctoring by Examus.
The greatest challenge was clustering Moodle properly. We figured out the optimum configuration for servers, came up with an architecture and concluded that this was the best available option.
Another problem we faced concerned productivity, which is always a serious issue, especially when the load increases rapidly rather than gradually. Exams are just such an event, imposing a much greater load on the system over a short span of time.
Recognizing the seriousness of the situation, Digital Services worked to gain maximum results from the available resources and set up a balanced configuration that would support the necessary number of examinees. As a result, many HSE University services were placed in Yandex cloud storage. This achieved a 10-fold increase in corporate capacity and ensured uninterrupted and simultaneous access for up to 10,000 examinees.
Because we verified and properly tested all technical solutions in advance, the second helpline for IT issues received only 30 inquiries from the more than 40,000 examinees. Each inquiry received a response and each case was dealt with individually.
In addition to technological tasks, we quickly made functional improvements to the system at the request of the eLearning Office. In particular, we improved the reporting of exam results, thus enabling the Office to feel confident that the system was working properly during this difficult time.
'Over three weeks in June, we had to conduct three times as many online exams as we previously had in a whole year'
Director for eLearning, eLearning Office
In April and May, educational offices announced that more than 40,000 students would take their exams online.
That is a very large number. Translated into work tasks, this meant that over three weeks in June, we would have to conduct three times as many online exams as we previously had in a year. In other words, in the six weeks remaining before exams, we had to change the informational system and business processes to ensure their quality at an explosive scale.
No Russian university had ever done this. To get an idea of the scale of what happened, just consider the results: now, 89 Russian universities use the Examus proctoring system. Altogether, counting the regular exam session, entrance exams for foreign students and so on, HSE University conducted 85,500 hours of proctored exams in June.
This is twice as much as all other 88 universities using Examus combined.
This was a major task for all university services. First, the university is not used to responding to large-scale challenges. Second, it is an important task: online formats are gradually becoming common university practice, and the quality of education is directly linked to how reliably and accurately students’ work is assessed.
HSE University now has unique experience in organizing a large-scale online exam, and our next step will be to analyze its various aspects in detail.