SET Goals & Results Application
How does SET enhance teaching processes?
It specifies which teaching methods have been successful and which ones need to be improved. SET is a good way to test your own hypotheses when developing new teaching materials, constructing new formats for working with students and conducting teaching experiments overall.
It highlights topics and subject modules that appeal to students. Not only do students get to evaluate the novelty of the course as a whole, but they also comment on course content and its relation to the curriculum. Therefore, you can use this data to update syllabi and teaching materials, as well as communicate with colleagues who form pre- and post-requisites for your academic disciplines.
It helps to design assessment elements and adjust the optimum course complexity. SET criteria include evaluation of course difficulty, which can be used to understand the average student’s workload during the course. Student comments help to detail areas for the development of assessment elements. For instance, this may be increasing preparation time for some tasks or tightening the assessment requirements for various types of work.
It allows you to reflect on your teaching achievements over time. SET is a tool with a long history and it is also an opportunity to keep your finger on the pulse. Quantitative evaluation and commentary data help you to know that you are moving in the right direction, or highlight problem areas and points of growth if something suddenly goes wrong.
Should students be able to assess their teachers? How should this feedback be structured?
Yes, students should have an opportunity to evaluate teachers, as this can help them to improve the quality of the education process and enhance teaching practice. Feedback from students can provide important information on how teachers can develop their knowledge and skills, as well as the way they can influence students more effectively. Nevertheless, feedback should also be accessible, anonymous and honest so that students feel comfortable giving it.
How are SET results used?
1. In your independent work with student feedback
Analyzing grades, reading comments and considering students’ suggestions on course content or teaching approaches help give foundations for adjustments and, sometimes, drastic changes to a course when it is next implemented. Teachers usually receive evaluations within a couple of weeks after the end of the examination period.
Every year, I review and analyze SET scores; I look very closely at the detailed responses and student opinions. It is those comments, which are most useful for developing or reconstructing the syllabus. At the same time, I try to take into account the frequency of certain opinions. The more often certain feedback, comments or proposals are repeated, the higher its value for me and the more likely I will decide on changing something in my course. I look at single negative reviews seriously only if they contain at least some argued statements. If students simply write that they “did not like the lectures” and do not explain why, there is much less confidence in such reviews. In the end, at the University, we learn how to argue our position and justify our judgments, even if they are evaluative
Alexey Vdovin
Associate Professor at the School of Philological Studies (from the material for the HSE University publication “Okna Rosta")
Almost all reviews are helpful in one way or another. Some mention specific moments that have caught on or, conversely, have not, while others simply improve the mood. When you read that on Saturday morning at 9 am the students selflessly and happily come to class, you realize that here it is - the recognition! Well, when you think of a review with the word ‘Еsаulove’, you smile all the time. For me, SET is a good opportunity to get feedback from students and alumni. This has been especially important in the past two years. I had to teach in a constantly changing environment, so without feedback you don't always know how well you're doing.
Daniil Esaulov
Visiting Lecturer at the International College of Economics and Finance
(from the material; for the HSE University publication of «HSE University life»)
2. In various competitive procedures.
- Competition for Faculty Positions. SET scores are one of the key criteria for the HSE University Competition for Faculty Positions. The minimum annual average score for passing the Competition for Faculty Positions is three points for Academic and Practice-Oriented tracks. The score is significantly higher for teaching tracks and should be specified by the Faculty (usually within 4.3-4.7 points).
- Re-entry of teachers working under terms of affiliated employment. SET data is used in this procedure under similar rules to the Competition for Faculty Positions.
- Tutor evaluation procedures. SET results are mandatory for the tutor evaluation procedure. The same rules as in the Competition for Faculty Positions are applied here. More information on tutor assessments and the application of SET results can be found at this link.
3. When planning teaching load in schools/departments
SET results are taken into account when organizing teaching load distribution for the upcoming academic year. School administrators decide which courses will be assigned to teachers, as well as in which educational programmes and for which years and groups they will be taught.
SET is deeply integrated into the business processes and staffing policy of the HSE’s School of Foreign Languages. We have our own personnel committee, which develops criteria for evaluating teaching skills, such as class assessments. We develop our own additional criteria for the HSE University Competition for Faculty Positions and Tutor Effectiveness Monitoring, and we also regularly attend our instructors’ classes. In each of these processes, we primarily use SET data to document areas that require correction. Student comments often indicate possible problems in communication, content delivery, and testing. Although feedback does not always provide complete information about what is happening in a course, we have learned to apply survey results more in a more universal fashion.
Finally, SET is an important tool for learning how to provide feedback and conduct business communication overall. We always carefully discuss potential changes with students and respond to any constructive proposals. At the same time, we aim to show that difficulties are solved not only by replacing instructors, but also by providing methodological and communication support to teachers and groups. I believe that our approach teaches students to be constructive and specific
Ekaterina Kolesnikova
Head of the HSE School of Foreign Languages
We carefully work with SET data at all stages of staffing policy. After each SET wave, we further sort and visualize the data of evaluations and comments, help instructors work with these evaluations and offer advice on how to correctly interpret them. When our instructors receive results, we discuss possible changes and directions for development – and this option is open even for those who may have high grades.
Finally, SET data really helps us to not just solve current problems, but to grow and develop something new. We analyze evaluations and comments in detail to create new online courses and consider all students' proposals as to what topics need to be covered and which prominent teachers can oversee them.
Evgeny Sokolov
Head of HSE University’s Big Data and Information Retrieval School
In any case, the “three-point rule” is the ultimatum:
The three-point rule (SET):
A teacher is not allowed to teach a subject in the following academic year if their average score in SET for that course in the current year is less than three points.
However, there is a possibility of exemption from this rule if the result below three points was only assigned once. For this option, it is necessary to send two documents to the Electronic Document Management System, addressed to the First Vice Rector responsible for education process at HSE University:
- An explanatory note from the teacher describing possible reasons for the low scores.
- An official note from the academic programme leader and department head suggesting that the teacher's workload be maintained, but proposing measures that the department and academic programme can take to prevent any violations of this provision in the future.
The First Vice Rector will make a final decision based on the submitted documents.
If a result below three points is obtained twice (and an exemption has already been used once), the teacher must be replaced, while the possibility of exemption will not be provided.
4. When designing a curriculum
SET is used by academic supervisors when updating curricula for the next academic year. Using the results of course and teacher evaluations, along with student responses to open-ended questions about the organization of learning processes, academic supervisors can understand which courses can be still included in the programme's curriculum, in what order and in which year they will be taught, what the syllabi of these courses should consist of and how they should be interlinked.
When reviewing SET results, the educational programme leadership should assess the success of certain courses and determine what needs to be modified in their programmes.
Vadim Radaev
HSE University First Vice Rector (from the из interview on the HSE University portal)
SET provides valuable information for teachers and academic programme leaders when designing programmes. It is possible to rely on both comments and student ratings for a subject, while also seeing what results teachers overseeing the course have achieved. Of course, distortions may appear, but grades below three points should definitely be an alert. Moreover, if a course has ’distinction marks’, this is a good argument for including it in the programme
Head of the HSE Academic Council Commision on Full Degree Programmes, Academic Director of the “Pedagogy” Master’s Programme
5. Determining a list of best courses
SET is also a tool that encourages and assesses your progress. While teaching SET scores help you to analyze your development personally, course content that received the maximum SET scores for the previous academic year are available to all students on the course page. This helps to assess and demonstrate your progress as a teacher and a methodologist Urbi et Orbi.
SET solves several problems at once in the least painful way. The University’s administration receives data about both the quality of a particular course and student satisfaction with learning. Students have a feedback tool, which ensures a necessary level of loyalty and cooperation in the community while using it effectively. Teachers themselves get the opportunity to learn how specific elements of courses, that are of little interest to programme administration, but are nevertheless important for instructors, work. Both universal and anonymous, SET proves to be an excellent and indispensable tool for analyzing what is happening in the teaching process.
Andrey Starodubtsev
Andrey Starodubtsev, Associate Professor and ex-Dean of St. Petersburg School of Social Sciences and Area Studies (from his author's column for the HSE University publication “Okna Rosta” (Ru))
ChatGPT 3.5
AI Chatbot(from an interview with «Okna rosta»)