2026/2027
Разработка контрольно-измерительных материалов в психологии и образовании
Статус:
Маго-лего
Кто читает:
Департамент образовательных программ
Где читается:
Институт образования
Когда читается:
2, 3 модуль
Охват аудитории:
для своего кампуса
Преподаватели:
Тарасова Ксения Вадимовна
Язык:
английский
Кредиты:
6
Контактные часы:
58
Course Syllabus
Abstract
The course follows the complete assessment development process prior to psychometric validation. It begins with defining the purpose of an instrument and determining whether a new measure is needed or whether an existing one can be adopted or adapted. Students learn how theoretical constructs, psychological traits, competencies, and educational outcomes are defined through relevant literature and translated into observable indicators suitable for assessment. The course introduces key theoretical concepts in educational and psychological measurement, including construct representation, validity, reliability, fairness, and intended score interpretation. Students develop detailed test specifications that connect the construct, target population, content areas, task formats, cognitive demands, and scoring procedures. Evidence-Centered Design is introduced as a methodological framework for linking assessment claims, required evidence, and assessment tasks. The course covers a range of assessment formats, including selected-response items, constructed-response tasks, rating scales, performance tasks, situational judgment tests, and scenario-based or digital assessments. Emphasis is placed on selecting formats appropriate to the assessment purpose, construct, target population, and stakes. Particular attention is given to qualitative validation procedures conducted before large-scale data collection.
Learning Objectives
- The objective of this course is to develop the theoretical knowledge and practical competencies required to design, document, and qualitatively validate assessment instruments in psychology and education.
Expected Learning Outcomes
- Define and illustrate core concepts of test development and test lifecycle. Analyse an existing instrument by identifying its construct and arguing whether a new development is justified.
- Apply educational taxonomies and cognitive complexity frameworks to assessment design; Explain the basic principles of Evidence-Centered Design and use them to connect assessment claims, evidence, and tasks; Develop a preliminary assessment blueprint.
- Distinguish between major assessment formats, including selected-response, constructed-response, Likert-type, rating-scale, and forced-choice items; Select item formats appropriate for different constructs and assessment purposes; Apply established item-writing principles when developing assessment tasks Use AI tools to support item generation and revision while critically evaluating the quality and construct relevance of AI-generated materials.
- Critically review assessment items for construct relevance, clarity, technical quality, and alignment with specifications; Distinguish between major sources of validity evidence collected during assessment development; Study preliminary scoring procedures and revise assessment materials based on review evidence.
- Explain the role of qualitative evidence in supporting the validity of assessment interpretations; Conduct and interpret cognitive interviews or cognitive laboratory procedures; Identify problems in item wording, interpretation, response processes, and construct representation.
- Distinguish between holistic and analytic scoring rubrics and select an appropriate approach; Develop clear performance criteria and achievement-level descriptors; Distinguish between norm-referenced and criterion-referenced score interpretation.
- Explain the theoretical principles of work-based and competency-based assessment; Distinguish between major approaches to measuring competencies and work-related behavior; Explain the principles underlying the development and scoring of Situational Judgment Tests; Compare the strengths and limitations of SJTs, work samples, and assessment-center methods in relation to different assessment purposes.
- Describe major principles and stages of cross-cultural test adaptation; Identify linguistic, cultural, and contextual factors that may introduce construct-irrelevant variance or bias into adapted assessments; Explain how Differential Item Functioning can contribute to the identification of potentially biased items in adapted assessments.
- Explain the principles of scenario-based and technology-enhanced assessment; Select appropriate digital assessment formats for different learning objectives and constructs; Evaluate the advantages and limitations of technology-enhanced assessment.
- Apply professional and ethical principles to decisions made during assessment development; Identify potential sources of bias, unfairness, and underrepresentation in human- and AI-generated assessment materials; Evaluate the appropriate division of responsibility between human developers and AI systems; Report the use of AI in assessment development transparently.
Course Contents
- Introduction to Test Development
- Theoretical framework 1. General principles and operationalisation of latent constructs
- Theoretical framework 2: Taxonomies, Assessment Blueprint and Design Specifications
- Item and Scale Development in the age of AI
- Presenting Items: Validation and Approbation Plan
- Qualitative Methods in Test Development
- Performance Tasks and Rubrics
- Work-based evaluation: Situational Judgment Tests (SJTs) and Assessment-centers
- Cross-Cultural Test Adaptation and Assessment Equivalence
- Scenario-Based and Digital Assessment
- Ethical Issues in Assessment Development in the Age of AI
Assessment Elements
- Individual Test
- Group Project: Test Blueprint and Test Booklet for Psychological Construct
- Group Project: Educational Test Blueprint and Test Booklet
- Professional Review Activities
- Participation in SeminarsParticipation in Seminars reflects students’ regular and substantive engagement in practical course activities, including discussions, group work, item-writing exercises, SJT development, test adaptation, and other short in-class tasks. The assessment points: Item and Scale Development in the Age of AI, Work-Based Assessment: SJTs and Assessment Centers; Cross-Cultural Test Adaptation.
Interim Assessment
- 2026/2027 3rd module0.3 * Group Project: Educational Test Blueprint and Test Booklet + 0.05 * Professional Review Activities + 0.3 * Group Project: Test Blueprint and Test Booklet for Psychological Construct + 0.2 * Individual Test + 0.05 * Participation in Seminars
Bibliography
Recommended Core Bibliography
- Psychological assessment and theory : creating and using psychological tests, Kaplan, R. M., 2013
- Белая книга этики в сфере искусственного интеллекта : 42 ответа на популярные вопросы развития технологии ИИ, , 2024
- Введение в классическую и современную теорию тестов, Крокер, Л., 2012
- Крокер Л., Алгина Д., Звонников В.И. и др. - Введение в классическую и современную теорию тестов - 978-5-98704-437-5 - Логос - 2020 - https://znanium.ru/catalog/document?id=367349 - 367349 - ZNANIUM
Recommended Additional Bibliography
- Assessment methods in recruitment, selection & performance : a manager's guide to psychometric testing, interviews and assessment centres, Edenborough, R., 2005
- Psychological testing and assessment : an introduction to tests and measurement, Cohen, R. J., 2013