Methodology of Measurеment in Psychology and Education

Master 2022/2023

Category 'Best Course for New Knowledge and Skills'

Type: Compulsory course (Science of Learning and Assessment)

Area of studies: Psychology

Delivered by: Institute of Education

Where: Institute of Education

When: 1 year, 1, 2 module

Mode of studies: offline

Open to: students of all HSE University campuses

Instructors: Irina Uglanova

Master’s programme: Science of Learning and Assessment

Language: English

ECTS credits: 6

Contact hours: 60

Full Syllabus

Abstract

The course provides students with the basic concepts of psychological and educational measurement: theories of measurement, test and item development, reliability and validity. The course starts with a historical review of the development of measurement in psychology and shows how the basic concepts changed over time. The most important issues of these concepts are discussed. The students will learn the main steps they need to go through in order to design a test or questionnaire and to ensure their psychometric quality. The guidance we use is aligned with the latest editions of the Educational measurement and the Standards of psychological measurement by APA and NCME. During the course the students will apply their theoretical knowledge to the development of their own questionnaire. The students will learn what such item chatecteristics as difficulty and discrimination are, how to calculate them, and how to deal with errors of measurement. We will dedicate a significant amount of time to the interpretation of test scores. The social consequences of standardized testing will also be discussed. The students have to be ready for extensive reading of texts on general methodological issues of measurement in social science as well as for group work and discussion.

Learning Objectives

Understand the theory of quantitative psychology and psychometrics Interpret fundamental concepts (e.g. measurement, validity, scales) and its' criticism Design, conduct, analyze, interpret and report Classical Test Theory analyses Understand fundamentals of Generalizability Theory Communicate research findings effectively Ensure social justice in testing and assessment

Expected Learning Outcomes

A student knows the history of the discipline and subfields
Student aware of advantages and limitations of quantitative psychology
Students can correctly apply key terminology of the field.
Students meet basic issues of quantitative psychometrics and learn how to avoid (if possible) the issues in their real practice.
Students master how to create operationalization of a latent construct
Students face the issue of "arbitrary metrics" and will learn how to avoid the problem in their practice.
Students can appropriately use basic validity terms.
Students will enable to plan their own validity studies (criterion validity, content validity)
Students face the controversial framework of the validity concept and will be able to build well-grounded validity desing
Students will be able to critically consider basic assumptions in psychological measurement.
Students master how to design, conduct, analyze, interpret and report Classical Test Theory analyses
Students will capable to interpret basics of CTT
Students aware of g-theory as one of basic psychometric approaches.
Students capable to apply g-theory in thier own research
Students communicate research findings effectively
Students ensure equity and social justice in testing and assessment
Students capable to design the feedback to the respondents in their own research

Course Contents

History of quantitative psychology. The role of quantitative psychology in human decision making
Psychological constructs and operational definitions
Operationalization Issues and Arbitrary Metrics
Validity concept (1). Criterion and content validity.
Validity concept (2) Construct validity. Modern approaches to validity
Scales of Measurement and the Criticism of the Concept.
Introduction to Classical Test Theory. Reliability estimates in CTT. Item analysis.
Introduction to Generalizability theory
Interpreting Test Results

Assessment Elements

Тест
Тест состоит из нескольких блоков. Включает вопросы с выбором одного или нескольких вариантов ответа, задания на сопоставление, задания с кратким ответом и кратким развернутым ответом.
Домашние задания
Групповая работа
Подготовка работ в мини-группах (2-3 человека) и презентация результатов. 1. Мини-мета-анализ по систематизации подходов валидизации определенного конструкта. 2. Проведение анализа и подготовка отчета в рамках КТТ.
Контрольные работы
1. Контрльная работа в формате эссе. Эссе с опорой на научную литературу по проблеме измерений в психологии и концепции валидности. 2. Контрльная работа в формате теста. По теме "Классическая теория тестирования".
Доклад
Доклад по заданной теме в мини-группах. Один доклад за 2 модуля. Тема доклада предполагает глубокий анализ одной или двух статей как базовых, и большее количество как дополнительных.

Interim Assessment

2022/2023 2nd module
0.15 * Доклад + 0.2 * Контрольные работы + 0.25 * Тест + 0.2 * Домашние задания + 0.2 * Групповая работа

Bibliography

Recommended Core Bibliography

Archuleta, K. (2009). Evidence of Construct-Related Validity for Assessment Centers: More Pieces of the Inferential Pie. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsndl&AN=edsndl.oai.union.ndltd.org.tamu.edu.oai.repository.tamu.edu.1969.1.ETD-TAMU-2009-05-579
Coulacoglou, C., & Saklofske, D. H. (2017). Psychometrics and Psychological Assessment : Principles and Applications. London: Academic Press. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=1169380
Epskamp, S., Maris, G. K. J., Waldorp, L. J., & Borsboom, D. (2016). Network Psychometrics.
Gary L. Canivez, Timothy R. Konold, Jason M. Collins, & Greg Wilson. (n.d.). Construct Validity of the Wechsler Abbreviated Scale of Intelligence and Wide Range Intelligence Test: Convergent and Structural Validity.
Helena Espirito Santo. (2017). Editorial: Validity in assessment instruments. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.D179CC42
Introduction to classical and modern test theory, Crocker, L., 2008
James, A., Algina, J., & Swaminathan, H. (2015). Psychometrics: Classical Test Theory. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.7E3F0482
Joel Michell. (2000). Normal science, pathological science and psychometrics. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.858D34B6
Kane, M. (2000). Current Concerns in Validity Theory. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED446094
Keith A. Markus, & Denny Borsboom. (2013). Frontiers of Test Validity Theory : Measurement, Causation, and Meaning. Routledge.
Kline, P. (2014). The New Psychometrics : Science, Psychology and Measurement. Hoboken: Routledge. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=687083
Leary, M., Kelly, K., Cottrell, C., & Schreindorfer, L. (2013). Construct Validity of the Need to Belong Scale: Mapping the Nomological Network. Journal of Personality Assessment, 95(6), 610–624. https://doi.org/10.1080/00223891.2013.819511
Measurement and evaluation in psychology and education, Thorndike, R. M., 2010
Nako, Z., & Barnard, A. (2012). Construct validity of competency dimensions in a leadership assessment and development centre. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.F9B61AE3
Prabhaker Mishra, C M Pandey, Uttam Singh, & Anshul Gupta. (2018). Scales of measurement and presentation of statistical data. Annals of Cardiac Anaesthesia, (4), 419. https://doi.org/10.4103/aca.ACA_131_18
Ronald K. Hambleton, & Russell W. Jones. (n.d.). ========================= = ITEMS. Instructional Topics in Educational Measurement ========================= An NCME Instructional Module on Comparison of Classical Test Theory and Item Response Theory and Their Applications to Test Development. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.46F2C9F0
Stephen N. Haynes, David C. S. Richard, & Edward S. Kubany. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.E05795A1
Wallis, T. (2004). Psychological tests are not really measuring what they claim to measure: A re-evaluation of the concept of construct validity. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsbas&AN=edsbas.F42F5769
Психологическое измерение : учеб. пособие для вузов, Тюменева, Ю. А., 2007
Психологическое тестирование, Анастази, А., 2002
Психологическое тестирование, Анастази, А., 2003
Психологическое тестирование, Анастази, А., 2007

Recommended Additional Bibliography

Measurement and evaluation in psychology and education, Thorndike, R. M., 2014
Rust, J., & Golombok, S. (2014). Modern Psychometrics : The Science of Psychological Assessment (Vol. 3rd ed). London: Routledge. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsebk&AN=812538

Course Syllabus