• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site
For visually-impairedUser profile (HSE staff only)SearchMenu

Providing Comparability of Test Results for Scenario-Based Assessment

Student: Dariia Gracheva

Supervisor: Irina Uglanova

Faculty: Institute of Education

Educational Programme: Measurements in Psychology and Education (Master)

Year of Graduation: 2020

The purpose of the paper is to provide comparability evidence of scenario-based assessment. The use of scenario-based tasks for measuring complex constructs has become a modern direction of educational and psychological testing, which sets new practical and research tasks, including checking the comparability of SBTs forms. However, the methodology of comparability of SBTs is underdeveloped and there is a shortage of research answers this issue. In this study, we used test data from computerized SBT aimed at measuring critical thinking of primary school children. The research design was developed considering the age of target sample. The confirmatory factor analysis (CFA) was applied to examine the comparability of forms. In order to provide comparability evidence of SBT forms methodology of invariance testing with CFA was used. Configural and strong (equal factor loadings and thresholds) invariance with categorical variables was tested across task forms along with the mean comparison of all latent factors. The study employs CFA bi-factor model with two main factors related to components of critical thinking (analysis and conclusion) and two additional uncorrelated factors to explain the common context for indicators. Findings indicate that factor structure show a good fit for both task forms. Also, forms are invariant with respect to factor loadings and partly to thresholds. Thus, forms can be considered comparable on the main factors related to critical thinking. The context of the scenario probably affects the subjects' perception of the scenario and their observed behavior during the test, but not their critical thinking score. Based on the results of the paper, practical recommendations are offered to improve the instrument.

Student Theses at HSE must be completed in accordance with the University Rules and regulations specified by each educational programme.

Summaries of all theses must be published and made freely available on the HSE website.

The full text of a thesis can be published in open access on the HSE website only if the authoring student (copyright holder) agrees, or, if the thesis was written by a team of students, if all the co-authors (copyright holders) agree. After a thesis is published on the HSE website, it obtains the status of an online publication.

Student theses are objects of copyright and their use is subject to limitations in accordance with the Russian Federation’s law on intellectual property.

In the event that a thesis is quoted or otherwise used, reference to the author’s name and the source of quotation is required.

Search all student theses