UX-Research Method in Sociology: Exploring Design Effects in Web Surveys

Online surveys and online experiments are becoming an important practice in conducting sociological research in Russia, however, the legitimacy of using this method of collecting data is still questionable due to its shortcomings associated with various measurement errors and unrepresentative sampling. A number of studies show that the design of an online survey can influence the perception and behavior of respondents, which can then be reflected in the data. Since conducting online surveys implies that respondents use a computer or mobile device to answer questions, research on ways to improve data quality should take place within the domain of human-computer interaction and address concepts such as user interface and usability. These concepts are widely used in UX research, which, however, have little relationship to the academic environment. This paper is an attempt to conduct a methodological study using one of the UX methods - usability testing - and show its practical application to study the impact of online survey design on data. In addition to usability testing, the study conducted a factorial experiment that assessed the impact of the format of the questionnaire with rating scales in the questionnaire, as well as the addition of a color image to the data. The results of the study showed that the presentation of a number of questions with rating scales in the form of a list in an online survey is preferable to a table. Despite the fact that the format as a whole does not affect the average ratings of convenience and complexity among all respondents, it appears to be significant for the group of people taking the survey via a mobile phone. The choice in favor of the list format is supported by data on a lower share of non-responses among those who take the survey via mobile devices and a decrease in the cognitive load associated with the lack of the need to recall the answer options for each judgment. There was no effect of adding an image to the questionnaire. The data obtained through usability testing is consistent with the results of the factorial experiment and at the same time allows you to look at the existing problems of the toolkit from a different angle - from the point of view of user experience. Thus, we can talk about the practical significance of the application of UX methods in methodological research.