Data Quality in PC and Mobile Web Surveys: Grids and Item-by-item Formats

With Internet proliferation researchers now have big opportunities ragarding online methods of data gathering yet with big challenges and even problems with data quality. One of these problems is connected with web-survey optimization for mobile devices. This research is focused on the influence of employment of various question sets' formats (grid or item-by-item) on data quality in web surveys on personal computers (PC) and smartphones. The research method is a two-wave web crossover-designed experiment with respondents being randomly assigned to one of the six experimental conditions, which differ by the format of question sets' presentation and by device needed to complete the survey. 1672 respondents completed the survey in the first wave 1101 of which also participated in the second wave of the experiment. Employment of item-by-item format led to higher predictive validity of separate scale questions and to lower level of straightlining response behavior compared with grid format. Moreover consistency of responses to question sets was higher when item-by-item format was employed both on PC and smartphones compared with grid format being used on both devices. However grid format was linked with less completion time of question sets and with higher level of inter-item correlations (Cronbach's alpha). Measurement equivalence test showed that different survey modes, question formats and survey designs had led to varied measurement of some latent variables. Thus it is advised to employ item-by-item format on PC and smartphones to obtain data with better quality. However I should emphasize that every web-based study needs individual survey design decisions and researchers must consider all possible effects on the data of the design they use as well as testing a survey beforehand.