Mixing modes of data collection in Swiss social surveys: Methodological report of the LIVES-FORS mixed mode experiment

Survey-based data collection makes a fundamental contribution to social science research in Switzerland, and survey data form the empirical base of much of the research into vulnerability across the life course being carried out by the Swiss National Centre for Competence in Research, ‘LIVES’. Because different features of the design of a survey can have implications for the quality of the data collected, optimising the survey design is key to ensuring the accuracy of the conclusions drawn from analyses of the data, and hence for the validity of both theoretical and policy developments derived from these. In this paper we present a methodological report of an experiment designed to provide evidence about which survey designs work best in the Swiss context, to maximise the quality of future quantitative research. The purpose of the study was to investigate the relation between survey design – in particular, the mode or modes of data collection used – and different sources of error affecting data quality, including coverage, nonresponse and measurement error. A key motivation for the research was the growing popularity of ‘mixed mode’ data collection, which offers a number of promising solutions to challenges facing traditional surveys (like under-coverage, declining response rates and rising costs), but carries the disadvantage of confounded measurement and selection errors that can compromise the comparability of the data collected in different modes. In this study, we compare single mode surveys (mail, CATI and web) and sequential mixed mode surveys (CATI plus mail, and web plus mail plus CATI/CAPI) with respect to response rates and the representativeness of the responding sample. The results lend support to the conclusion that mixing modes sequentially can help to increase response rates and improve sample representativeness, though differences are observed as a function of the availability of telephone numbers for sample members. Furthermore, these apparent benefits of mixing modes may not justify the additional burden such designs place on resources. Future research should, therefore, be directed at evaluating the cost-error trade-offs involved in different ways of combining modes from a total survey error perspective.