Mixing modes of data collection in surveys: A methodological review

In this paper I provide a review of the literature related to mixing modes of data collection in surveys. The paper is structured in the following way: I begin with an overview of the range of mode choices available to survey researchers and their advantages and disadvantages with respect to a range of criteria, including their impact on data quality (so-called ‘mode effects’). Increasingly, survey designers are exploiting the potential offered by using a combination of data collection modes, either to offset the weaknesses of a particular mode with the strengths of another (Dillman, 2000) or, for example, to try to reduce the overall costs of fieldwork. The paper describes the major challenges involved in mixing modes, focusing in particular on the problem of mode effects on measurement error – or how people respond to survey questions. Modes of data collection appear to affect respondents’ answers by influencing either a) the amount of effort needed to answer the question; or b) the respondents’ willingness to answer questions honestly (Holbrook et al., 2003). The first type of influence can lead to a range of response errors referred to as ‘satisficing effects’ (Krosnick, 1991), while the second type of influence can lead to a tendency to give socially desirable answers. Mode effects have implications for the comparability of data collected in different modes; understanding their causes is, therefore, an important step in developing ways of reducing the negative impact of mixing modes on data quality. After describing some mechanisms through which modes result in measurement error, I discuss what, if anything, researchers can do to tackle the problem.