Do respondents using smartphones produce lower quality data? Evidence from the UK Understanding Society mixed-device survey

We live in a digital age with high level of use of technologies. Surveys have started adopting technologies including smartphones for data collection. There is a move towards online data collection in the UK, including an ambition to collect 75% of household responses online in the UK 2021 Census. Major social household surveys in the UK have either transitioned to online data collection or are in the process of preparation for the transitioning. The Covid-19 pandemic forced rapid transitions to online data collection for many social surveys globally, with this mode of data collection being the only possibility at the moment. There are still concerns regarding allowing respondents to use smartphones to respond to surveys and not much is known about data quality produced by respondents using smartphones for survey completion in the UK context. This paper uses the first available in the UK, large scale mixed-device survey, Understanding Society Wave 8 where 40% of the sample were assigned to online mode of data collection. It allows comparison of data quality between different devices within the online mode of data collection with a special focus on smartphones. This analysis is very timely and fills the gap in knowledge.

Descriptive analysis and then various regressions are used depending on the outcome variables to study data quality indicators associated with different devices in the online part of the survey. The following data quality indicators are assessed: break-off rates, item nonresponse, response style indicators, completion times, differential reporting indicators including self-reporting of risky behaviours, and consent to data linkage. Comparisons to limited results available in the UK are drawn. The results suggest that even in the context of non-optimised for smartphone questionnaire, we should not be concerned about respondents using smartphones for future social surveys, even for longer surveys such as the Understanding Society, as break off rates are very low and data quality between devices is not very different.