Do respondents using smartphones produce lower quality data? Evidence from the UK Understanding Society mixed-device survey

Maslovskaya, Olga and Smith, Peter and Durrant, Gabriele (2020) Do respondents using smartphones produce lower quality data? Evidence from the UK Understanding Society mixed-device survey. NCRM Working Paper. NCRM. (Unpublished)

[img] PDF
Available under License Creative Commons Attribution.

621kB

Abstract

We live in a digital age with high level of use of technologies. Surveys have started adopting technologies including smartphones for data collection. There is a move towards online data collection in the UK, including an ambition to collect 75% of household responses online in the UK 2021 Census. Major social household surveys in the UK have either transitioned to online data collection or are in the process of preparation for the transitioning. The Covid-19 pandemic forced rapid transitions to online data collection for many social surveys globally, with this mode of data collection being the only possibility at the moment. There are still concerns regarding allowing respondents to use smartphones to respond to surveys and not much is known about data quality produced by respondents using smartphones for survey completion in the UK context. This paper uses the first available in the UK, large scale mixed-device survey, Understanding Society Wave 8 where 40% of the sample were assigned to online mode of data collection. It allows comparison of data quality between different devices within the online mode of data collection with a special focus on smartphones. This analysis is very timely and fills the gap in knowledge. Descriptive analysis and then various regressions are used depending on the outcome variables to study data quality indicators associated with different devices in the online part of the survey. The following data quality indicators are assessed: break-off rates, item nonresponse, response style indicators, completion times, differential reporting indicators including self-reporting of risky behaviours, and consent to data linkage. Comparisons to limited results available in the UK are drawn. The results suggest that even in the context of non-optimised for smartphone questionnaire, we should not be concerned about respondents using smartphones for future social surveys, even for longer surveys such as the Understanding Society, as break off rates are very low and data quality between devices is not very different.

Item Type:Working Paper (NCRM Working Paper)
Subjects:1. Frameworks for Research and Research Designs > 1.6 Survey Research
1. Frameworks for Research and Research Designs > 1.7 Cross-Sectional Research
1. Frameworks for Research and Research Designs > 1.20 Secondary Analysis
1. Frameworks for Research and Research Designs > 1.24 Frameworks for Research and Research Designs (other)
2. Data Collection > 2.11 Online Data Collection
5. Quantitative Data Handling and Data Analysis > 5.5 Regression Methods
5. Quantitative Data Handling and Data Analysis > 5.5 Regression Methods > 5.5.1 Ordinary least squares (OLS)
5. Quantitative Data Handling and Data Analysis > 5.5 Regression Methods > 5.5.2 Generalized liner model (GLM)
ID Code:4322
Deposited By: NCRM users
Deposited On:04 Jun 2020 09:51
Last Modified:15 Jun 2020 11:20

Repository Staff Only: item control page