Estimating stochastic survey response errors using the multitrait-multierror model

Response errors of different types, including acquiescence, social desirability, and random error, are well-known to be present in surveys simultaneously and to bias substantive results. Nevertheless, most methods developed to estimate and correct for such errors concentrate on a single error type at a time. Consequently, estimation of response errors is inefficient and their relative importance unknown. Furthermore, if multiple potential errors are not evaluated simultaneously, questionnaire pretests may give the wrong answer regarding the best question form. In this paper, we propose a new method to estimate and control for multiple types of errors concurrently, which we call the “multitrait-multierror” (MTME) approach. MTME combines the theory of experimental design with latent variable modeling to efficiently estimate response errors of different types simultaneously and evaluate which are most impactful on a given question. We demonstrate the usefulness of our method using six commonly asked questions on attitudes towards immigrants in a representative UK study. For these questions, method effect (11-point vs. 2-point scales) was one of the largest response errors, impacting both reliability as well as the size of social desirability.