The Problem of Missing Observations in Paired Comparisons Experiments

This talk discusses the problem of missing values in paired comparison experiments. We start by a discussion of the types of missingness encountered in paired comparisons (from missing by design through to missing responses because of unwillingness to divulge) and then relate this to the conventional categorisations used in the missing data literature. We then develop a model that can take account of both missingness and dependence across responses. The model is an extension to a log-linear model proposed by Dittrich, Hatzinger and Katzenbeisser (2002). An augmented frequency table is constructed in which indicators correspond to whether or not a decision is observed or not. A particular advantage of this approach is that various types of nonresponse mechanisms can also be included and tested as nested models. In addition, covariates can be included. Algorithmic approaches include direct maximisation of the likelihood, and the use of composite links (Thomson and Baker, 1981) The method is illustrated on an educational study carried out by one of the authors, which aimed to use a paired comparison approach to assess the relative importance of teaching styles.