The role of question characteristics in designing and evaluating survey questions

When developing questions for standardized measurement, question writers often focus on characteristics of the question (e.g. length, difficulty). Although we know much about the effects of some question characteristics on data quality, our field is still in the process of developing a comprehensive typology in which question characteristics are cataloged and organized, effects on interviewers’ and respondents’ processing are understood, and effects on data quality are documented. We provide an overview of some of the approaches used to conceptualize, measure, and code question characteristics. Our review leads us to (i) a conceptual model for exploring effects of question characteristics on interviewers’ and respondents’ cognitive processing and data quality and (ii) a typology of question characteristics. We present two case studies that compare how well summary measures of question characteristics - or “problems” associated with those characteristics - predict data quality. We conclude with an agenda for future studies of question characteristics.