Empowering women or increasing response bias? Experimental evidence from Congo

Women's empowerment interventions are an important component of many development programs; surveys are a popular tool to measure how such programs affect gender-related attitudes. We consider how differences in gender response bias between the treatment and control groups complicate the use of attitudinal questions to evaluate women's empowerment interventions. We examine data from two experiments: a randomized development program aimed to empower women in 1,250 Congolese villages and a follow-up survey experiment. After demonstrating strong response biases in gender-related questions as a function of interviewer gender, we show that these biases vary as a function of program treatment status. These findings mean that the estimated average treatment effects of women's empowerment programs are sensitive to both the gender composition of the respondent sample as well as the interviewer pool. We conclude with lessons for survey implementation and pre-analysis plans.