When content is king: using topic models to analyze online innovation crowdsourcing

Over the past twenty years, a major development in firms’ innovation strategies has been the emergence of crowdsourcing as a tool to stimulate new ideas. A growing literature has examined the process by which firms select ideas from such discussions for further development. This paper focuses upon a surprisingly neglected factor: the substantive content of employee contributions. We explore the utility of topic modelling as a means, first, to construct themes as analytic objects; second, to describe the prevalence of themes in a discussion; and, third (and most important) as a platform for the development of measures suited to addressing a wide range of research questions. Drawing on data from a company-wide innovation dialogue at IBM, we illustrate this argument by highlighting four affordances of topic modelling: (1) identifying features of written contributions that predict their subsequent recognition; (2) identifying significant collective concerns and the degree of consensus around them; (3) examining the effects on selection of alignment of posts, respectively, with elite priorities and collective concerns; and (4) assessing whether a dialogue has produced deliberative learning. Including measures of post content changed conclusions about the impact of poster status on idea selection, and revealed considerable consensus on issue salience across organisational levels and geographic regions. Posts selected for recognition were aligned disproportionately with cues from organisation elites but also reflected collective preoccupations. Finally, extended deliberation appears to have improved discussion quality and, in some instances, led to productive reframing of the problems discussed.