An example of evaluation research as a cottage industry: The Technical Quality and Impact of Arts Audience Studies

Research resources are likely to be critical determinants of the technical quality of nongovernmental evaluation research, and quality in turn may affect the utility of the research. It is hypothesized that technical quality is a function of (1) the principal investigator's research background (prior research experience and formal training), (2) the study's financial resources, and (3) the study's institutional setting (the type of organization housing the study, the prior research experience of the housing organization, the profession of the investigator, and the ties between the organization housing the study and the organization that is the subject of the research). Further, it is hypothesized that technical quality has a significant bearing on the policy application of the completed research. These predictions are examined with data on eighty-six studies of museums and performing-arts attenders in the United States, acquired from the principal investigators and from study reports. Five of the research resource elements examined are found to have little impact on technical quality, but two have a major impact-the study's financial resources and the profession of the principal investigator. The research resource dimensions collectively explain 63% of the variance in study quality. Contrary to expectations, however, technical quality and utility are largely uncorrelated. The latter finding may be the result of the relative underdevelopment of applied research in the nonprofit private sector.