Stakeholder Involvement Approaches to Evaluation - AJE article

Stakeholder Involvement Approaches to Evaluation

A somewhat controversial American Journal of Evaluation "Letter to the Editor" is available online.   The article is titled, "Collaborative, Participatory, and Empowerment Evaluation:  Building a Strong Conceptual Foundation for Stakeholder Involvement Approaches to Evaluation."  The authors are: David Fetterman, Liliana Rodriguez-Campos, Abraham Wandersman, and Rita Goldfarb O'Sullivan.

The letter is controversial for three reasons:  1) it is a strong response to a colleague's opposing position; 2) it does not privilege one approach over another; and 3) it was powerful enough to elicit a response.

The letter highlights the differences between similar stakeholder involvement approaches to evaluation, focusing on the role of the evaluator.  The letter is designed to help evaluators select the most appropriate approach for the task at hand. It also helps evaluators establish realistic expectations for grantees, community members, and sponsors concerning the use of these approaches.  The online version provides a preview of the complete article.  The hard copy will be out in March, 2014 - Volume 35 Issue 1, pp. 142 - 146.

The preview of the first part of the article is provided below: 

     Defining, compartmentalizing, and differentiating among stakeholder involvement approaches to evaluation, such as collaborative, participatory, and empowerment evaluation, enhance conceptual clarity. It also informs practice, helping evaluators select the most appropriate approach for the task at hand. This view of science and practice is presented in response to the argument of Cousins, Whitmore,and Shulha (2013) that efforts to differentiate among approaches have been ‘‘unwarranted and ultimately unproductive’’ (p. 15).

     Over the past couple of decades, members of the American Evaluation Association’s (AEA) Collaborative, Participatory, and Empowerment Evaluation Topical Interest Group (CPE-TIG) have labored to build a strong theoretical and empirical foundation of stakeholder involvement approaches in evaluation. This includes identifying the essential features of collaborative, participatory, and empowerment evaluation. It also includes highlighting similarities and differences among these three major approaches to stakeholder involvement. Our primary disagreement with the article by Cousins et al. concerns the value and appropriateness of (1) differentiating among the stakeholder involvement approaches; (2) misleading characterization; (3) confounding and comingling terms, and (4) using collaborative inquiry as the umbrella term for stakeholder involvement approaches.

Differentiating Among the Stakeholder Involvement Approaches A long list of colleagues has recommended that evaluation approaches to stakeholder involvement be differentiated (Miller & Campbell, 2006; Patton, 1997, 2005; Scriven, 1997, 2005; Sechrest, 1997; Stufflebeam, 1994), and many have helped to define and identify similarities and differences among these approaches (Fetterman, 2001; Fetterman, Deitz, & Gesundheit, 2010; Fetterman, Kaftarian, & Wandersman, 1996; Fetterman & Wandersman, 2005, 2007; Rodrı´guez-Campos & Rincones-Go´mez, 2013; O’Sullivan, 2004; Shulha, 2010).

However, Cousins et al. ask, ‘‘Why is it important to have sharp distinctions among these approaches and to whose benefit?’’ (p.14) and conclude, ‘‘we find the investment in compartmentalizing genres of collaborative, participatory, and empowerment evaluation unwarranted and ultimately unproductive’’ (p. 15). Our view is that it is the nature of science and good practice to be precise, define terms, and explain differences among similar approaches in order to build on knowledge and improve practice. Differentiation of approaches helps evaluators select the most appropriate stakeholder involvement approach in the field. Hence, we advocate for distinguishing among approaches as follows:

Collaborative evaluators are in charge of the evaluation, but they create an ongoing engagement between evaluators and stakeholders, contributing to stronger evaluation designs, enhanced data collection and analysis, and results stakeholders understand and use. Collaborative evaluation covers the broadest scope of practice, ranging from an evaluator’s consultation with the client to full-scale collaboration with specific stakeholders in every stage of the evaluation (Rodrı´guez-Campos & O’Sullivan, 2010).

Participatory evaluators jointly share control of the evaluation. Participatory evaluations range from program staff members and participants participating in the evaluator’s agenda to participation in an evaluation that is jointly designed and implemented by the evaluator and program staff members. They encourage participants to become involved in defining the evaluation, developing instruments, collecting and analyzing data, and reporting and disseminating results (Shulha, 2010). Typically ‘‘control begins with the evaluator but is divested to program community members over time and with experience’’ (Cousins, Whitmore, & Shulha., 2013, p.14).

Empowerment evaluators view program staff members, program participants, and community members as in control of the evaluation. However, empowerment evaluators do not abdicate their responsibility and leave the community to conduct the evaluation solely by itself. They serve as critical friends or coaches to help keep the process on track, rigorous, responsive, and relevant. Empowerment evaluations are not conducted in a vacuum. They are conducted within the conventional constraints and requirements of any organization. However, participants determine how best to meet those external requirements and goals (Fetterman & Wandersman, 2010).

For the complete version of the article go online or pick up the hard copy of the journal in March.

Contact Dr. David Fetterman for additional information about the article and the response to this article in the American Journal of Evaluation.