Patton's Book Review and Our Response is Out
Patton's Review of Our New Book is Out
(joined by our response to it)
Patton's review of Fetterman, Kaftarian, and Wandersman's new book is out in Evaluation and Program Planning. Jonathan Morell is the editor. We have provided a few highlights to give you the flavor of the discussion. It is followed by some of the dialogue that was generated in the AEA listserv EVALTALK by this book review and response.
Patton Book Review
Empowerment Evaluation: Knowledge and Tools for Self-Assessment, Evaluation Capacity Building, and Accountability, 2nd edition, D.M. Fetterman, S.J. Kaftarian, A. Wandersman (Eds.). Sage Publications (2015)
This edited volume has done the great service of clarifying, at least for me, what constitutes the core of empowerment evaluation (EE).
Stewart Donaldson, current (2015) president of the American Evaluation Association and one of evaluation’s most distinguished Thought Leaders (including recipient of the 2013 AEA Lazarsfeld Award for contributions to evaluation theory), has written the Foreword to the book and I find no reason to quarrel with his assessment of EE.
“This book marks the 21st anniversary of empowerment evaluation, an approach that has literally altered the landscape of evaluation. David M. Fetterman introduced the approach as part of his presidential address to the American Evaluation Association in 1993. Since that time, it has gone viral and is practiced vigorously throughout the United States and in more than 16 countries. Empowerment evaluation has been a leader in the development of stakeholder involvement approaches to evaluation, setting a high bar for quality and rigor. In addition, empowerment evaluation’s respect for community knowledge and commitment to the people’s right to build their own evaluation capacity has influenced the evaluation mainstream, particularly concerning evaluation capacity building. One of empowerment evaluation’s most significant contributions to the field has been to improving evaluation use and knowledge utilization. This book represents the culmination of decades of dialogue." (Donaldson)
In 2009, Stewart Donaldson, current (2015) president of the American Evaluation Association, organized and moderated a debate about the value of empowerment evaluation between David M. Fetterman, Michael Scriven, and me at the Claremont Colleges (Donaldson, Patton, Fetterman, & Scriven, 2010). In the debate I noted that 10 empowerment evaluation principles seemed like a lot to manage, so I asked David Fetterman, which of the 10 principles are actually critical?
The fidelity issue focuses on the extent to which a specific evaluation sufficiently incorporates the core characteristics of the overall approach to justify labeling that evaluation by its designated name.
Perhaps I should add that the issue of evaluation approach fidelity is much on my mind since I have been facing the issue with both utilization-focused evaluation and developmental evaluation…I recently identified nine essential elements of developmental evaluation
So, after 21 years and global recognition, this is what the lead EE conceptualizers have to say about empowerment. ‘‘People empower themselves.’’ I understand that this is the politically correct thing to say. I also understand that such a sentiment could be experienced as insulting and offensive with connotations of blame the victim. However experienced, it is such a simple-minded framing of the complexities of empowerment, that I found myself embarrassed for the editors that they would conclude such an important book with such a trite, glib, and insubstantial observation.
…there is much of value here, strong evidence of an approach that, well facilitated and comprehensively engaged, can make a substantial difference. I will look forward to a third edition that further clarifies and elaborates what it means to be well-facilitated and comprehensively engaged, and perhaps even brings some sophistication to the issue of empowerment itself.
Empowerment evaluation is a systematic way of thinking: A response to Michael Patton
- David M. Fettermana, , , Abraham Wandersmanb, 1, , Shakeh Kaftarianc, 2,
The essence of EE is a systematic way of thinking, not a single principle, concept, or method. Empowerment evaluation, first and foremost, helps people evaluate their own programs and initiatives. It is the use of evaluation concepts,techniques, and findings to foster improvement and self-determination
It is an evaluation approach that aims to increase the likelihood that programs will achieve results by increasing the capacity of program stakeholders, to plan, implement, and evaluate their own programs (Fetterman & Wandersman, 2007).
We presented the theories, concepts, principles, and steps guiding empowerment evaluation. However, his (Patton's) focus was almost exclusively on the principles.
It is the gestalt or whole package that makes it work. Empowerment evaluation theory, concepts, principles, and steps are used to guide practice. Patton’s critique is off-target because it focuses on individual parts or principles, failing to recognize that empowerment evaluation is more than the sum of its parts (including ‘‘essential’’ parts).
We explained how these principles work together synergistically. For example, the first principle, improvement, reflects the pragmatic and utilitarian nature of empowerment evaluation. The aim is to help people improve their programs and practice and succeed in accomplishing their objectives. Community ownership is required to make this happen in a meaningful and sustained manner. This is linked to process use. The more people take ownership of the evaluation, the more committed they are to using the evaluation findings and follow through on the recommendations. Authentic community ownership requires inclusion. It cannot be a single elite group making all the decisions….The same type of synergy and interconnectivity applies to the remaining combination of principles.
11. "People empower themselves"
Patton rescued this gem from the obscurity of our appendix. It speaks to the heart of empowerment evaluation. It brings us full circle in our response to his review. We began by explaining how people are in charge of their evaluation in an empowerment evaluation. It is appropriate to conclude with this reminder. We made the principles explicit because we found too many empowerment evaluations were in name only (see Miller & Campbell, 2006; Fetterman & Wandersman, 2007). Similarly, we think it is appropriate to remind colleagues that empowerment evaluation places control in hands of staff and community members. This simple but fundamental feature is what most clearly defines empowerment. For some, this may be cryptic, embarrassing, PC, or even pabulum. For us it succinctly captures the essence of self-determination verses dependency, empowerment verses disempowerment.
We appreciate Patton’s placement of empowerment evaluation in the ‘‘pantheon of major approaches’’ (2015, p. x). We also appreciate his invitation to continue this engaging and productive dialog. It is our hope that this response illuminates and contributes to the evolving dialog about the science and practice of empowerment evaluation
Highlights of EVALTALK Discussion
_/_/_/_/
|
May 26 (1 day ago)
 | | ||
While not its only virtue, it is worth commenting on the fact that
empowerment evaluation and its leaders have always set the standard for
that Great Commandment of ethical professional evaluation: proactively seek
and interactively respect and respond to critical commentary.
Michael Scriven
empowerment evaluation and its leaders have always set the standard for
that Great Commandment of ethical professional evaluation: proactively seek
and interactively respect and respond to critical commentary.
Michael Scriven
|
12:25 AM (14 hours ago)
| | ||
DF: Hi Michael
On Tue, May 26, 2015 at 11:22 AM, Michael Scriven <mjscriv1@gmail.com> wrote:
Fetterman, D.M., Rodriguez-Campos, L., Wandersman, A., and O'Sullivan, R.G. (2015). Collaborative, Participatory, and Empowerment Evaluation Building a Strong Conceptual Foundation for Stakeholder Involvement Approaches to Evaluation (A Response to Cousins, Whitmore, and Shulha, 2013). American Journal of Evaluation March 2014 vol. 35 no. 1 144-148
While not its only virtue, it is worth commenting on the fact that
empowerment evaluation and its leaders have always set the standard for
that Great Commandment of ethical professional evaluation: proactively seek
and interactively respect and respond to critical commentary.
DF: Many thanks.
We certainly try to invite and engage the issues in a meaningful and authentic manner. We value and appreciate our critical friends like you Michael. We continue to invite our colleagues to engage in critical reflection and dialogue in an effort to strengthen the quality of our work.
For those new to our debates, please see a sample of our exchanges:
1. Video Debate:
Scriven, Patton, Fetterman Debate:
2. Distinguishing Between Collaborative, Participatory, and Empowerment Evaluation (stakeholder involvement approaches to evaluation)
http://aje.sagepub.com/
3. Invited Cousins to Serve as a Critical Friend - Writing a Chapter in One of Our Books
Cousins, J.B. (2005). Will the real empowerment evaluation please stand up? A critical friend perspective. In D.M. Fetterman & A. Wandersman (Eds.), Empowerment Evaluation Principles in Practice (pp. 183-208). New York: Guilford.
4. Summary of Empowerment Evaluation Issues and Debates
Miller, R.L. and Campbell, R. (2006). Taking stock of empowerment evaluation: An empirical review. American Journal of Evaluation, 27(9), 296-319.
Smith, N.L. (2007). Empowerment Evaluation as Evaluation Ideology. American Journal of Evaluation, 28(2), 169-178.
Fetterman, D. and Wandersman, A. (2007). Empowerment evaluation: Yesterday, today, and tomorrow. American Journal of Evaluation, 28(2), 179-198.
5. Review of Empowerment Evaluation Book (2005) - Empowerment Evaluation Principles in Practice
Fetterman, D.M. and Wandersman, A. (2005). Empowerment Evaluation Principles in Practice. New York: Guilford Publications.
Patton, M. (2005). Toward distinguishing empowerment evaluation and placing it in a larger context: Take two. American Journal of Evaluation, 26, 408-414.
Scriven, M. (2005). Review of the book: Empowerment Evaluation Principles in Practice, 26(3), 415-417.
Fetterman, D.M. (2005). In Response to Drs. Patton and Scriven. American Journal of Evaluation, Vol. 26 No. 3, September 2005 418-420
http://www.stes-apes.med.ulg.
6. Review of Empowerment Evaluation Book (1996) Empowerment Evaluation: Knowledge and Tools for Self-Assessment & Accountability
Empowerment Evaluation: Knowledge and Tools for Self-Assessment & Accountability, edited by David M. Fetterman, Shakeh J. Kaftarian, and Abraham Wandersman. Thousand Oaks, CA: Sage Publications, 1996
Patton, M. (1997). Toward Distinguishing Empowerment Evaluation and Placing It In A Larger Context. American Journal of Evaluation 1997; 18; 147
Scriven, M. (1997). Empowerment Evaluation Examined. American Journal of Evaluation 1997; 18; 165
Fetterman, D.M. (1997). Empowerment Evaluation: A Response to Patton and Scriven. American Journal of Evaluation 1997; 18; 253
7. Additional Early Book Review Comments (highlighting the tone and substance of the early debates)
Altman: In the concluding chapter, Fetterman writes: "I believe that evaluation is basic -- like reading, writing, and arithmetic. I believe that evaluation should be a fundamental skill, an integral part of any educated citizen's repertoire. I also believe that anyone can learn the basic skills of evaluation...We need every tool we can find to respond to the pressing social and environmental problems we face (p. 383)". To this I say, "right on!" "Let's get to work."
Brown: ...the purpose of empowerment evaluation is not simply to empirically estimate a program's worth but to develop skills needed for ongoing self-assessment, so that evaluation itself is institutionalized and made sustainable at the program level. As such, evaluation becomes a means of achieving organizational strength and renewal, effectively turning the classic definition of evaluation--an object of interest is compared against a standard of acceptability--on its head.
Wild: Fetterman et al. have nailed their theses to the door of the cathedral. Now the question is, How tolerant is the establishment of dissent? (Wild, p. 172.)
_/_/_/_/
|
May 26 (1 day ago)
| | ||
In contexts characterized by inequality and exclusion empowerment evaluation is the front end of evaluation for an equitable society.
|
1:24 AM (13 hours ago)
| | ||
DF: Hi Bob
On Tue, May 26, 2015 at 12:54 PM, Robert Picciotto <r.picciotto@btinternet.com> wrote:
In contexts characterized by inequality and exclusion empowerment evaluation is the front end of evaluation for an equitable society.
DF: You are very kind.
For colleagues interested in learning more about how empowerment evaluation is being used to facilitate this kind of change see examples in our most recent empowerment evaluation book:
Fetterman, D.M., Kaftarian, S., and Wandersman, A. (2015). Empowerment Evaluation: Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability. Thousand Oaks, CA: Sage. A few examples are provided below:
Chapter 5 – Peruvian women using EE to refine their crafts to ensure that they are more marketable in the international market; using EE to facilitate the sale of their crafts on the Internet (to expand their market and by pass the middle-person who was devouring most of their profits (Sastre-Merino et al)
Chapter 6 – Visible Learning Model teachers/educators serving as evaluators of the impact of their work (increasing the probability that they will have a sustained impact) (Clinton and Hattie)
Chapter 7 - Community members using empowerment evaluation to bridge the digital divide in communities of color (Hewlett-Packard's $15 Million Digital Villages) (Fetterman)
The Tribal Digital Village, operating across tribal lines, used EE to help construct the largest unlicensed wireless systems in the US and a digital printing press - providing a sustainable alternative to gaming (Fetterman, 2015a,b,c).
Chapter 11 – Fourth and fifth graders using EE to make their school more inviting and accepting and reflective of the school culture and values (Langhout & Ferna´ndez, 2015).
Chapter 12 – Staff members and program participants mainstreaming self-assessment throughout their community-based organizations (Suarez-Balcazar et al., 2015).
Chapter 14 – minority staff and community members pooling data across non-profits to demonstrate effectiveness in reducing tobacco consumption and saving the State millions in excess medical costs (Fetterman et al., 2015).
A more detailed insight into the use of EE to help bridge the digital divide in communities of color is provided in:
Fetterman, D.M. (2013). Empowerment Evaluation in the Digital Villages: Hewlett-Packard's $15 Million Race Toward Social Justice. Stanford: Stanford University Press.
Abbreviated descriptions are provided in Stanford's Social Innovation Review:
In addition, EE was used to transform Stanford University's medical education curriculum. See Fetterman, D.M., Deitz, J., and Gesundheit, N. (2010). Empowerment Evaluation: A Collaborative Approach to Evaluating and Transforming a Medical School Curriculum. Academic Medicine, 85(5):813-820.
This effort was important because it set the tone for another EE that is dramatically transforming medical education and making a contribution to improving equity.
Our medical education research and evaluation team has documented the absence of LGBT-related content in medical schools. The median reported time dedicated to LGBT-related content in medical schools throughout the United States and Canada was 5 hours. We published our findings in JAMA. See:
Obedin-Maliver, J., Goldsmith, E., Stewart, L., White, W., Tran, E., Brenman, S., Wells, M., Fetterman, D.M., Garcia, G., and Lunn, M. (2011). Lesbian, Gay, Bisexual, and Transgender - Related Content in Undergraduate Medical Education. JAMA, September 7, 20111 - Vol. 306, No. 9: 971-977
This glaring omission in medical curriculum has received national and international media attention. Our EE-related work is helping medical educators respond to this omission. Inequities associated with LGBT representation and treatment represents one of the most pressing human rights issues of our time. It is our hope that our work will contribute to greater equity in our society.
This is only a small sample of our work which also extends to communities confronting drug abuse, sexual violence, and poverty. The work described in brief in this posting is in alignment with EE's 10 guiding principles, in this case focusing on social justice. Additional example are available upon request.
_/_/_/_/
|
4:19 PM (22 hours ago)
| | ||
I find the issue of evaluation approach fidelity interesting here.
Are not 'Utilization-Focused Empowerment Evaluation' and 'Developmental
Empowerment Evaluation' also viable approaches (perhaps even 'hi-fi', given
their specificity)?
I like Wikipedia for semantic fodder here. The reference for fidelity as it
applies to program evaluation dryly states that the term denotes how
closely a set of procedures were implemented as they were supposed to have
been. This corresponds with the high fidelity (hi-fi) concept under the
'Audio' section of the entry which is all about 'accurate reproduction'.
Here is what I find interesting in the entry: "The converse term 'lo-fi',
does not necessarily mean 'low fidelity', rather that the production ethic
aims for 'gritty authenticity' over perfect production."
I feel like this concept of 'gritty authenticity' may be helpful to throw
into the mix here . . . maybe not.
Thanks for the interesting debate!
Are not 'Utilization-Focused Empowerment Evaluation' and 'Developmental
Empowerment Evaluation' also viable approaches (perhaps even 'hi-fi', given
their specificity)?
I like Wikipedia for semantic fodder here. The reference for fidelity as it
applies to program evaluation dryly states that the term denotes how
closely a set of procedures were implemented as they were supposed to have
been. This corresponds with the high fidelity (hi-fi) concept under the
'Audio' section of the entry which is all about 'accurate reproduction'.
Here is what I find interesting in the entry: "The converse term 'lo-fi',
does not necessarily mean 'low fidelity', rather that the production ethic
aims for 'gritty authenticity' over perfect production."
I feel like this concept of 'gritty authenticity' may be helpful to throw
into the mix here . . . maybe not.
Thanks for the interesting debate!
|
10:13 AM (5 hours ago)
| | ||
DF Hi Nathan
On Tue, May 26, 2015 at 4:19 PM, Nathan Horst <nathan.horst@gmail.com> wrote:
I find the issue of evaluation approach fidelity interesting here.
Are not 'Utilization-Focused Empowerment Evaluation' and 'Developmental
Empowerment Evaluation' also viable approaches (perhaps even 'hi-fi', given
their specificity)?
DF: Theoretically, there is no reason to object to this kind of combination or synthesis.
However, I would recommend that the evaluator have a good handle on how to use them separately before fusing and applying them together in practice.
Empowerment evaluation is strongly influenced by utilization-focused evaluation, particularly with its emphasis on process use (to increase the probability of use). In addition, since evaluation has multiple purposes (such as development, accountability, and knowledge, as Chelimsky highlights), the overlap between developmental evaluation and empowerment evaluation is particularly apparent when the purpose of evaluation is development.
I think the overlap between empowerment evaluation and developmental evaluation is most notable when describing the role of the empowerment evaluator or critical friend.
Empowerment evaluators are critical friends. They do not play the role of an outsider or expert removed from the initiative. Instead they serve as a trusted colleague who can coach stakeholders as they engage in self-evaluation, strategic planning, and program implementation. They help set the tone for discussions about evaluation results, and findings, modeling and informally articulating the philosophy behind the empowerment evaluation approach. Specifically, the critical friend should help to establish a positive learning climate in which the views of all stakeholders are respected, input from all parties is solicited, and conversation is guided so as to encourage comments that are constructive and improvement oriented.
Critical friends model a communication style that is inviting, unassuming, nonjudgmental, and supportive, so that stakeholders will feel comfortable in speaking openly about issues and concerns. Critical friends should also help to remind members of the group about what they have in common, including shared goals and institutional values. They are also responsible for asking some of the hard questions like: "what do you mean by this", "why did you give this rating (what's the evidence)", and "did you do it"?
In essence, they help to facilitate dialogue and assessment stages of an evaluation.
I like Wikipedia for semantic fodder here. The reference for fidelity as it
applies to program evaluation dryly states that the term denotes how
closely a set of procedures were implemented as they were supposed to have
been. This corresponds with the high fidelity (hi-fi) concept under the
'Audio' section of the entry which is all about 'accurate reproduction'.
Here is what I find interesting in the entry: "The converse term 'lo-fi',
does not necessarily mean 'low fidelity', rather that the production ethic
aims for 'gritty authenticity' over perfect production."
I feel like this concept of 'gritty authenticity' may be helpful to throw
into the mix here . . . maybe not.
DF: This is useful - thanks. If I might build on your thoughts I would say that the concern I had in the review was that it was focused on technical perfect production or a replication of critical principles guiding empowerment evaluation, as compared with the gritty authenticity that is produced from the gestalt or whole package of empowerment evaluation theory, concepts, principles, and steps that make it work.
We wrote our earlier book, Empowerment Evaluation Principles in Practice, to enhance fidelity, particularly as it relates to the spirit, not the letter, of the law. It made many of our guiding principles explicit instead of implicit.
It was written to help our colleagues reproduce the same sounds we could hear in the process of conducting an empowerment evaluation, with minimal amounts of noise and distortion.
Our latest book, Empowerment Evaluation: Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability, emphasized the importance of empowerment evaluation's entire orchestra, including theories, concepts, principles, and steps required to faithfully reproduce the leitmotif or refrain in our heads. We think this is a more meaningful and useful form of fidelity.
I hope this brief posting is responsive to your questions and comments. Thanks for the metaphor to advance the conversation.
Thanks for the interesting debate!
Df: Thank you and best wishes.
-David
_/_/_/_/
|
6:25 AM (8 hours ago)
| | ||
David,
A hearty congratulations to you and your colleagues on your new book. Both the book and your response to critiques of it (as an example of how to be civil yet forthright in asserting one's viewpoint) will be must reads for doctoral courses on research methods that I teach.
David
A hearty congratulations to you and your colleagues on your new book. Both the book and your response to critiques of it (as an example of how to be civil yet forthright in asserting one's viewpoint) will be must reads for doctoral courses on research methods that I teach.
David
|
10:37 AM (4 hours ago)
| | ||
DF: Hi David
On Wed, May 27, 2015 at 6:25 AM, Dubois, David <dldubois@uic.edu> wrote:
David,
A hearty congratulations to you and your colleagues on your new book.
DF: Thanks. It marks the 21st anniversary of empowerment evaluation so it is quite a celebration for all of us.
In addition, this book is a paradox much like Plutarch's ship. Plutarch raised the question of whether a ship, once restored by replacing every piece, is still the same ship. This collection began as a "simple" revision of the book that helped launch the empowerment evaluation approach - Empowerment Evaluation: Knowledge and Tools for Self-Assessment and Accountability. However, in the process of creating a "simple" revision and update, we have replaced every single chapter. In addition, we have added theories, concepts, principles, steps, and tools that did not explicitly exist when the approach was first launched. The revision is so radical that even the title of the book as been changed to explicitly include the term evaluation capacity building. Part of the paradox is that many of the principles, for example, were implicit. Capacity building was always a fundamental part of the approach.
Nevertheless, we believe this collection represents a transformation, literally decades beyond our first voyage. Ultimately, we leave it to the reader to determine if this is the same ship with new sails or an entirely new vessel as they chart their own journey across this book's ocean of discourse, insight, and experience.
Both the book and your response to critiques of it (as an example of how to be civil yet forthright in asserting one's viewpoint) will be must reads for doctoral courses on research
methods that I teach.
DF: You are most kind. I try to convey the importance of your words to my Stanford students, as well as students I work with throughout the world. Ideally, this approach will produce more light than heat in our intellectual exchanges.
Best wishes.
- David
_/_/_/_/
|
9:03 AM (6 hours ago)
| | ||
"People empower themselves."
Wasn't this the radical message conveyed in the Declaration of American Independence?
"We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness."
What do these potent words mean to you? More importantly, how do you incorporate them into your practice?
Best,
Chad
DF: Hi Chad
Wasn't this the radical message conveyed in the Declaration of American Independence?
"We hold these truths to be self-evident, that all men are created equal, that they are endowed by their Creator with certain unalienable Rights, that among these are Life, Liberty and the pursuit of Happiness."
What do these potent words mean to you? More importantly, how do you incorporate them into your practice?
Best,
Chad
DF: Hi Chad
My mom on the east coast, who I call every morning, and another close friend and colleague, equally as far away, both really resonated with your posting.
My mom used to start every class and workshop with "what do you want to get out of this?" The purpose of her question was to tailor her words to their needs, to cultivate ownership, to help them empower themselves.
In empowerment evaluation, we do the same thing by placing the practice of evaluation in their hands with our guidance as coaches and critical friends.
We agree with Christens, Peterson, and Speer (2014) that ‘‘Empowerment evaluation is embedded in authentic and meaningful engagement processes, interpersonal relationships, diverse perspectives, and socio-political, cultural, institutional, and economic arrangements’’. However, it is nothing if it does not cultivate an environment conducive to helping people empower themselves.
Thanks for sharing your insights and I hope this brief response lets you know that your posting, and the posting of our other colleagues. on this thread has generated a tremendous amount of discussion and commentary in the community, (like ripples in a pond extending much further than this stream of thought on EVALTALK).
Best wishes
- David
Semi-Concluding Comments: Thanks
|
10:26 AM (1 minute ago)
| | ||
Hi
First, many thanks for the wonderful exchange on EVALTALK yesterday concerning Patton's review of our new empowerment evaluation book and our response. It was very much appreciated. (Particular thanks are extended to Michael Scriven, Robert Picciotto, Nathan Horst, Mohammad Hasan Mohaqua Moein, David Dubois, Chad Green, and of course Michael Patton for writing the review.)
Second, thanks to all of you who have emailed, called, and Skyped me to discuss the multi-level set of issues associated with this exchange and approach. The outpouring has been overwhelming (and thus this broadcast approach to thanking everyone).
However, I want to extend my appreciation to Lois-ellen Datta and Liliana Rodriguez-Campos in particular for sharing their thought provoking and insightful comments. Much of their own work complements ours.
For all of you who have asked for assistance tracking down all of the EVALTALK conversations, you can find them on the empowerment evaluation blog. The blog is at:
We will, of course, take into consideration all that has been shared with us as we continue to refine and improve our practice.
Best wishes.
_/_/_/_/
One More Philosophical Observation/Comment
|
3:59 PM (0 minutes ago)
| | ||
DF; Hi Stan and Bill
On Mon, Jun 1, 2015 at 12:41 AM, Bill Fear <williamjamespsych@gmail.com> wrote:
RE:
>>>>
> As for me empowerment evaluation has been helpful.
DF: Thanks Stan. You have been a "power" user of empowerment evaluation with spectacular results with and for the folks you work with.
> It was one of the tools that we used to ultimately create a management
> training program that was a recipient of the Council on Accreditation
> Innovative Practice Award back in 2012. The program is currently training
> its 5th cohort.
DF: For those who want to know more about Stan's work in this area take a look at our empowerment evaluation blog at:
>
> Again there are different evaluation models and contextually if it helps
> in identifying key issues to improve program performance in my mind it has
> value.
DF: For additional clarity about similarities and differences concerning stakeholder involvement approaches to evaluation, comparing collaborative, participatory, and empowerment evaluation, take a look
at our recent AJE article on the topic. Here is the citation:
Fetterman, D.M., Rodriguez-Campos, L., Wandersman, A., and O'Sullivan, R.G. (2014). Collaborative, Participatory, and Empowerment Evaluation: Building a Strong Conceptual Foundation for Stakeholder Involvement Approaches to Evaluation (A Response to Cousins, Whitmore, and Shulah, 2013). American Journal of Evaluation,35(1):144-1148. (Also see Cousins et al's response in the same issue.)
>
>>>>
As I said, my commentary:
does not, in any way whatsoever, devalue the efforts of those involved,
> does not discredit the institution, and does not take anything away from
> anybody....The real work is done between the breach of the canonical
> narrative and the reconstitution of the canonical narrative...we
> should recognise the importance of the narrative as a whole over time.
DF: Thanks for appreciating that early developmental stage of empowerment evaluation's introduction to the field. For those of us who lived it, it was an exciting, tumultuous, and enlightening time. Some colleagues embraced it and understood the shift and utility of the approach almost immediately. At the same time, some said they felt threatened by it or at least thought it might undermine or threaten the guild, since it was giving our evaluation tools away. Much of it was recorded in the literature. Many of the citations can be found on the empowerment evaluation web site at:
Colleagues were asking fundamental questions, including: What is evaluation? Who am I (as an evaluator)? What is my role (in an evaluation and in society as an evaluator)? I concluded one of my earlier books with this observation about that period of time: it "was an echo of Prospero's blessing to Miranda in The Tempest, this experience brought both calm seas and auspicious gales. I am (and continue to be) appreciative of my colleagues, both for and against this approach, who have taken the time to engage in this important dialogue."
This was as you note Bill and important contribution in its own right - it just didn't stop there.
Quite clearly any tool that can be used to the desired purpose is a
valuable tool in that context. I did not say any different to that. I am
sure there are many people around the world who have found EE a useful
tool. That acknowledgement was inherent in my discussion as noted above.
Df: Thanks Bill. Yes - I agree EE has been found useful on a global scale in a relatively short period of time. The long list of examples in over 16 countries can be found in our blogs, articles, radio interviews, and books:
Blogs
Stanford Social Innovations Review
Chronicle of Philanthropy
Empowerment evaluation
Empowerment evaluation (search Wikipedia)
Articles
Fetterman, D.M. and Wandersman, A. (2007). Empowerment evaluation: yesterday, today, and tomorrow. American Journal of Evaluation, 28(2):179-198.
Fetterman, D.M., Deitz, J., and Gesundheit, N. (2010). Empowerment evaluation: a collaborative approach to evaluating and transforming a medical school curriculum. Academic Medicine, 85(5):813-820.
Radio Interviews
Books
Fetterman, D.M., Kaftarian, S., and Wandersman, A. (2015). Empowerment evaluation: knowledge and tools for self-assessment, evaluation capacity building, and accountability. Thousand Oaks, CA: Sage.
Fetterman, D.M. (2013). Empowerment evaluation inn the digital villages: Hewlett-Packard's $15 million race toward social justice. Stanford: Stanford University Press.
Fetterman, D.M. and Wandersman, A. (2005). Empowerment evaluation principles in practice. New York: Guilford Publications.
Fetterman, D.M. 2001). Foundations of empowerment evaluation. Thousand Oaks, CA: Sage.
Fetterman, D.M. (1996). Empowerment evaluation: knowledge and tools for self-assessment and accountability. Thousand Oaks, CA: Sage.
My point went a whole lot further than that. My point was that this shows
the clear signature pattern of an institutional narrative and narrative is
a natural form. This goes against notions of 'absolute' agency but fits
with softer understandings of agency such as those of Kahneman and Bandura
and Bruner.
DF: It is interesting that you cite Bandura since so much of our views on self-determination, self-regulation, and self-efficacy are linked to his work. I remember bouncing my ideas off of him while at Stanford to see how closely we were aligned.
This understanding of narrative goes against Lyotard in subtle
ways that are often misunderstood as Lyotards theories seem to imply
absolute agency. The further point was related to the nature of
institutions as the as yet unrecognised and unexplored iteration of
insitutions - there has been a call to recognise this iterative nature for
at least 20 years not least due to the damage assumptions of progressive
linearity and the myth of accelerating pace of change are causing.
DF: I can appreciate the larger perspective of change you are looking at but I think it highlights the oppressive forces of inertia and social norms, which trust me, I can also certainly appreciate.
But what I think it does not fully appreciate is the nature of silent scientific revolutions. I have been watching and I guess been a part of this gradual transformation for some time. I think our ephemeral discussion is taking place in a larger paradigmatic context. Many fields are experiencing a change in direction not unlike this one. A marked shift is taking place in the professional allegiance of many evaluators today and over the last couple of decades. It has to do with building capacity and contributing to sustainability in place-based community contexts through evaluation. As with any change in science, the shift is gradual, involving both subjective and objective considerations. Thomas Kuhn, the preeminent historian and philosopher of science who explored the evolution of scientific revolutions, explained that the acceptance of a new paradigm depends on the phenomena of prior crisis and faith, as well as on numerous hard-headed arguments.
"The man (or woman) who embraces a new paradigm at an early stage must often do so in defiance of the evidence provided by problem-solving. He (she) must, that is, have faith that the new paradigm will succeed with the many large problems that confront it, knowing only that the older paradigm has failed with a a few. A decision of that kind can only be made on faith."
"That is one of the reasons why prior crisis proves so important. Scientists who have not experienced it will seldom renounce the hard evidence of problem-solving to follow what may easily prove and will be widely regarded as a will-o'-the wisp. But crisis alone is not enough. There must also be a basis, though it need be neither rational nor ultimately correct, for faith in the particular candidate chosen..."
This is not to suggest that new paradigms triumph ultimately through some mystical aesthetic. On the contrary, very few men (or women) desert a tradition for these reasons alone. Often those who do, turn out to have been mislead. But if a paradigm is ever to triumph it must gain some first supporters, men (and women) who will develop it to the point where hard-headed arguments can be produced and multiplied. And even these arguments, when they come, are not individually decisive. Because scientists are reasonable men (and women), one or another argument will ultimately persuade many of them. But there is no single argument that can or should persuade them all. Rather than a single group conversion what occurs in an increasing shift in the distribution of professional allegiances" (1962,. p 158).
One of the really big problems facing social and policy evaluation is that
it is increasingly out of touch with these sorts of concerns having
focussed on 'in-house' theory and methodology. One consequence is that
many of the big impact evaluations are now being done by established
disciplines and/or commercial practices. I have not problem with
commercial practices doing these big hitting evaluations because they are
honest in what they do and typically - if they are big enough organizations
- highly skilled and work to a high standard with a weak political agenda.
So, again, as I said, what I said does not in any way whatoever devalue the
efforts of those involved, does not discredit the institution, and does not
take anything away from anybody. Quite clearly any tool that can be used
to the desired purpose is a valuable tool in that context.
DF: We are in complete agreement here.
If one can think of the bigger picture one can begin to imagine the
consequences of engaging with the narrative pattern of institution
building, deliberately or otherwise, and the implications of this.
Furthermore, one should always respect, and indeed in most cases admire,
those individuals who have the courage adn fortitude to commit decades of
their life to pursuing goals that reach far beyond their own personal
sphere of existence.
What I said does not in any way whatoever devalue the efforts of
those involved, does not discredit the institution, and does not take
anything away from anybody
Df: No offense taken, at least in these quarters, and I don't think Stan takes offense either. I do think, however, we are seeing how this simple technique and approach toward viewing the world and helping others see their roles more clearly has in some cases incrementally and in other cases dramatically and swiftly changed the tide in entire communities. I think that is all that Stan and I, and many others, are saying in response the what some might view as a more cynical approach; one in which you see the cycle as coming back to the norm instead of shifting it ever so slightly and creating that inevitable ripple in social behavior over time.
Thanks in any case for the 30,000 foot perspective on change over time. Best wishes.
- David
Postscript
DF: Hi Bill
First, glad you liked the Thomas Kuhn reference. One of my undergraduate majors was devoted to studying the history of science, so he has been a long time hero of mine.
Second, concerning your statement: "I did not say things come back to the 'norm' (If I did, again, it was a slip of the keyboard)."
I think some of us interpreted your comment as returning to the norm, because you said:
"However, once EE is formally established as an institution and the
canonical narrative reconstituted and institutionalised things return to
'normal'. That is, once the dust has settled after three decades or so we
are back to where we started and we may never have left..."
I appreciate the distinction, but I think you can see why the phrase "after three decades or so we are back to where we started and we may never have left" resulted in our interpretation of returning to the norm.
In any case, you raise a very interesting point, one I first learned from a good friend of mine - Henry Levin (economics professor at Stanford at the time). He taught me to think in terms of generations, instead of 3 to 5 year demonstration projects.
When I would go over to his office and complain about losing a well trained staff person in one of my projects, he would remind me that I if I truly believed in large-scale, long-term systemic change, then I would "rejoice" in "placing" a well trained person in another part of the community to carry on the work. (It did not help in the short run since I still had at train a new person, but he was right and it got me thinking in terms of a completely different time frame - the kind of time frame you reference in your posting - which I very much appreciate.)
The reason I mention this is because you raise a very important point concerning our vision of where we have always wanted to see things go (in the "long term").
I concluded my presidential address (back in 1993), introducing empowerment evaluation to the field, with the following hope or desire:
"The ultimate test of any new approach is that as it becomes more clearly defined, useful, and acceptable, it becomes absorbed into the mainstream of evaluation. I look forward to the day when it will be simply one more tool in the evaluator’s toolbox."
So it seems like in many respects, minus the inevitable misinterpretations that occur in these discussions, we are very much in agreement about this pattern (in this case as a desired outcome from the beginning).
Best wishes Bill and thanks again for the exchange.
- David
Contact Dr. David Fetterman if assistance is needed securing these publications.
Comments