Sunday, August 19, 2018

Empowerment Evaluation and Freirian Pedogogy in Portuguese



The collection Pedagogy of Evaluation highlights evaluation approaches that have been influenced by Freirean pedagogy and share Freirean values, modes of engagement, and desired outcomes. These approaches include:  social justice-focused evaluations, democratic deliberative evaluation, empowerment evaluation, feminist evaluation, transformative evaluation, and critical systems evaluation.

This collection was recently translated into Portuguese.  One of the chapters is "Transformative Empowerment Evaluation and Freirean Pedagogy:  Alignment with an Emancipatory Tradition," by David Fetterman. It is: "Pedagogia da Avaliacao e Paulo Freire:  Incluir para transformar," iPortuguese. 


The empowerment chapter describes how empowerment evaluation and Freirean pedagogy share a common emancipatory tradition.  These approaches help people learn to confront the status quo, by questioning assumptions and prescribed roles, unpacking myths, rejecting dehumanization, and no longer blindly accepting the "truth" about how things are or can be.  They help people think critically about the world around them.

In addition, the art work is stunning.  This is a Silvia Fittipaldi art edition.  Claudius Ceccon's drawing are informative, insightful, and rich in color and expression.  


He is both an artist and social activist.  He is also the father of Claudia Ceccon.  (Claudia and Thomaz Chianca have a chapter in this publication titled, "Pedagogy in Process Applied to Evaluation:  Learning from Paulo Freire's Work in Guinea-Bissau".)

Claudius is also the Director of Centro de Criacao de Imagem Popular.  It is a not for profit and independent NGO that has been developing for the past three decades Education and Communication projects.  Their mission is to contribute to the process of democratisation in Brazil by producing information and developing methodologies aimed at influencing public policies and strengthening citizenship's fundamental rights.

The translation was sponsored by Fundacao Roberto Marinho.  Thanks are also extended to Madza Ednir for her assistance in participating in the translation and dissemination of this effort.

Issuu is a digital publishing platform that makes it easy to publish online.


Tuesday, February 27, 2018

Using Google Sheets and Forms and Zoom to Remotely Facilitate an Empowerment Evaluation Exercise


by David Fetterman, Jason Ravitz, and Kathy Haynie

Jason Ravitz (Google) and David Fetterman (Fetterman & Associates, past-president of AEA, and founder of empowerment evaluation) have been using empowerment evaluation in various educational settings, including a graduate school program and with Kathy Haynie (Haynie Research and Evaluation) and Tom McKlin (The Findings Group), in our work with two computer science education evaluation learning communities.
Empowerment evaluation is the use of evaluation concepts, techniques, and findings to foster improvement and self-determination.  This approach aims to increase the likelihood that programs will achieve results by increasing the capacity of program stakeholders to plan, implement, and evaluate their own programs.
3-Step Approach.  One empowerment evaluation approach involves helping a group: 1) establish their mission; 2) take stock of their current status; and 3) plan for the future.  Additional tools include an evaluation dashboard to help communities monitor their own progress.
CS/STEM Learning Communities. The “Evaluation Wrecking Crew” includes over 60 CS education evaluators across the country. A second group (with some overlap) is the NSF-funded Computer Science Outcomes Networked Improvement Community (CSONIC).  
We have joined forces to: 1) build a CS/STEM repository of evaluation instruments and approaches; 2) build a common hub for the community, with the assistance of Oak Ridge Associated Universities; and 3) educate the CS community about the value and role of evaluation to improve the quality of CS and STEM education.  We meet biweekly using Zoom video-conferencing software.  

http://aea365.org/blog/using-google-sheets-and-forms-and-zoom-to-remotely-facilitate-an-empowerment-evaluation-exercise-by-david-fetterman-jason-ravitz-and-kathy-haynie/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+aea365+%28AEA365%29
Kathy Haynie (Haynie Research and Evaluation) 
Remotely Facilitates Bi-monthly Meetings
Online Spreadsheet. Jason designed a 3-step online spreadsheet, using Google Sheets, to facilitate the empowerment evaluation process used in both the Evaluation Wrecking Crew and CSONIC workshops.
Mission. Our collaborative process allowed workshop members to remotely record their views about the mission or purpose of the group.  Later comments were transformed into a mission statement (using Google Docs).
Taking Stock. A second sheet in the spreadsheet was devoted to “brainstorming” a list of the group’s most important activities. Members prioritized the list by “voting” for the most important activities to evaluate as a group.
A third sheet was populated with the list of the prioritized activities. The online workshop participants used a 1 (low) to 10 (high) scale to rate their performance on the “taking stock” sheet.  We used the results to facilitate a dialogue about the ratings using videoconferencing software and referencing participants’ ratings.
Planning for the Future.  We used a fourth sheet to help the group record plans for the future, specifying goals, strategies, and evidence.
Evaluation Dashboard.  A final sheet was devoted to the dashboard to help us monitor our own performance.  It included: goals, strategies, and evidence.
Computer Science Education Evaluators 
Conducting an Empowerment Evaluation Online
Rad Resources:
Free Template.  This spreadsheet is available (free) to use to facilitate your own empowerment evaluation exercise remotely: tinyurl.com/eeblank.
Other free tools we have used include Google Forms to help graduate students evaluate their own as well as their peers’ work.  We used these data to assess students’ performance, and in the process, make mid-course corrections concerning our instruction.  Finally, we used Google Evaluation Worksheets to help them refine their proposals:  /tinyurl.com/evalworksheet-google Additional resources can be found here.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on theaea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Friday, February 23, 2018

Abraham Wandersman Presents GTO in Thailand

Dr. Abraham Wandersman in Thailand

Dr. Abraham Wandersman from the University of South Carolina, Columbia presented Getting-to-Outcomes (GTO) to funders and policy makers in Bangkok, Thailand this week. His most recent book with Drs. Fetterman and Kaftarian is:  Empowerment Evaluation: Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability (Sage Publications).


Colleagues in this picture include from left to right: 

Dr. Oranee from R2 R, Dr. Akarin from R2R, Dr. Jadej from NHSO (National Health Security Office, Dr. Wandersman, Dr. Somsak, Dr. Bundit (Deputy CEO Health Promotion Fund (HPF), Dr. Choochai NHSO, and Dr. Pairote HPF.

“Thanks you to Prof wandersman for introducing us to GTO and how it could be applied to ensure outcome and not only for “scaling up good models from research”. Most of us present in this meeting have been doing various things to ensure whatever we did, providing services, purchasing services or granting projects on health promotion, will lead to better and bigger patient outcome or health outcome. Examples and ideas from the exchange will be valuable for us to find new ways of getting outcome with your GTO being one of the approaches and tools for doing so. Your experiences with empowerment evaluation and learning helped to shed better light to our efforts in using various methods to achieve learning health system while working to get better outcomes in the complex health system such as Thailand.”

Dr. Somsak Chunharas was a Menschel Senior Leadership Fellow at the Harvard T.H. Chan School of Public Health in the Fall 1 2017 term. He formerly served as Deputy Minister of Health for Thailand and is currently President of National Health Foundation in Thailand, an NGO promoting and coordinating evidence-based health policy and system development.

Thursday, September 14, 2017

2017 AEA Outstanding Evaluation Award Winner

Dr. Abraham Wandersman and his multidisciplinary evaluation team are the recipients of the 2017 AEA Outstanding Evaluation Award.


Principal Investigator


They evaluated the RWJF funded Spreading Community Accelerators through Learning and Evaluation (SCALE) initiative; a portion of the 100 Million Lives initiative, focusing on low-income US communities that had high capacity collaborations to address health and social issues and improve health equity.



The evaluation design monitored, modeled, and provided actionable insights to manage the complexity of an evolving program theory and support model, while providing areas for improvement.


The Getting To Outcomes ® framework, described in the recently revised edition of Empowerment Evaluation (Fetterman, Kaftarian, and Wandersman), was a systematic, no-nonsense approach to facilitating and measuring interventions to achieve outcomes.

The elements of the SCALE evaluation that made it exemplary, included the up-front work in planning for data collection and stakeholder involvement, the rapid-cycle provision of feedback, and the content knowledge that the evaluators brought to the situation.

The multi-method design and the demonstrated usefulness of findings also attest to it as an exemplar. In addition, the approach has been adopted by the Centers for Disease Control and Prevention and Morehouse School of Medicine.


We invite you to celebrate Dr. Wandersman's accomplishments with us at both the the AEA luncheon and the CPE business meeting.

Friday, August 25, 2017

Mareike Meyn - Empowerment Evaluation and Digitization of Rural Germany

Digitization: Status Quo and future trends – a new impact on life in rural areas?

Mareike Meyn visited David Fetterman earlier this year in California. Mareike is a McCloy Fellow on Global Trends by the American Council on Germany.


She interviewed David Fetterman about his book Empowerment Evaluation in the Digital Villages: Hewlett-Packard's $15 Million Race Toward Social Justice (Stanford University Press).


She was interested in learning more about the use of empowerment evaluation in rural areas to bridge the digital divide. David's book highlights effective strategies in this area. She is applying what she has learned to rural digital development in Germany. An abstract of her work is provided below:

Digitization: Status Quo and future trends – a new impact on life in rural areas?

Digital possibilities create change. This change is necessary, especially when one looks at rural areas in Germany and their struggle to maintain social services, access to education, possibilities of employment and the like, under the pressure of the demographic development such as aging and exodus of young, qualified citizens. In the U.S. there are many innovative, digital hubs in urban centers such as in Seattle, San Francisco or New York, where people, so it seems, know how to use all the opportunities of the digital age. Not only in urban settings, but also in some rural parts of America such as Mississippi, where some unique strategies on how to transit into the digital age can be found. The paper presents insights on how Americans employ today's digital possibilities regarding the structural framework, dissemination of knowledge, the mindset, and all encompassing community development approaches for digital empowerment. Findings are that the potentials for rural communities are tremendous. Applied in a well-thought of way, digital possibilities can not only create change, but may even create a future leap-frog effect for rural communities.

Contact Information:
m.meyn@andreas-hermes-akademie.de
Documentary of the Journey: www.meynland.tumblr.com

Wednesday, June 21, 2017

Empowerment Evaluation’s 21st Anniversary: A Celebration, Comment & Critique


ABSTRACT

David Fetterman introduced empowerment evaluation to the field of evaluation during his presidential address 21 years ago. Since that time it has been used in over 16 countries, ranging from corporate offices of Google and Hewlett-Packard to squatter settlements and townships in South Africa. Empowerment evaluation has been used by the W.K. Kellogg Foundation, Centers for Disease Control and Prevention, US. Department of Education, Stanford University School of Medicine, and Native American tribes in reservations stretching from Michigan to San Diego.


David Fetterman, Shakeh Kaftarian, Abraham Wandersman, and many other empowerment evaluators, have published 5 books on empowerment evaluation. The 21st anniversary of this approach was celebrated with a panel of luminaries who have helped shape empowerment evaluation with their critiques, concerns, and congratulations. They included Drs. Steward Donaldson, Michael Scriven, Michael Patton, and Marvin Alkin. Their comments are illuminating, engaging, and presented in this special topic edition of E&PP.



Celebrating the 21st anniversary of empowerment evaluation with our critical friends David Fetterman and Abraham Wandersman
ABSTRACT

This special topic edition of E&PP presents the insights of luminaries in the field who have helped shape empowerment evaluation with their critiques, concerns, and congratulations. We celebrate their contributions to empowerment evaluation. This special topic edition of E&PP presents their comments about an evaluation approach that, according to president Stewart Donaldson, has “gone viral” across the globe (Donaldson, 2015).
To set the stage for these critical friends’ comments, additional context for their discussion is provided. In addition, this special topic edition concludes with a brief comment on their thoughts.

Empowerment evaluation: An approach that has literally altered the landscape of evaluationStewart Donaldson
ABSTRACT

The quest for credible and actionable evidence to improve decision making, foster improvement, enhance self-determination, and promote social betterment is now a global phenomenon. Evaluation theorists and practitioners alike have responded to and overcome the challenges that limited the effectiveness and usefulness of traditional evaluation approaches primarily focused on seeking rigorous scientific knowledge about social programs and policies. No modern evaluation approach has received a more robust welcome from stakeholders across the globe than empowerment evaluation.
Empowerment evaluation has been a leader in the development of stakeholder involvement approaches to evaluation, setting a high bar. In addition, empowerment evaluation’s respect for community knowledge and commitment to the people’s right to build their own evaluation capacity has influenced the evaluation mainstream, particularly concerning evaluation capacity building. Empowerment evaluation’s most significant contributions to the field have been to improving evaluation use and knowledge utilization. 

Empowerment evaluation: Exemplary is its openness to dialogue, reflective practice, and process useMichael Quinn Patton

ABSTRACT

On the occasion of the 21 st anniversary of empowerment evaluation, congratulations are in order for having established global credibility, demonstrated utility, and for its exemplary openness to dialogue, reflective practice, and process use.
I remember well the 1993 annual convention of the American Evaluation Association in Dallas when David introduced the idea of empowerment evaluation in his presidential keynote. It was an innovative and radical approach that was met with much initial skepticism, but the approach has certainly prevailed, gaining not only legitimacy but utility, and, as evidenced in the documentary record, is being implemented and appreciated worldwide.
I've had the privilege over the years of engaging in dialogue with David, Abe, and others about various aspects of empowerment evaluation. Certainly ones of the ways in which empowerment evaluation is exemplary is its openness to dialogue and reflective practice. 

Empowerment evaluation 21 years later: There is much to admire about empowerment evaluation
Michael Scriven
ABSTRACT
There is much to admire about empowerment evaluation, and on this festive occasion, I will begin with the features that I most admire.

1 This approach begins with the people who know the most of any group about the actual operation of the program (or the product, policy, person, etc. if we go beyond program evaluation). This knowledge is often highly inaccessible for external evaluators and often crucial to the validity of the evaluation.
2 Empowerment Evaluation (EE) is dealing directly with the agency for implementation, and hence the people in perhaps the best position to implement recommendations for improvement.
3 Using program staff as the evaluators gives one access to and perhaps the best chance of control over abuses of staff and impactees.
4 Although this is not an unmixed blessing, it is often important that EE frequently converts agents into advocates.
5 In converting agents into advocates, EE can sometimes transcend the limits of a particular program and make them advocates for a methodology, not only EE, but serious tools used in its implementation.
6 EE provides a great machinery for three functions that are related to evaluation and frequently required in order to maximize its implementation: marketing, explaining, and justifying a program.
7 A powerful and possibly unique (in practice) level of the ethical and pragmatic use of meta-evaluation. I try to match David on this, and indeed advocate to David on this, by going further than his enthusiasms for the use of the “critical friend” to the use of “critical enemy” but am less successful. However, I never think of empirical evaluation without reflecting on his inspirational example of treating his critics as friends—and not just friends but helpers—as they indeed are. The connection between us is close because we are both part of that small group who really believe that proposition and act on it.

When is a theory a theory? A case example
Marvin C. Alkin
ABSTRACT

This discussion comments on the approximately 20 years history of writings on the prescriptive theory called Empowerment Evaluation. To do so, involves examining how “Empowerment Evaluation Theory” has been defined at various points of time (particularly 1996 and now in 2015). Defining a theory is different from judging the success of a theory. This latter topic has been addressed elsewhere by Michael Scriven, Michael Patton, and Brad Cousins. I am initially guided by the work of Robin Miller (2010) who has written on the issue of how to judge the success of a theory. In doing so, she provided potential standards for judging the adequacy of theories. My task is not judging the adequacy or success of the Empowerment Evaluation prescriptive theory in practice, but determining how well the theory is delineated. That is, to what extent do the writings qualify as a prescriptive theory. 

Celebratory reflections, appreciations, clarifications, and comments
David Fetterman and Abraham Wandersman
ABSTRACT

Our thanks are extended to each one of our critical friends. They have been an integral part of our lives for over two decades. They have enriched our understanding and held the bar high. They, along with the communities we work with, inspire us to improve both in theory and in practice. We conclude this special topic edition of Evaluation and Program Planning empowerment evaluation’s 21 anniversary celebration – with a few reflections, appreciations, clarifications, and comments concerning the panelists’ remarks. 

Saturday, April 08, 2017

Empowerment Evaluation and Computer Science Education Evaluation
NSF Funded Initiative



This is the Computer Science Outcomes Networked Improvement Community's empowerment evaluation working group. The learning community of computer science education evaluators and STEM evaluators came together to create a common agenda for the future.

This exercise highlights the groups' efforts to establish a 1) mission, 2) take stock or assess their efforts to-date, and 3) plan for the future.



Kathy Haynie and Tom McKlin are responsible for the CSONIC effort. David Fetterman and Jason Ravitz facilitated the empowerment evaluation exercise.

The session is available for public viewing here on YouTube.




Empire Radio Interview about Empowerment Evaluation
 with Dr. David Fetterman


Dr. Fetterman was interviewed on the Empire Radio Network.  He spoke about evaluation, including empowerment evaluation.  He highlighted a few of his projects, including his work with Google (with Jason Ravitz) and Hewlett-Packard.  Dr. Fetterman also described how his team is using empowerment evaluation at the Recovery Cafe (a peer assisted program for homeless and substance abuse populations in Seattle).  He also described his 12th year working with tobacco prevention programs in Arkansas.  Dr. Fetterman concluded his interview by mentioning how his mom was an inspiration and role model. The interview is about 8 minutes long.  

Select here to listen to the broadcast. 



Friday, May 27, 2016

Empowerment Evaluation in Google Series

Empowerment Evaluation in Google


Collaborative, Participatory, and Empowerment Evaluation Sponsored Series

As Co-chair of the Collaborative, Participatory, and Empowerment Evaluation TIG with Liliana Rodriguez-Campos, we are highlighting our "Empowerment Evaluation at Google (and Beyond) Series" and encourage you to join us.  
It includes an AEA 365 Coffee Break, an AEA365 blog posting, and an AEA eStudy.  It is a rare opportunity to get a glimpse into what we are learning while facilitating capacity building and specifically evaluation training. We have provided the dates and described the series events below:
"Coffee Break" Webinar
“Coffee Break”:  Using the Power of Rubrics and Technology for Empowerment Evaluation at Google and Beyond - Fetterman and Ravitz
We highlighted how we use rubrics to enhance group learning in empowerment evaluation at Google and in higher education, specifically the Pacifica Graduate Institute.  In addition, we discussed how an analysis of the patterns of student self- and peer ratings helps instructors determine where students/participants understand the concepts and where additional attention is merited.
Some of the “free” technology we used included:  Doctopus (manage classroom assignments and assessments), Goobrics for Students (self- and peer assessment), and Google Forms (online surveys).  We are also using an Evaluation Planning Worksheet to facilitate the process.



Members can view the recorded webinar.


AEA 365 Blog

June 5:  “AEA 365”: We are also hosting an AEA365 blog posting about empowerment evaluation at Google on June 5 and throughout the week.  It is titled:  Using the Power of Rubrics and Technology for Empowerment Evaluation at Google and Beyond.  Please come and add your insights and questions.




eStudy Webinar (two 1/5 hour sessions)

June  16 & 23  eStudy:  We will complete the series with an eStudy about Empowerment Evaluation, Google,Higher Education, and the use of rubrics to focus group learning (two 1.5 hour sessions).  It is titled: Online tools and strategies for empowerment evaluation at Google and beyond:  Building evaluation capacity by harnessing the power of rubrics.  It will provide additional depth into empowerment evaluation concepts and online tools and strategies, ranging from process use and the use of critical friends to using a virtual classroom to facilitate group learning and enhance instruction.  

This is scheduled for June 16 & 23 2:00-3:30 pm EST.  

It will provide additional depth into empowerment evaluation concepts and online tools and strategies, ranging from process use and the use of critical friends to using a virtual classroom to facilitate group learning.  To register go to this page and scroll down:  http://www.eval.org/p/cm/ld/fid=121  
Please join us for any part of or all of the series we have planned for you.  If you have any questions, please feel free to contact me at fettermanassociates@gmail.com

For additional information

* Wikipedia and our empowerment evaluation blog at:  evaluation.blogspot.com.  

* Our latest book:  Fetterman, D.M, Kaftarian, S., and Wandersman. A. (2015).  Empowerment Evaluation:  Knowledge and Tools for Self-Assessment, Evaluation Capacity Building, and Accountability. Thousand Oaks, Ca: Sage.

* Article describing use of technology:  Ravitz, J., & Hoadley, C. (2005). Supporting change and scholarship through review of online resources in professional development settings. British Journal of Educational Technology, 36(6), 957-974.  http://academia.edu/1139425


Friday, April 29, 2016

"Communities of Conversation: and Empowerment Evaluation

"Communities of Conversation"
and Empowerment Evaluation

"Communities of Conversation" Webinar

Dr. David Fetterman and Ms. Melanie Ogleton were invited to speak at a “Communities of Conversation” webinar session, supported by the Substance Abuse and Mental Health Service Agency.  

Webinar participants evaluate or are responsible for operating community-wide substance use recovery programs.  

David and Melanie responded to questions about using empowerment evaluation in Peer Recovery Support Services. Topics ranged from hiring an empowerment evaluator to fitting empowerment evaluation in existing federal contracts.  

The conversation is available at:  http://altarum.adobeconnect.com/p1ahrmrourj/

Contact Leah Dyson and Adam Viera at Altarum Institute for additional information. 

Tuesday, November 24, 2015

21st Anniversary of Empowerment Evaluation Celebration

American Evaluation Association Celebration of Empowerment Evaluation

The 21st anniversary of empowerment evaluation celebration was held at the American Evaluation Association professional meeting in Chicago, IL., November 2015.  



Right to Left: David Fetterman, Stewart Donaldson, Abraham Wandersman, Marvin Alkin, and Michael Scriven.  (Michael Patton was available by video.)

Luminaries in the field complemented and critiqued the approach, including Michael Scriven, Michael Patton, Marv Alkin, and Stewart Donaldson. 



Michael Scriven


Michael Patton 
(His comments were taped due to a schedule conflict.
They are available on YouTube 



Marv Alkin



Stewart Donaldson

David Fetterman & Abraham Wandersman highlighted case examples and fielded questions. 



David Fetterman
(speaking at the podium)


Abraham Wandersman

It was quite an intellectual celebration. The room was packed - colleagues lined the walls, filled the aisles, and our colleagues even spilled out into the hallway.



The atmosphere was filled with excitement and anticipation.  The comments were primarily complementary and constructive.  However, the session would not have been authentic without the normal academic critique.  

We took copious notes, as usual, and plan to continue to refine and improve our work.  Many thanks for the celebratory comments and critiques.  

Happy 21st anniversary empowerment evaluation.








Sunday, November 15, 2015

Road Maps

Road Maps:  Are We There Yet?


Using Road Maps as a Tool for Participatory Community Engagement

Every time a community engagement process is launched, community members want to know and understand the purpose and outcomes. More importantly is the fact that everyone involved deserves to know this information at the very start.

To answer this question, our organization Communities in Collaboration | Comunidades en ColaboraciĆ³n has developed what we call a Road Map. This simple visual tool quickly illustrates the steps in any process that we are proposing. It doesn’t matter whether it’s a 2-year process, or a 2-month process; any process can be boiled down to three or four major stages.

At the beginning of the process, our Road Maps help people build shared expectations about what the steps, goals, and outcomes will be. The Road Map helps us stay oriented toward those goals and outcomes. And when we come to the end, it reminds us of the major steps we’ve taken. At any point, it truly acts as a map – when we ask “where are we and how did we get here?” the Road Map is there to guide us.

For more information about Road Maps and the example of a recent project above contact: Susana Morales Konishi at Communities in Collaboration.