Tuesday, February 27, 2018

Using Google Sheets and Forms and Zoom to Remotely Facilitate an Empowerment Evaluation Exercise

by David Fetterman, Jason Ravitz, and Kathy Haynie

Jason Ravitz (Google) and David Fetterman (Fetterman & Associates, past-president of AEA, and founder of empowerment evaluation) have been using empowerment evaluation in various educational settings, including a graduate school program and with Kathy Haynie (Haynie Research and Evaluation) and Tom McKlin (The Findings Group), in our work with two computer science education evaluation learning communities.
Empowerment evaluation is the use of evaluation concepts, techniques, and findings to foster improvement and self-determination.  This approach aims to increase the likelihood that programs will achieve results by increasing the capacity of program stakeholders to plan, implement, and evaluate their own programs.
3-Step Approach.  One empowerment evaluation approach involves helping a group: 1) establish their mission; 2) take stock of their current status; and 3) plan for the future.  Additional tools include an evaluation dashboard to help communities monitor their own progress.
CS/STEM Learning Communities. The “Evaluation Wrecking Crew” includes over 60 CS education evaluators across the country. A second group (with some overlap) is the NSF-funded Computer Science Outcomes Networked Improvement Community (CSONIC).  
We have joined forces to: 1) build a CS/STEM repository of evaluation instruments and approaches; 2) build a common hub for the community, with the assistance of Oak Ridge Associated Universities; and 3) educate the CS community about the value and role of evaluation to improve the quality of CS and STEM education.  We meet biweekly using Zoom video-conferencing software.  

Kathy Haynie (Haynie Research and Evaluation) 
Remotely Facilitates Bi-monthly Meetings
Online Spreadsheet. Jason designed a 3-step online spreadsheet, using Google Sheets, to facilitate the empowerment evaluation process used in both the Evaluation Wrecking Crew and CSONIC workshops.
Mission. Our collaborative process allowed workshop members to remotely record their views about the mission or purpose of the group.  Later comments were transformed into a mission statement (using Google Docs).
Taking Stock. A second sheet in the spreadsheet was devoted to “brainstorming” a list of the group’s most important activities. Members prioritized the list by “voting” for the most important activities to evaluate as a group.
A third sheet was populated with the list of the prioritized activities. The online workshop participants used a 1 (low) to 10 (high) scale to rate their performance on the “taking stock” sheet.  We used the results to facilitate a dialogue about the ratings using videoconferencing software and referencing participants’ ratings.
Planning for the Future.  We used a fourth sheet to help the group record plans for the future, specifying goals, strategies, and evidence.
Evaluation Dashboard.  A final sheet was devoted to the dashboard to help us monitor our own performance.  It included: goals, strategies, and evidence.
Computer Science Education Evaluators 
Conducting an Empowerment Evaluation Online
Rad Resources:
Free Template.  This spreadsheet is available (free) to use to facilitate your own empowerment evaluation exercise remotely: tinyurl.com/eeblank.
Other free tools we have used include Google Forms to help graduate students evaluate their own as well as their peers’ work.  We used these data to assess students’ performance, and in the process, make mid-course corrections concerning our instruction.  Finally, we used Google Evaluation Worksheets to help them refine their proposals:  /tinyurl.com/evalworksheet-google Additional resources can be found here.
Do you have questions, concerns, kudos, or content to extend this aea365 contribution? Please add them in the comments section for this post on theaea365 webpage so that we may enrich our community of practice. Would you like to submit an aea365 Tip? Please send a note of interest to aea365@eval.org. aea365 is sponsored by the American Evaluation Association and provides a Tip-a-Day by and for evaluators.

Friday, February 23, 2018

Abraham Wandersman Presents GTO in Thailand

Dr. Abraham Wandersman in Thailand

Dr. Abraham Wandersman from the University of South Carolina, Columbia presented Getting-to-Outcomes (GTO) to funders and policy makers in Bangkok, Thailand this week. His most recent book with Drs. Fetterman and Kaftarian is:  Empowerment Evaluation: Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability (Sage Publications).

Colleagues in this picture include from left to right: 

Dr. Oranee from R2 R, Dr. Akarin from R2R, Dr. Jadej from NHSO (National Health Security Office, Dr. Wandersman, Dr. Somsak, Dr. Bundit (Deputy CEO Health Promotion Fund (HPF), Dr. Choochai NHSO, and Dr. Pairote HPF.

“Thanks you to Prof wandersman for introducing us to GTO and how it could be applied to ensure outcome and not only for “scaling up good models from research”. Most of us present in this meeting have been doing various things to ensure whatever we did, providing services, purchasing services or granting projects on health promotion, will lead to better and bigger patient outcome or health outcome. Examples and ideas from the exchange will be valuable for us to find new ways of getting outcome with your GTO being one of the approaches and tools for doing so. Your experiences with empowerment evaluation and learning helped to shed better light to our efforts in using various methods to achieve learning health system while working to get better outcomes in the complex health system such as Thailand.”

Dr. Somsak Chunharas was a Menschel Senior Leadership Fellow at the Harvard T.H. Chan School of Public Health in the Fall 1 2017 term. He formerly served as Deputy Minister of Health for Thailand and is currently President of National Health Foundation in Thailand, an NGO promoting and coordinating evidence-based health policy and system development.