Successful Program Evaluation
()
About this ebook
Successful Program Evaluation demystifies the subject and gives the reader tools to develop stronger programs and grant proposals. It doesn't tell how to conduct a full-scale evaluation (most professional evaluators are PhD's), but how to talk to an evaluator, how to to decide if you need one, and if not how to create simple systems to measure project success on your own.
This book is for grantwriters, nonprofit managers, board members and anyone else interested in providing quality services, increasing chances of grant success and strengthening their organization. Evaluating your programs is the key success, in grantwriting and in program delivery. Analyzing program outcomes can help you find areas that need extra attention and implement cost efficiencies or improve service delivery.
Michael Wells
Michael Wells is an award-winning newspaper journalist and photographer living in the Pacific Northwest. He is from Kentucky, near Kentucky Lake. He is a graduate of Murray State University. Now a resident of the Pacific Northwest he likes to explore the vast wilderness and untamed rivers of the region with his camera.
Read more from Michael Wells
Online Trading and Stock Investing for Beginners: The Easy Way to Start Trading and Getting Rich in the Stock Market Rating: 5 out of 5 stars5/5Debt Elimination and Wealth Creation for Beginners: The Easy Way to Get Out of Debt and Build Wealth from Nothing Rating: 0 out of 5 stars0 ratingsWords to Music Rating: 0 out of 5 stars0 ratings
Related to Successful Program Evaluation
Related ebooks
Nonprofit Program Evaluation Made Simple: Get your Data. Show your Impact. Improve your Programs. Rating: 0 out of 5 stars0 ratingsManager's Guide to Program Evaluation: 2nd Edition: Planning, Contracting, & Managing for Useful Results Rating: 0 out of 5 stars0 ratingsFive Life Stages: Where You Are, Where You're Going, and What to Expect When You Get There Rating: 0 out of 5 stars0 ratingsFeedback That Works for Nonprofit Organizations Rating: 0 out of 5 stars0 ratingsThe Fieldstone Alliance Nonprofit Guide to Conducting Successful Focus Groups Rating: 0 out of 5 stars0 ratingsImpact Evaluation of Development Interventions: A Practical Guide Rating: 0 out of 5 stars0 ratingsActionable Evaluation Basics: Getting succinct answers to the most important questions [minibook] Rating: 0 out of 5 stars0 ratingsProgram Evaluation in Social Research Rating: 4 out of 5 stars4/5The Manager's Guide to Program Evaluation: Planning, Contracting, & Managing for Useful Results Rating: 3 out of 5 stars3/5Monitoring, Evaluation and Research Rating: 0 out of 5 stars0 ratingsProgram evaluation Complete Self-Assessment Guide Rating: 0 out of 5 stars0 ratingsEthics, Qualitative And Quantitative Methods In Public Health Research Rating: 0 out of 5 stars0 ratingsMonitoring And Evaluation A Complete Guide - 2020 Edition Rating: 3 out of 5 stars3/5A Guide to Project Monitoring & Evaluation Rating: 2 out of 5 stars2/5Monitoring and evaluation Skills Complete Self-Assessment Guide Rating: 0 out of 5 stars0 ratingsMonitoring and evaluation Second Edition Rating: 0 out of 5 stars0 ratingsPlanning Monitoring and Evaluation for Public and Private Sectors Rating: 0 out of 5 stars0 ratingsQualitative Research A Complete Guide - 2020 Edition Rating: 0 out of 5 stars0 ratingsStrategic Planning Workbook for Nonprofit Organizations, Revised and Updated Rating: 4 out of 5 stars4/5International Development: A Casebook for Effective Management Rating: 0 out of 5 stars0 ratingsManaging Nonprofit Organizations Rating: 0 out of 5 stars0 ratingsCollaborative Evaluations: Step-by-Step, Second Edition Rating: 0 out of 5 stars0 ratingsMethods That Matter: Integrating Mixed Methods for More Effective Social Science Research Rating: 5 out of 5 stars5/5The Behavioral Foundations of Public Policy Rating: 5 out of 5 stars5/5Policy Analysis A Complete Guide - 2020 Edition Rating: 5 out of 5 stars5/5Handbook for Monitoring and Evaluation of Child Labour in Agriculture Rating: 0 out of 5 stars0 ratingsThe Lobbying and Advocacy Handbook for Nonprofit Organizations, Second Edition: Shaping Public Policy at the State and Local Level Rating: 0 out of 5 stars0 ratingsEvaluation for Better Results Rating: 5 out of 5 stars5/5Policy Analyst A Complete Guide - 2020 Edition Rating: 0 out of 5 stars0 ratings
Social Science For You
Dumbing Us Down - 25th Anniversary Edition: The Hidden Curriculum of Compulsory Schooling Rating: 4 out of 5 stars4/5A People's History of the United States Rating: 4 out of 5 stars4/5The Denial of Death Rating: 4 out of 5 stars4/5All About Love: New Visions Rating: 4 out of 5 stars4/5Men Explain Things to Me Rating: 4 out of 5 stars4/5Come As You Are: Revised and Updated: The Surprising New Science That Will Transform Your Sex Life Rating: 4 out of 5 stars4/5You're Not Listening: What You're Missing and Why It Matters Rating: 4 out of 5 stars4/5The Art of Witty Banter: Be Clever, Quick, & Magnetic Rating: 4 out of 5 stars4/5The Sun Does Shine: How I Found Life and Freedom on Death Row (Oprah's Book Club Selection) Rating: 4 out of 5 stars4/5The Like Switch: An Ex-FBI Agent's Guide to Influencing, Attracting, and Winning People Over Rating: 4 out of 5 stars4/5Prisoners of Geography: Ten Maps That Explain Everything About the World Rating: 4 out of 5 stars4/5Verbal Judo, Second Edition: The Gentle Art of Persuasion Rating: 4 out of 5 stars4/5Just Mercy: a story of justice and redemption Rating: 5 out of 5 stars5/5Close Encounters with Addiction Rating: 5 out of 5 stars5/5Slaves in the Family Rating: 4 out of 5 stars4/5My Secret Garden: Women's Sexual Fantasies Rating: 4 out of 5 stars4/5Fervent: A Woman's Battle Plan to Serious, Specific, and Strategic Prayer Rating: 5 out of 5 stars5/5Homicide: A Year on the Killing Streets Rating: 4 out of 5 stars4/5Women Don't Owe You Pretty Rating: 4 out of 5 stars4/5King, Warrior, Magician, Lover: Rediscovering the Archetypes of the Mature Masculine Rating: 4 out of 5 stars4/5
Reviews for Successful Program Evaluation
0 ratings0 reviews
Book preview
Successful Program Evaluation - Michael Wells
Successful Program Evaluation
Grantwriting Beyond the Basics, Book 3
By: Michael K. Wells
Originally published in paper by the now defunct Portland State University Continuing Education Press
Copyright 2007 & 2012 Michael K. Wells
All Rights Reserved
Grants Northwest, 1020 SW Taylor, Suite 545, Portland, Oregon 97205
www.grantsnorthwest.com
ISBN:
Smashwords Edition 2013
This ebook is licensed for your personal enjoyment only. This ebook may not be re-sold or given away to other people. If you would like to share this book with another person, please purchase an additional copy for each recipient. If you’re reading this book and did not purchase it, or it was not purchased for your use only, then please purchase your own copy. Thank you for respecting the hard work of this author.
Other books in the Grantwriting Beyond the Basics series;
Book 1. Proven Strategies Professionals Use to Make Their Proposals Work
Book 2. Understanding Nonprofit Finances
Acknowledgements:
Grantmakers for Effective Organizations (GEO) for use of their Due Diligence Tool.
The Evaluation Forum at Organizational Research Services, Inc. for use of their logic model examples.
Dedication
Dedicated to my daughters Maya, Melissa and Julie.
You're made me so proud and my life so rich.
Foreword
When I started writing grants 30 years ago, funders didn’t routinely ask for proof that we were making a difference. Grantseekers were able to say, Give us the money and we’ll do this cool thing,
and that was that.
Those days are gone.
Today, there is tremendous pressure, not only to articulate clearly the problem we’re addressing and to quantify what we’re going to do about it, but to then to measure whether we affected any change. The pressure is coming from sources both inside and outside our organizations.
Fortunately, Michael’s insightful book will help ease that pressure.
As you have probably experienced, grantwriters are often drivers of change in organizations, and nowhere is that more obvious than in program evaluation. If grantors didn’t ask us to justify their support, many of our organizations wouldn’t hold themselves accountable. So grantwriters, those of us with the most visceral relationship with grantors, are the ones asking the hard questions inside our organizations and, indeed, changing behavior.
At least as problematic as the requests from funders is the resistance we often encounter from our own staff, who often view evaluation as a test that they might not pass. They’ll suggest the clients are too vulnerable to be asked how they’re faring, or they’ll say it’s impossible to prove whether prevention worked, or they’ll object that no one can quantify the impact of art. Often, those who raise money are considered the dark side
of the organization, and we’re told that client information is too confidential to be shared with us.
This book offers responses for each protest, and helps us show our colleagues why evaluation is in our own best interest, strengthening our programs, our organizations, and the outcomes of those we serve.
Once we start to evaluate our work, we discover it’s hard to know just how to characterize effectiveness. For example, a woman had been working with a group of Native American fathers on early childhood parenting. On the last night of class she told them she was worried about how to measure the effectiveness of the program for the funder. From the back row came the quiet voice of a shy young man: With all due respect, ma’am, you could ask us.
On the other hand, sometimes the outcomes are complex. An evaluator once asked a troubled and divided family how their counseling process had gone. We hate that therapist,
came their unified response. She’s no good, and we all agree that being with her was a waste of time!
So when the evaluator approached the therapist for her point of view, he was surprised to hear her say, This process was a complete success!
A probing question revealed her rationale. My goal with this family was to get a disparate, disconnected group of individuals to come together to fight a common enemy.
It’s easy to get intimidated or overwhelmed when the situation requires scientific or statistical evaluation. We are, after all, usually writers and not scientists or statisticians. Our fears are quelled, however, by Michael’s patient passages that make a daunting topic manageable. All three of Michael’s books in the Grantwriting Beyond the Basics series have done the same thing—taken a subject a bit beyond our reach and brought it within our grasp, making us more competent and confident, and more able representatives of the important work of our sector. The best part is that he guides us through the hard parts with the familiar kindness of a wise old friend.
He brings just the right combination of experience and skill to the task. He’s worked in nonprofits as staff, board, volunteer, and consultant, so he knows firsthand the quirks, fears and reservations people have about being judged. Stints in the business world have given him a results-oriented perspective. Over 20 years writing grants have taken him through the journey of learning his subjects from the inside out. The Grantwriting Beyond the Basics series takes this accumulated knowledge and lays it out for the reader.
As grantmakers grow more sophisticated and demanding about evaluation, it is incumbent upon us to help our organizations respond with skill and grace. The journey to effective evaluation may be arduous, but now we have a roadmap.
—Susan Howlett
Susan Howlett is Seattle-based grantwriter, consultant and trainer. She is co-author of GETTING FUNDED: The Complete Guide to Writing Grant Proposals. Information about Susan can be found at http://www.susanhowlett.com/.
Table of Contents
Acknowledgements
Foreword
Introduction
SECTION 1:
UNDERSTANDING EVALUATION
Chapter 1: Before You Start
Why Do Evaluation?
Resistance to Evaluation
Do You Need a Formal Evaluation?
Performance Monitoring
Piggyback Evaluation Reports
Program Staff Evaluation Reports
Board of Directors Evaluation Reports
Reporting on Internal Evaluation
Organizational Commitment
Chapter 2: About Program Evaluation
Basics of Evaluation
Choosing the Appropriate Evaluation
Roots of Program Evaluation
Conducting Quantitative Evaluation
Conducting Qualitative Evaluation
Evaluation Instruments
Research Design Symbols
Evaluation of Economic Costs
Chapter 3: Evaluation Considerations
Evaluability
Reliability and Validity
Confidentiality, Anonymity, and Privacy
Human Subjects Issues
Cultural Competence
Chapter 4: Statistics And Data Analysis
Levels of Measurement
Descriptive Statistics
Inferential Statistics
Chapter 5: Analyzing Qualitative Data
Thematic Content Analysis
Constant-Comparative Method
Grounded Theory
Enumerating and Diagramming
SECTION 2:
EVALUATION IN GRANT PROPOSALS
Chapter 6: What Funders Want
Foundations
GEO Due Diligence Tool
Government Agencies
Chapter 7: Coordinating Grants And Evaluation Planning
Read the Guidelines
Compose the Organizational Description
Research the Needs Statement
Define the Goals and Objectives
Using Logic Models
Narrative and Procedures
Timelines
Evaluation
Sustainability
Personnel
Replication and Dissemination
Chapter 8: Logic Models And Theory Of Change
Sample Logic Model
Developing a Logic Model and Program Theory
Goals and Objectives vs. Outputs and Outcomes
Program Logic Models for Grants
Chapter 9: Developing An Evaluation Plan
Chapter 10: Writing Your Evaluation Section
Chapter 11: Research-Based Concepts In Evaluation
Evidence-Based Practices
Best Practices
Benchmarks and Baselines
Chapter 12: Using An Outside Evaluator
Why Use an Outside Evaluator?
How to Find an Outside Evaluator
Choosing an Evaluator
Working with an Evaluator
Chapter 13: Examples Of Evaluation Sections
1. Fairview Terrace: From a Small Grant Proposal by Letter to a Family Foundation
2. Florentine Chamber Orchestra: From a Proposal to a General Purpose Foundation
3. Fairview Terrace: From a Capital Proposal to a General Purpose Foundation
4. Jefferson County Community Action: From a Grant Proposal to a State Department of Education for the 21st Century Community Learning Center
5. Jefferson County Community Action: From a Demonstration Grant to the Centers for Disease Control and Prevention (CDC) for School-Based Violence Prevention
SECTION 3: EVALUATION RESOURCES
Chapter 14: General Information
Internet Resources
Libraries
Bookstores
Chapter 15: Evaluation Instruments
Glossary
Annotated Bibliography
Index
***
Introduction
For many grantwriters, evaluation competes with finances as the least favorite part of developing proposals. It seems terribly complex, it can feel like it’s forced on you by funders, and you may experience resistance from the program people whose cooperation you need to develop a good proposal. The evaluation section is treated as a necessary evil and an afterthought—something you throw together after the real work of your proposal is done—so it is often one of the weakest parts of many grants. In a competitive funding round, a strong evaluation section may make the difference between getting funded or turned down.
Yet if you embrace evaluation and incorporate it into your grant development process, it will strengthen your whole proposal. You’ll develop more compelling needs statements, create stronger goals and objectives, and write a better narratives. Your proposals will be more fundable, and once funded, the grant programs will be more successful.
In fact, your whole organization will benefit, because evaluation is becoming more important not just in grant proposals but in all aspects of operation. The organizations that survive and thrive will be the ones that understand, measure, and work to improve themselves and their programs. And the way that’s done is through program evaluation.
Successful Program Evaluation is intended to demystify evaluation and give you the tools to develop stronger grant evaluation sections. It will not tell you how to do a full-scale evaluation with statistical outcomes, which I believe requires more training than one book can promise (most professional evaluators are Ph.D.’s). As with financial statements, you don’t need to be able to producefull-scale evaluations, you just only have understand evaluation enough to decide whether and how to approach it, and to be able to describe it.
Evaluation as a field is in flux and can be frustrating. There are different definitions for the same words and different words for the same