Choosing an evaluation approach that fits
Evaluation can be expensive...
But it doesn't have to be.
Finding the right evaluation approach for your program or organization is a matter of striking a balance. On one hand, you have questions like:
- What do we want to learn?
- How mature is this program?
- What strategic decisions are at stake?
- What are the expectations of our funders or constituents?
On the other hand, there are questions such as:
- How much time are we willing to wait for insights?
- What is the appropriate amount to budget for evaluation?
- What is the current culture and capacity of my organization around evaluation?
- What relevant evidence or data already exists?
As a firm with roots in strategic planning and facilitation, BERK takes a pragmatic and collaborative approach. Our advice? Start with your learning question. You might find an evaluation approach that fits your team and budget just right. Some questions you might have include:
How could this implementation go better? Critical self-reflection is in some ways the most basic form of evaluation. Facilitating "evaluative thinking" is an approach that leans on your own team to think about implementation and how to improve. A well-timed workshop with a neutral third-party is all you need to carve out the time and space do this. Added bonus: your team feels empowered to improve practices when they identify the issues and solutions, rather than having an outsider assess their performance.
How much bang are we getting for our buck? This question points to economic evaluation methods. This type of evaluation is a focused effort to crunch the numbers in relation to the dollars. The crux lies in identifying the right "bang" and having the quantitative data to track it. Commonly monitored metrics like participation can be straightforward exercises and may be all you need at a given point in time with a given set of stakeholders. For more complex outcomes, we need to get creative about measurement and meaning.
Is this a viable program? It's common in a fast-paced world to set out with a pilot or demonstration program that is designed to try something new but be immediately subject to pressure to grow and scale before the first step out of the gate. We've all heard the term "building the plane as we're flying it," and formative evaluation is the instrument panel, taking stock of the flight as we do so. This type of inquiry is vital to challenging the assumption of scale. Sometimes it is better in the long run to pull the ripcord and make room for continued innovation, and formative evaluation can help see that.
Is our program hypothesis panning out as we thought? Program and process evaluation compares the outcomes of your implementation to what you thought would happen, and why it did or did not. Whether laid out nicely in an implementation plan or logic model, or more implicit in the way a program is happening, programs have grounding in some theory or hypothesis about what works. An evaluator can help articulate that hypothesis if it isn't already, and examine which program components, behaviors, and practices are driving the outcomes you are seeing.