Evaluation Builds the Credibility Needed for Sustainable Funding
I recently had a conversation with a client about the difficult funding environment their organization is navigating.
Like a lot of organizations right now, their leadership is looking closely at programs and asking hard questions. What should we continue funding? Where should we invest more? And, inevitably, what might we have to let go?
During our conversation, my client mentioned an evaluation report we had completed together for one of their programs. We had used publicly available data from their county and state to examine population-level changes in behavioral outcomes among the youth their program was serving. And the results looked promising.
My client shared that the report had been well received by leadership. They of course couldn’t say that the evaluation was the only reason the program retained its funding. Funding decisions are rarely that simple. But they believed it played a significant role in keeping the program funded. And in a moment when other programs are being cut, their program is going to continue for another year, and maybe even grow!
These words meant the world to me—it’s our CIF model here at Common Good Data put into practice. The client gained clarity over their problem and proposed a great program to solve it. They implemented the program and tracked its impact on those it served. And lastly, they used that impact to build the case for continued funding and support.
We often talk about evaluation as just a necessary requirement. Something that goes into a grant proposal, gets submitted in a progress report, or checks a box for a funder. If we’re lucky, maybe we even learn something along the way about how the program is running—but even that is more of a “nice to have.”
But evaluation has the capacity to do much more than check a box. At its best, it can give a funder the evidence they need to answer a very consequential question: Is your program worth the investment—or better yet:
Can I trust you?
Think about the last thing you bought online
Imagine you’re shopping for something on Amazon, Etsy, Facebook Marketplace, or just about anywhere else online.
You probably don’t make your decision based solely on the seller telling you that the product is great. You look for evidence, check photos, read the product description, and look at the ratings and reviews. Of course, you also compare the price with other alternatives. Depending on what you’re buying, you might even dig into specifications, dimensions, materials, warranties, or other details.
Most importantly, you pay attention to the evidence that matters to you.
If you’re buying a used dining table, you probably care about its dimensions and condition. If you’re buying hiking boots, you’re more interested in fit, durability, and whether other people found them comfortable after ten miles on a trail. If you’re buying new IKEA BILLY bookcases during a summer room design makeover (I might have some recent experience with this one…), you want to make sure that those shelves will get the job done and bring sanity to your living spaces.
Ultimately, you’re trying to decide: Will this solve my problem, and is it worth what I’m being asked to spend? Can I trust that this product will help me accomplish this goal or task?
Funders and organizational leaders are trying to answer similar questions. They have limited resources and many worthwhile places to put them. They are looking at programs and trying to understand what those programs accomplish, how confident they can be in the evidence, and whether continued investment makes sense.
And that’s your job: provide the evidence needed to help a funder trust that you’re doing great work, so they’ll make a funding decision in your favor.
Outputs are important. But they’re only the beginning.
A great deal of evaluation in human services focuses on counting things.
How many people did we serve?
How many trainings did we conduct?
How many referrals did we make?
How many hours of services did we provide?
How many materials and literature did we distribute?
These are legitimate measures. We need to know whether programs are reaching people and whether implementation is occurring as intended. And when I work with my clients, we always start with the outputs, because they’re the easiest to count and because they are usually required by the funders.
But notice what all of those measures have in common: they tell us what the program did, but they don’t necessarily tell us what happened as a result of the program.
That distinction becomes especially important when resources are scarce. Imagine a leadership team looking at five programs, all doing worthwhile work. On one hand, you have a program that can report that it served 500 young people and delivered 2,000 hours of programming. On the other, you have a program who can share those same numbers but can also show that the young people it serves experienced a meaningful reduction in the problem the organization is trying to address. All other things being equal, we would all choose to fund the latter organization, right?
But getting from outputs to outcomes can take time. Usually, an organization begins with the low-hanging fruit of measuring outputs and eventually works their way toward building the capacity to measure outcomes. This is because it takes time to build evaluation capacity to measure complex things like behavioral outcomes.
A useful way to think about it is like this:
Activity → Reach → Outcome → Impact → Value
What did we do?
Who did we reach?
What changed? (this one might be the most important)
Why does that change matter?
And, ultimately, is this something worth continuing to invest in?
Not every evaluation can answer every one of these questions right away. But if you want your programs and services to last, you have to continue to move up this chain to start measuring outcomes, impact, and the value returned.
Choose evidence that matters to the person making the decision
This brings me back to my client’s program. We knew leadership is always examining a number of different programs and initiatives. We also knew that, on their face, all of those programs mattered. Each had staff working hard, people being served, and a compelling reason to exist. So simply documenting effort wasn’t going to distinguish our program. We needed to figure out how to get the data needed to answer a more difficult question:
What evidence do we need to gather to help leadership understand the difference we knew this program was making?
The answer wasn’t the number of young people served, the number of hours staff spent working, much money had been raised, or how many activities had been completed. All the other programs had that data, too.
We focused instead on an outcome much closer to the reason the program existed in the first place: changes in antisocial and problem behaviors among the youth population being served. And we were able to use public data to look at those changes over a multi-year period. And with these hard data, we were able to make a compelling case to leadership that the funding invested in the program was delivering measurable results..
Good evaluation is not the same as good marketing
There are obvious limits to the previous shopping analogy. Human services programs are not products on Amazon. Communities aren’t customers in a marketplace, and decisions about mental health, substance use prevention, housing, child welfare, or other human services should never be reduced to a simplistic calculation of which program produces the biggest number for the lowest price.
There is another important distinction. The purpose of evaluation isn’t to make a program look successful. It actually has to be successful. There’s no room for false advertising in the health and human services world!
A good product listing selects the most persuasive information available to make a sale. A good evaluation has a higher obligation. We have to be honest about uncertainty, limitations, unintended results, and findings that don’t support the story we hoped to tell. Your trustworthiness and your credibility as an organization matter more than the evaluation result.
Still, I think there is something useful in the analogy. Both situations depend on trust and evidence. People make decisions when they have information they believe is credible and relevant to the problem they’re trying to solve. For those of us working in human services, that creates an opportunity to think differently about evaluation:
What would someone need to know to understand why this work matters?
What evidence would make them more confident in the program?
What evidence would I accept that would suggest we’re being successful?
What outcomes would demonstrate meaningful change in people’s lives?
What qualitative stories would help explain what the numbers cannot?
What information about cost would help them understand the value of that change?
What’s common to all these questions? The focus is not just on the program you’re evaluating. It’s also on the quality of the relationship you have with person who is ultimately making decisions about the future of your programs and services. It’s also about whether they trust you.
This is what distinguishes applied evaluation in practice-contexts from traditional research: you’re building evidence designed to communicate your impact to an audience whose opinion may determine whether or not you have the financial resources to continue serve your mission. To do that, you really have to think in terms of the funder’s interests, not just your own.
Your evaluation should be able to travel without you
When budgets get tight, decisions about programs often happen in rooms where program staff aren’t even present. To quote the musical Hamliton, you might not be “in the room where it happens.”
A department director may be presenting a budget to executive leadership. A state agency may be defending an appropriation. A nonprofit executive may be sitting with a board finance committee. A funder may be comparing renewal requests.
Your evidence has to work in those rooms, too. A strong evaluation gives the people who believe in a program something concrete to carry into those conversations. And it’s more than counts of those you served, it always includes “hard” outcomes that show you’ve moved the needle on something that really matters.
And sometimes the evidence doesn’t require an expensive new study or sophisticated data infrastructure. In the case of my client’s program, publicly available data gave us an opportunity to examine an outcome that mattered. The important step was knowing what question we were trying to answer and finding credible evidence that could help answer it.
It’s all about trust.
This reflection has focused on the importance of moving from outputs to outcomes to make a compelling case to funders to support your work. But why do funders want outcomes, anyway?
I like to think of evaluation as just another pathway to building trust. We might think of trust in terms of a good handshake, a person who follows through on a project, or someone you can confide in and has your back. But trust between institutions and organizations requires a little more than the person-to-person trust indicators. For behavioral health and human services organizations, that trust is built through generating evidence of consistent success on the outcomes that matter. Strong evaluation builds the credibility upon which your organization rests.
So the next time you dust off that logic model, redesign a survey, or launch some focus groups, think about it as an exercise in building trust. It will bring a whole new perspective to your work!
