By Elizabeth Schuster, Environmental Economist
INTRODUCTION
Nonprofits are constantly asked: “What impact are you making?”
You know that your nonprofit benefits communities. There’s a good chance you are having an impact on the local economy. And in many cases, your programs also benefit ecosystems.
But when you look for specific data to make the case, you may hit a wall.
You want to do an impact assessment, but:
- You’re concerned it can’t be done for less than $100,000.
- You worry that funders and communities won’t trust your numbers – what if the analysis is not extensive enough?
- You’ve heard you need a university research partner—otherwise, will anyone trust your findings?
Over the years, I’ve learned that it is possible to do program impact assessments with high quality data, without a 6-figure budget.
Here are my 4 tips to use a more streamlined approach while still ensuring quality in your data & analysis.
TIP 1: DON’T CUT CORNERS IN THE ASSESSMENT PHASE
Phase 1 of a nonprofit impact assessments is what I describe as the Assessment Phase. It begins with understanding all aspects of the program(s) being assessed. This may include completing a result chain to map out the steps, to show how short-term deliverables add up to long-term outcomes.
Phase 1 also typically includes listening to stakeholder perspectives. Many of my creative ideas around how to demonstrate impact in a more meaningful way has come from being inspired by ideas from stakeholders.
Also, Phase 1 is a time where we review existing data available.
I usually spend two to three months in the assessment phase. Investing this time upfront to fully understand your programs, stakeholders, and data is truly worth the time. It helps you save time at every additional step later in the process. This strong foundation not only means the rest of the process will go faster, but it also sets you up for higher quality research.
TIP 2: NARROW AND SIMPLIFY THE PROCESS FOR DATA ANALYSIS
Phase 2 has two elements. First, it starts with narrowing which outcomes you care about. This is critical. Part of what drives up the cost in the process is collecting and analyzing too much data. What we usually find is that there are only a small handful of outcomes or data points that really matter. This subset of outcomes represents the core essence of what matters for your program’s impact.
Second, replicate existing studies where possible. Here’s a simple example to illustrate the point:
My colleague and I did an economic impact assessment of the $49 million Vermont Everyone Eats program. We were able to use an existing economic multiplier from a past study. This allowed us to show that the program led to $78 million in additional spending beyond the initial $49 million, by restaurants and farmers re-circulating those dollars back into local inputs and local labor pools.
If we hadn’t had access to that existing multiplier, it would have taken more resources to capture and apply the multiplier.
TIP 3: USE TRIANGULATION IN YOUR DATA ANALYSIS
Impact assessments are only considered “high quality” if you can show in a causal way that your program led to an outcome. If the outcome would have happened anyway, then you should not claim that the outcome happened because of your organization. So, how do you show that your organization contributed meaningfully to that outcome?
When on a shoestring budget, you’re likely to have small sample sizes. This means it’s often impossible to show causality in a way that’s statistically significant. This is why triangulation helps. This means comparing the findings from different data sets. If you get the same findings from three sources of data (hence the word “triangulation”), this increases trust in the quality of the findings.
Different sources of data for comparison might include:
- Data from a report to a funder
- Data from your annual report or organizational dashboard
- Data from a public source like a governmental database
- Spatial data
- New data from a survey or focus groups
TIP 4: ALWAYS BE PLANNING FOR A SPECIFIC TARGET AUDIENCE AND HOW YOU WILL PRESENT THE FINDINGS
Every step of the process, be thinking about who your target audience is and what data would be interesting to them. Moreover, think about their decision-making process and how they might use your findings to inform decisions.
If you always have that on your radar while doing the analysis, you can capture both stories and relevant information as you go. That way, when you finish and it comes to putting together a case for support or infographic, all that information is readily available.
CONCLUSION
The key to a lower cost, high integrity impact assessment is to spend time in the assessment phase at the beginning. That is truly the most important element and sets you up for a strong foundation. Then, build off existing research and data. Next, use triangulation to show impact. Finally, make sure you are considering your target audience.
This will help you be incredibly focused on only the right kinds of data, meaning you won’t waste resources on data that will never be used. Much of the wasted resources in impact assessments come from collecting and analyzing too much data that isn’t relevant. By streamlining your process, you can save time and put extra emphasis on higher quality data that matters.