If you’ve spent any length of time working or volunteering for direct-service nonprofits, I hope you can agree with me on two things:
(1) the work so many of them do is incredible; and,
(2) measuring the quality / effectiveness / impact of that work is incredibly difficult.
Us data people can get a bad rap as bean-counters who think that everything that counts can be counted. You heard it here: that’s a lie. In nonprofits as in all of life, quantitative measurement is not the end-all-be-all. The work of making a difference in another human being’s life can’t be fully reduced to numbers.
Some of you need to hear this because you’ve had a lingering suspicion that all this talk of “data storytelling” is really just an unhealthy fixation on statistics. If that’s you, breathe deep. We’ve got some balance here.
And yet, if you stop reading here, you’ll have missed the entire point. Cuz guess what: as much as it can’t be the whole story, data is indispensable if we want to have confidence we’re actually helping people. One reason data can be an afterthought is that, too often, nonprofit leaders take for granted that their organizations are making a difference.
Here’s the reality:
Many of us nonprofit leaders do not spend enough time thinking about — no, obsessing about — whether our organization is actually helping people.
Rather, we hide behind the inherent difficulty of nonprofit impact measurement, and we settle for almost-answers.
What are almost-answers? They’re anything in the ballpark of the question that passes as an answer on its surface, but fails on closer scrutiny. To give an everyday example: if I ask you, “How are you?”, and you tell me, “I’m so busy!”, the conversation may move on; but if I stop to think about it more, I realize that you didn’t really answer my question. You told me how full your schedule is, but that still leaves me wondering, in the midst of your busyness: how are you?
Far too often, nonprofits believe they’re measuring and reporting on participant impact, but they’re not; rather, they’re giving almost-answers which belong to other questions:
To what extent are you serving who you said you would serve? (participant fit)
To what extent are you doing what you said you would do? (program outputs)
To what extent are you documenting things correctly? (reporting compliance)
Don’t get me wrong: these are important questions. But the reality is that none of these questions gets at participant life change.
Besides leaning on the crutch of almost-answers, we also settle for anecdotes. Of course we’re helping people — “just look at Michael!”, or, “just look at Maria!”. But without data, we have no way of knowing that the experience of Michael or Maria or any other individual participant represents our broader service population. So while participant stories are powerful, anecdotes can’t give us confidence that we’re actually helping people.
What does this mean? It means we could have heartstrings-tugging anecdotes, and stellar almost-answers on things like participant fit, program outputs, and reporting compliance, and still fail to actually help people. It means we could be doing all this work and not be making the difference we claim to be making. If there’s something that deserves to keep us up at night, this is it.
By now, I hope you’re wondering:
How can data give us nonprofit leaders confidence we’re actually helping people?
Great question — we’ll tackle that next time in Part II.
