Data with a soul

When most people hear the word data, they imagine spreadsheets, dashboards and annual reports. Few imagine compassion. Yet behind every social impact statistic sits a person. Every percentage represents a family. Every graph is really a story about human lives.

Good data does not replace humanity. It helps us honour it.

One of the great challenges facing philanthropy today is not a shortage of generosity. Around the world, billions of dollars are invested every year by governments, foundations, corporations and individual donors trying to make society better. The challenge is knowing whether those good intentions are actually making the difference we hope they are.

That requires something the charitable sector sometimes finds uncomfortable: clear evidence. This was a core theme of my recent conversation with Tom Steele from the Centre for Evidence and Implementation as part of The Giving Habitpodcast series. That conversation informs this week’s article.

Social impact organisations are often built by extraordinary people. They see injustice. They act. They build something from nothing. That entrepreneurial energy is one of the sector’s greatest strengths. It can also become one of its blind spots.

When an organisation has invested years of effort into a programme, recruited passionate staff, attracted volunteers and built strong community relationships, it becomes emotionally attached to its own solution. That is entirely understandable. But emotion creates risk.

The uncomfortable question every organisation eventually has to ask is not: “Did we work hard?” It is: “Did we actually make the difference we intended?”

Those are very different questions. Busyness is not impact. Outputs are not outcomes. Good intentions are not evidence either. Sometimes the bravest thing an organisation can do is discover that something it believed was working... isn’t.

There is, of course, an opposite danger. Some organisations become obsessed with measurement itself. Every activity generates another metric. Every conversation becomes another KPI. Every human interaction is reduced to a number. This is not evidence. It is administration.

Data without context can become as misleading as anecdotes without evidence. The best organisations understand that numbers and stories need each other. The numbers tell us whether change is happening. The stories help us understand why. One without the other leaves us partially blind.

Perhaps the most interesting shift that happens across the social sector is when we move beyond asking whether a programme works. Instead, more seasoned practitioners ask: How did it work? Why did it work? For whom did it work? Under what conditions did it work? And equally importantly... For whom did it not work?

These questions matter because social problems are rarely simple. An education programme that transforms one community may fail somewhere else. A mental health intervention that succeeds in one demographic may struggle in another. A pilot that performs brilliantly with twenty participants may collapse when expanded to two thousand.

Scale has a habit of exposing assumptions. In engineering, adding ten extra users may change very little. In social impact, every additional community introduces different cultures, different leadership, different capabilities and different relationships. Human systems are wonderfully complex. Evidence needs to respect that complexity rather than pretending it doesn’t exist.

Every leader has favourite ideas. Every donor has preferred causes. Every foundation develops investment theses. None of that is inherently problematic. The danger arrives when evidence becomes something we seek only to validate decisions we have already made.

Confirmation bias is not confined to politics or business. It exists just as readily inside philanthropy. How often do we commission evaluations hoping they will confirm success rather than reveal learning? How often do annual reports quietly celebrate activity while avoiding uncomfortable findings? How often do funding decisions reward confidence more than curiosity?

Integrity begins the moment we become genuinely willing to discover we were wrong. That is difficult. It is also liberating. Because learning organisations ultimately outperform defensive ones.

Perhaps the greatest tragedy in philanthropy is not failed programmes. It is hidden failure. When organisations fear losing funding, they naturally become reluctant to discuss what hasn’t worked. When funders demand certainty, grantees become experts at presenting certainty. Everyone becomes trapped in a system where success is performative rather than examined.

The irony is painful. Failure contains some of the richest evidence available. If we genuinely care about impact, we should reward thoughtful learning just as much as successful delivery. Imagine a funding culture where organisations felt psychologically safe enough to say: “We tried this.” Or “It didn’t achieve what we expected.” Or “Here’s what we learned.” Or even “Here’s how we’ll improve.”

That conversation would strengthen the entire ecosystem. Instead, many organisations understandably feel compelled to present polished narratives that unintentionally reduce learning for everyone else.

Good evaluation is sometimes portrayed as something done to organisations. In reality, the best evaluation is something done for them. It creates clarity. It helps leaders allocate scarce resources. It protects staff from burnout by identifying what genuinely creates value. It helps funders invest more intelligently.

Most importantly, it respects the communities organisations exist to serve. If we ask people to trust us with their lives, their stories and often their deepest vulnerabilities, we owe them more than hope. We owe them honesty. That means measuring what matters. Not because donors demand it. Because communities deserve it.

One encouraging trend is that meaningful evaluation is no longer reserved for large institutions with dedicated research teams. Every organisation, regardless of size, can begin asking better questions.

Who are we trying to help? What change are we hoping to create? How will we know if that change actually happened? What assumptions are we making? What would convince us we need to change course?

Those questions, and the work that goes into developing a robust theory of change, cost almost nothing. Yet they fundamentally reshape how organisations think. Evidence is not about creating thicker reports. It is about cultivating deeper curiosity.

Funders have an important role here too. Sometimes we unintentionally create the very behaviours we later criticise. If reporting requirements become exercises in compliance rather than learning, organisations will naturally optimise for compliance. If funding cycles are too short, genuine evaluation becomes almost impossible. If grants pay only for programme delivery while refusing to fund measurement and learning, we should not be surprised when evidence remains weak. If money is given is restricted or constrained through the grant making process, we should not be surprised is innovation is stifled.

Perhaps one of the most valuable investments philanthropy can make is not another programme. It is building the capability of organisations to test, learn and better understand their own impact. Not because data is fashionable, but because wisdom compounds. Every lesson learned by one organisation becomes knowledge that benefits hundreds of others.

If you lead an organisation, are you measuring what is easiest, or what matters most? If you are a donor, do you reward certainty, or do you reward learning? If you are a board member, when was the last time you asked not simply whether a programme succeeded, but why? If you are a volunteer, do you understand the change your organisation is actually trying to create? If you are building something new, have you fallen in love with your solution, or are you still deeply curious about the problem?

And perhaps the biggest question of all. If the evidence showed that your favourite programme wasn’t working... Would you have the courage to change it?

That is where integrity begins. Not with perfect data. Not with perfect certainty. But with the humility to follow the evidence wherever it leads, while never forgetting that every data point represents a human life. Because the very best evidence is not cold. It has a soul.

To learn more about the Centre for Evidence and Implementation click here.

To watch my full interview with Tom Steele click here.