Trust in the Age of AI

One of the most impactful pieces of advice was given to me by the inimitable Steph Cooper: “You Move at the speed of trust.” It that Stephen M. R. Covey gets credit for popularising the phrase, but Steph landed it with me.

When trust is high, decisions happen quickly and efficiently. When trust is low, everything slows down—more meetings, more approval layers, more verification steps. Trust, it turns out, is often the critical constraint that determines how fast teams, organisations, and markets can move.

It’s something I’ve come back to time and again, and observe frequently as being the root of teams and businesses struggling to meet their own expectations.

In the age of AI, this constraint will continue to be true, and is just going to get more complicated. We're won’t move at the speed of processing power, data availability, or algorithmic sophistication. We will move at the speed of trust.

KPMG recently published a global study into Trust, attitudes and use of Artificial Intelligence. New Zealand, my home, is generally to be found near the bottom of the charts, with low levels of trust and acceptance. I would like to think that this reflects a healthy level of scepticism and critical thinking among my fellow kiwis. Who’s to say?

When I observe teams grappling with how to work together I can’t help wonder about how we can successfully incorporate emerging, non-deterministic technology into how we do things.

AI capabilities are advancing exponentially. Large language models can write code, analyse data, generate creative content, and make recommendations across virtually any domain. AI agents can book flights, manage calendars, and negotiate on our behalf. The technology exists today to automate vast swaths of knowledge work.

Yet adoption remains surprisingly uneven. Some AI applications—like coding assistants—see rapid adoption among developers. Others—like AI financial advisors or autonomous hiring systems—face significant resistance.

The difference isn't necessarily capability as much as it is trust.

Consider coding agents such as GitHub Copilot. Developers don’t adopt it because it writes perfect code, because it doesn't. They are adopting it because it fits well into their existing trust verification processes of code review and testing. When Copilot suggests code, I can read it, understand it, test it, and validate that it does what I expect. The AI generates, but I verify. Trust builds through observable behaviour, not claimed capability.

I was recently building a small utility to convert text notes into a different format, using Cursor to do the heavy lifting. It got itself into a wicked loop trying to apply the correct formatting. Putting aside the fact that Python is silly, this demonstrates a pitfall of “autonomous systems”.

Given that nonsense, how can I possible trust an AI agent to do anything? Because I give the coding agents very narrow constraints to operate in, observe what it is doing and use the well proven quality practices like of breaking work down into small chunks and wrapping tests around changes.

Contrast this with AI systems that make black-box recommendations about hiring, lending, or medical diagnoses. Even when these systems demonstrate statistical superiority to human judgment, adoption is going to lag because the trust verification mechanisms are unclear or absent. How do you "code review" an AI's recommendation to reject a loan application?

From a systems perspective, trust functions like any other bottleneck. When trust is the limiting factor, adopting AI capability isn't going to improve overall system performance, it will just create more inventory waiting at the trust checkpoint.

I predict some organisations will successfully integrate AI while others will struggle despite similar technical resources. The difference will lie in how they design for trust optimisation, not just capability optimisation.

Organisations that successfully navigate this transition will likely design AI integration around three key principles:

• Observable AI Behaviour: Like code review for developers, they will create workflows where AI contributions are visible and reviewable. AI research comes with source attribution. AI recommendations come with reasoning and alternatives. AI decisions come with audit trails.

• Graceful Degradation: AI systems will need to fail predictably and recoverably. Reference that repeated failure of the Cursor agent to apply the correct formatting. That went on for ages before it eventually gave up and told me I’d have to fix it myself. Is that optimal? Possibly not, but I rather it did that than kept going indefinitely, or got creative with the problem and deleted the file. Trust builds through consistent, understandable behaviour patterns.

• Complementary Competencies: Rather than replacing human judgment, they will design AI to augment human capabilities in ways that make sense and utilise the things that people are good at. AI handles data processing and pattern recognition; humans handle context, values, and edge cases.

My belief is that AI is a tool, not an end in itself. But the range of possible applications presents opportunities everywhere. Take the challenge of building software products and bringing the benefit of AI to the users of your product.

We are already seeing people over optimise for speed in the race to make the most of AI. Given how low the success rate is in software projects, it is not surprising that we’re already seeing some pretty shambolic applications of AI.

Building AI features has introduced new challenges such as training better models and engineering faster inference. But how do you build systems where trust can be built through the development process itself?

This suggests refining the approach to how teams build products, emphasising human-centred design focused on trust, rather than technical benchmarks:

Build trust verification into your development workflow. Instead of testing AI capability in isolation, test how users actually verify AI behaviour in practice. Run user research sessions focused on trust formation, not just task completion. Instrument your systems to measure trust indicators, not just performance metrics.

Prototype trust patterns before AI sophistication. Start with simple AI capabilities that users can easily verify and understand. Build the feedback loops, review processes, and verification mechanisms that let users develop confidence. Only then expand AI autonomy within those proven trust frameworks.

Develop AI alongside user mental models. Deploy AI features incrementally and observe how users actually build trust over time. What verification behaviours do they develop naturally? Where do they hesitate? Use these insights to inform both AI capability roadmaps and interface design decisions.

There is an excellent book by David Birch, “Identity is the New Money”, which ten years ago argued that in a digital world, reputation will become increasingly important in transacting. Historically you could transact with someone primarily on the basis of shared trust in a fiat currency. In a digital world, your reputation as a reliable buyer or seller becomes increasingly important.

The trust constraint becomes even more complex as we move toward AI agents that interact with other AI agents on our behalf. When your AI assistant negotiates with a vendor's AI agent, who arbitrates trust? How do autonomous agents build reputation? How do we verify that AI agents are representing our interests faithfully in interactions we never directly observe?

Answering these questions presents technical challenges, but there are a whole load of fundamentals to explore before a technology solution can be found, and I believe identity and reputation are key pillars. How do we establish trust to enable AI-mediated transactions? Are we entering an “identity economy”?

Understanding trust as the critical constraint changes how we approach AI integration at every level—from individual tool adoption to organisational transformation to societal AI governance.

The organisations and products that succeed in the AI age won't necessarily be those with the most sophisticated algorithms. I believe they'll be those that most effectively solve the trust optimisation problem.

After all, if AI is just another tool for people to take advantage of, you still move at the speed of trust.

I am exploring the theme of trust in the age of AI, finding more questions than answers at the moment. This is probably going to end up as a series of posts on the topic, but I’ll try to keep an element of practicality as a I go, and reserve the right to meander into other areas of software and product development. If you want to stay in touch or share with others who might be interested, I’d really appreciate it.

Share Building and Exploring