Transformational Leadership and Continuous Improvement

There’s a lot of interesting reading in the 2024 DORA survey (https://dora.dev/survey/). For those unfamiliar with DORA, they started out as the DevOps Research Agency and are now part of Google and conduct research into effective development practices. They conduct an annual survey which polls tens of thousands of people worldwide on capabilities and practices of tech-driven organisations.

Much of the 2024 survey report is focused on AI, which is obviously the hot topic of 2024, and something I’ll come back to in future pieces.

What I want to focus on in this post was sparked by the following extract from the report:

What we’ve seen consistently over the last 10 years is that transformation is a requirement for success. What many organizations misunderstand is that transformation isn’t a destination, but a journey of continuous improvement.

The DORA survey findings point to a relationship between transformational leadership and improved organisational improvement. The report defines transformational leadership as having the following characteristics:

• Vision - holding a clear vision of where the team and organisation are going.

• Inspirational Communication - saying positive things about the team, make employees proud to be part of the organisation; encouraging people to see changing conditions as situations full of opportunities.

• Intellectual Stimulation - challenging team members to think about old problems in new ways and to rethink some of their basic assumptions about their work.

• Supportive leadership - considering others personal feelings before acting; behave in a manner which is thoughtful of others personal needs.

• Personal recognition - commending team members when they do a better-than-average job; acknowledging improvement in quality of team members’ work.

The survey finds that an increase of 25% in transformational leadership leads to a 9% increase in employee productivity. I’m still trying to figure out whether the message here is that for every point of improvement in team effectiveness you need 3 times as much improvement in leadership effectiveness. I’m not sure statistics work that way, but let’s say they do and this shall hereby be known as “Spragg’s Law of Leadership Leverage”. (That’s how these things work, right?)

DORA has also found that good leadership can also lead to:

• decrease in employee burnout

• increase in job satisfaction

• increase in team performance

• improved product performance

• improved organisational performance

This is backed up by their research since 2017, with statistically significant relationship between the scores of leaders across those five characteristics and the performance of their teams. This is also reflected in the Employee Net Promoter Scores (eNPS), i.e. how likely someone is to recommend working at a company.

The takeaway is that high performance is enabled by transformative leadership as defined by these behaviours, so to be a better leader, improve on those.

The findings by DORA are backed up by research by other groups. For example, a recent study conducted in South Korea found that people who actually have the organisational support to act on transformational leadership are more innovative and committed to change than those who don’t. That seems so blindingly obvious it shouldn’t need a study, but then there’s human nature to consider.

To state things simply, change requires time and resources. Leaders need to make budget available for improvement, not expect people to learn new things and address limitations in their downtime, and these activities need to be baked into the schedule. There are lots of different ways to do this, but ultimately it needs to be in the form of an official budget for time and money, otherwise it’s just wishful thinking.

In addition to the transformative leadership behaviours, leaders can also enable the adoption of technical and product management capabilities and practices. In the context of software delivery these are also correlated with high performance.

Enabling these practices looks like the following:

• delegating authority and autonomy to teams

• providing them the metrics and information needed to solve problems

• creating incentive structures around value delivery as opposed to feature delivery.

I initially struggled with the word transformation in an organisational context, as I initially inferred it to mean something radical and immediate. But it doesn’t need to be either, it just needs to result in something markedly different.

One organisation I’ve observed is currently going through their third agile transformation, and they currently look exactly the same as prior to their first. They are not the only one.

A common characteristic of those non-transformations is that of a long programme of work culminating in a sudden change on an “Agile Day” with a declaration that the organisation is now transformed.

These highly staged transformations are generally not successful, as often they focus changing job titles and reporting lines that are only superficially different. They don’t result in changed behaviours and practices. The leaders in the organisations often exhibit the first two characteristics of transformational leadership - a vision and inspirational communications - but lack the follow through.

Actual transformation is more like Netflix evolving from mailing DVDs to digital streaming, to production of its own content. Having started posting DVDs in 1998, they moved into streaming in 2007, and the organisation didn’t separate the two until 2011, and continued to offer DVDs until 2023!

Netflix is a big, macro-scale example, but the pattern can be observed in all sorts of places and at all sorts of scale. And given enough time, the accumulation of all these changes can be something radical.

The lesson is clear: successful transformation is more likely to be achieved through a gradual process of continuous improvement than through a grand management mandated big-bang.

The Improvement Kata is something that I have encouraged my teams to adopt and is backed up by the DORA research to be an effective path to transformation.

Codified by Mike Rother in the “Toyota Kata”, the method involves solving problems by applying a four step approach:

1. Understand the direction or challenge.

2. Grasp the current condition.

3. Define the target condition.

4. Iterate toward the target condition through experimentation

The Improvement Kata is supported through the Coaching Kata, which aims to support continuous improvement efforts through effective coaching. This is another way that a transformational leader can support their teams.

Coaching can unlock teams, but helping them move think through and plan their steps:

• What is your goal state?

• What is your current state?

• What obstacles are preventing you from reaching the goal state?

• What is your next step?

• When can we see what we have learned from taking this step?

At one of my previous organisations, we experimented with an experiment template to help teams define a hypothesis about how to move how to move to the target condition. It was initially useful to help support people’s thinking, and ultimately got absorbed into existing tools and processes to the point that teams had the kata embedded into their regular practices.

The impact was profound. One team saw their release frequency improve from once every three months to every two weeks. The increase in value delivered was seen in business results.

This stuff works.