Simpson’s Paradox

Organizations increasingly rely on:

• dashboards

• analytics

• KPIs

• performance summaries

And often, the conclusions appear straightforward.

Until someone segments the data differently.

Suddenly:

• the trend changes

• the conclusion weakens

• the “obvious” interpretation no longer holds

Why?

In statistics and decision science, this is explained through:

Simpson’s Paradox (Edward H. Simpson, 1972)

Simpson’s Paradox occurs when:

A trend visible in aggregated data disappears or reverses when the data is divided into subgroups.

Which means:

averages can hide important structural patterns underneath.

When evaluating performance data, ask:

1️. What does the aggregate metric suggest?

(overall growth, conversion, profitability, engagement, etc.)

2️. What happens when the data is segmented?

(region, product line, customer type, time period, demographic)

3️. Are hidden variables influencing the result?

(mix effects, market conditions, pricing differences, customer composition)

4️. Does the conclusion remain consistent across subgroups?

(stable trend vs contradictory patterns)

This shifts analytical thinking from:

“What does the average show?”
to
“What structure exists beneath the average?”

Let’s take Spotify.

Imagine leadership evaluates overall user engagement across the platform.

The aggregate data shows:

average listening hours per user are declining slightly.

At first glance, leadership may conclude:

user engagement is weakening overall.

But segmented analysis reveals something very different.

1️. What does the aggregate metric suggest?

Overall listening hours per user appear to be decreasing.

2️. What happens when the data is segmented?

When broken down by user type:

• premium subscribers are actually increasing listening time

• engagement decline is occurring primarily among newer free-tier users

3️. Are hidden variables influencing the result?

Yes.

The platform has recently added millions of new users from emerging markets with:

• lower average listening time

• different usage behavior

• lower monetization levels

This changes the overall average significantly.

4️. Does the conclusion remain consistent across subgroups?

No.

Core high-value users remain strongly engaged even while aggregate engagement metrics weaken.

The key insight is:

The aggregate metric concealed important differences in user behavior and customer composition.

Without segmentation, leadership might incorrectly:

• change product priorities

• misjudge retention quality

• misunderstand customer value dynamics

Strong leaders understand that:

• averages compress complexity

• aggregate metrics simplify reality

• data interpretation depends heavily on structure and segmentation

They ask:

What patterns become visible once the data is broken apart.