How objective is PIE as a prioritisation framework?

The PIE framework gets a bad rep. Critics often dismiss it as too subjective to be useful for test prioritisation.

But is this criticism fair, or is PIE just misunderstood?

PIE stands for Potential, Importance, and Ease. It’s a prioritisation framework. We spoke about these generally in our previous episode, in case you don’t know.

In terms of what causes this misunderstanding, I think it’s the first criterion, Potential, that catches most of the flak. The definition is “how much improvement can be made on a particular page,” which on the surface sounds pretty subjective.

But here’s the thing: that feeling of subjectivity comes from reading the definition without understanding the actual process behind it.

Chris Goward, who created the PIE framework (and the wider LIFT model), goes much deeper in his book You Should Test That and especially in his workshops. The framework isn’t meant to be:

“oh, well I think this probably scores XYZ” — Iqbal

Instead, Goward advocates for a rigorous, objective approach to scoring. Before you even start A/B testing, you should:

1. Audit all your test pages and areas

2. Rank them using measurable data

3. Use proxies like drop-off rate to quantify potential

For example:

“if a page has got high drop-off rate, then there’s a lot of improvement that can be made, and that can be quite objective.” — Iqbal

and:

“if a page has a 90% conversion rate, well then the drop-off rate is only 10%, which isn’t that high” — Matt

Think of it as identifying the gap between current performance and where the page could/should be performing. For example, a checkout page with 10 friction points discovered through usability testing has more potential than a cart page with only 2 friction points.

If the problem is that many using PIE, apply it in a subjective way, then the problem isn’t PIE itself, it’s that the framework requires work to apply properly.

To avoid this subjectivity trap, you need to:

• Conduct qualitative research

• Run user testing sessions

• Collect user feedback

• Analyze quantitative metrics

• Build an evidence base for your scores

PIE isn’t inherently subjective, but it does require you to bring objectivity to it. The framework itself is just a container. What you fill it with determines whether it’s useful or not.

The dissonance should be obvious: if you’re running A/B tests to make objective decisions, why would you use a subjective system to decide which tests to run?

So, do you use PIE? How do you make sure your test prioritisation is objective? Let us know in the comments.