S6E10 - Why CRO “Best Practices” Keep Failing (And AI Might Make It Worse)

In this episode, we dive into CRO “best practices” that completely failed in real-world experimentation. From homepage carousels unexpectedly winning tests to why most experimenters still don’t trust AI-generated CRO workflows, we unpack the messy reality behind optimization advice.

We also debate synthetic AI users for research, discuss the most useful survey questions in CRO, roast dark-mode websites, and explain why emotional branding can sometimes hurt usability.

Join our slack: https://join.slack.com/t/experiment-nation/shared_invite/zt-3zowejvoj-ujS_tbqfXcPkS9O0rnWhTQ

Topics include:

Why homepage carousels occasionally win
AI-powered A/B testing platforms
Synthetic users & fake research data
Best and worst CRO survey questions
Dark mode website UX
Why qualitative context still matters

Join the Experiment Nation Slack community to participate in future discussions and question threads.

Chapters
00:00 Intro & CRO community questions
01:46 CRO best practices that failed in real life
02:28 Do homepage carousels actually work?
05:02 Hero videos, agencies & emotional design
06:25 When carousels might make sense
08:24 Should AI run the entire CRO workflow?
11:48 Why experimenters still don’t trust AI
14:49 Best & worst survey questions in CRO
17:30 Emotional branding vs usable websites
18:26 Dark mode websites & readability debate
21:29 Synthetic AI users for testing & research
23:56 Why fake users can’t mimic real humans
26:32 Using AI as a research starting point
28:41 Closing thoughts & Slack community invite