Well, this newsletter was supposed to go out the week of Christmas. Then - it was supposed to go out this Monday. The delay was mine. ALL MINE. It’s 6:58 pm on New Year’s Eve and I just finished my 4th Mapping GEO article - an interview with Ross Simmonds - and it was a doozy of technical challenges. (Thank you so much, to my dearest friend and partner Kate Murr, for everything you did to help bring it to the finish line! But more than that - for being so amazing.) With that - below is the newsletter we crafted for you.
Happy New Year. May your 2026 be amazing and bright.
If your family is like mine, recently at least four of you have said, “Clark, that’s the gift that keeps on giving the whole year.” If your family isn’t as into Christmas Vacation or the x-of-the-month club as mine is, that’s okay. You get it. Some gifts keep paying dividends.
As the year winds down, I’ve been thinking about the “gifts” that research and experimentation have given us: clarity, the ability to ask increasingly better questions, empathy for customers, humility, and adaptability. What gifts would you add to the list?
Topping my most profound professional gifts this year: the bounty I’m learning working on the Mapping GEO series. So far, I’ve learned from Alisa Scharf, Jono Alderson, Nick Haigler, and now Ross Simmonds.
After all these conversations, I’ve still had one nagging question: If algorithms are mediating discovery, why does so much of it still feel deeply human?
This month, Ross created a total lightbulb moment for me (especially in the era of agentic AI and LLMs). The online world isn’t as different from the analog one as I once thought.
Our conversation is about what has changed in the discovery process, what has stayed the same, and why this is important to experimenters and brands. Spoiler:
Listening still beats guessing.
Trust still does the heavy lifting.
Distribution isn’t optional just because you wrote something good.
But the consequences of ignoring these fundamentals are higher today than ever. Learn with me why designing for humans still wins and what that even means.
🎁Special content this month includes a three-part video series investigating AI attention mechanics from Kat Ribant of Ask-Y and seven steps to gain the competitive advantage through experimentation from Luke Sonnet of GrowthBook.
I’ve screamed at my AI. You probably have, too.
Five corrections later, it’s still giving me the same useless answer. Or it “forgets” the one number I actually need from a conversation we’ve been having for twenty minutes. Or it keeps rephrasing the same generic response, no matter how I ask the question.
Turns out, it’s not being difficult. It’s attention mechanics—and once I understood how they work, everything changed.
I made a video series breaking down the three problems that were driving me crazy: attention dilution (why details disappear in long chats), context poisoning (why corrections make things worse), and mode collapse (why the AI gets stuck repeating the same kind of answer). Each one comes with fixes I now use every day.
If you’re building prompts with real data and business logic, this isn’t nice-to-know stuff. It’s how you stop fighting and start getting actual results.
Watch the series and have a Happy New Year!
Only 10-30% of experiments produce a clear winner—and that’s not a problem, it’s a reality. This article shows how high-performing teams design experiments to learn faster and make better decisions, even when results are neutral. From clearly defining the experiment’s goal to deciding how long to run it, learn the vital steps that turn experimentation into a real competitive advantage. Read the full article here.
What began as a concept for a rockabilly, Elvis-like, kinda punk song about Santa being too data-driven that it spirals out of control ended up being Josh Silverbauer’s third annual Christmas songs! Listen here!
No Hacks has been 🔥. Jes Scholz, SEO futurist and growth consultant, joined Sani Manic for a don’t-miss conversion, “AI Agents Are Here (And They Hate Your Website).” Then, Joe Doveton and Sani talked about the urgent realities facing brands that rely on web traffic in “Why AI is Killing your Clicks: The New Metrics for a Zero-Click World.” Finally, Sani wrapped up 2025 in an episode beginning with ten seconds of radical silence. Check it out.
A new TLC Slack channel was created at the recommendation of Geoffrey Wortham —The #tlc-recipe-book! People have shared recipes from all over the world. Delicious! (Including Kelly’s family favorite - HAND CAKE!!!)
More video content! Katrin Ribant has made three animated explainer flicks that explain why you’re yelling at your LLM: Mode Collapse, Context Poisoning, and Attention Dilution.
Luke Sonnet shares the patterns he sees that cause experimentation to fail and how to design experiments to avoid them.
We grappled to understand whether GO is back. People used face melt emojis. Dennis van der Heijden buckled down and wrote some interesting content. Carmel Makaya explains further in this video.
Cory Underwood, AKA Dr. Doom, shares the latest CCPA requirements re. onboarding new tools and collecting/sharing/selling personal data in 2026.
Jason Packer wanted to definitively prove that OpenAI was scraping, and so he set up an old-fashioned SEO sting operation showing that, without question, OpenAI was indeed scraping Google SERPs. Is this the beginning of how we nail down a line between benign and malicious scraping? Don’t miss this read.
Have you joined the convert-sation yet? We hope you do. It’s a great place to engage and learn more about fellow TLC members.
Well, we did it y’all! This month wrapped up a YEAR of convert-sations with the Convert crew and your peers! Did you learn more about each other? I certainly did! And great news! The folks at Convert have signed on for 2026, so we get to keep on learning about each other in this fun, engaging way.
This is a really special community effort that takes creativity, time, and engagement. I can’t wait to start all over again next year!
This month, as we prepared for our festive traditions, our Convert-sations were rockin’ around the rituals and traditions in our experimentation work. While my dreams of creating an analyst’s advent calendar were not realized, we did share tips and solutions for code freezes, collaborative work, and rituals we love. Here are this month’s winners.
🥇1st Place: Merritt Aho (TLC: @merrittaho ) Merritt answered the question: “What’s one unique ritual your team does, and others should consider?”
“Prior to sharing exp results broadly, we assess the likelihood that a test result will be problematic for any of the stakeholders. If it will, we take the time to walk them through the result and respond to questions prior to sharing with others. This sometimes creates more analysis work, but always makes sharing and actioning easier. This is in a setting where we present exp results to a fairly large group on a regular basis. ‘Pre-wiring’ for short.”
🥈2nd Place: Eddie Aguilar (TLC: @Eddie Aguilar) Eddie replied to a question about: “How often do you revisit and evolve your collaborative rituals so that they truly add value?”
“I used to sit on calls all day with specific clients, while they watched me just to build that trust. It was also a multi-billion-dollar corporation I was managing engineering and optimization for. So when I first joined, that trust wasn’t there, and it’s a bit hard to really build it so far away. So we would “pair program” while the experiments got built, then QA together, then launch together live. That then became weekly, to monthly, to quarterly, then eventually H1 and H2 needed only. While it’s not the most optimal way, it helped early on to build trust with some of the biggest names in the industry for me, which then led to more effective collaboration. Pair program, even if it’s async, just allow time to sit with the POC to build trust in each other’s work styles, and make effective collaboration happen.”
🥉3rd Place: Florent Buisson (TLC: @florent) Florent shared a great reminder on what small habits make you a better or more consistent experimenter:
“Not original, but ‘leave the place cleaner than you found it.’ It’s easy to get caught in the day-to-day and always postpone for doing later, a massive refactoring (do we ever though?). But you can also identify and improve/fix just one little thing in the code/data at a time. It’s also a good opportunity to sharpen those critical thinking and architectural skills. Sometimes, I literally just add a single test in dbt, such as “this variable should never be null”, and a couple of times this has triggered errors and warnings that have allowed me to catch bugs before anybody even complains.”
We urge you to hop on to the #convert-sations Slack channel to see how other TLC members weighed in — and share your thoughts for next month’s questions!
Jan 23: TLC Conversation: Build Your Experimentation Flywheel & Good CRO, Bad CRO Murder Mystery! with Dzifa Mensah & Sarah Cridland.
Feb 24-26: The Conference known as Experimentation island - Tickets starting at $1750 still available! (in-house teams save!)
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.