Scattered notes on (retro) computing, data analysis, statistics, SEO, and things that change
It happens with every reasonably serious project: you export the keyword list from Search Console or a tool, and you find yourself facing thousands of rows. Three thousand, ten thousand queries. Reading them one by one is unthinkable, and grouping them by hand by feel is slow, subjective and impossible to reproduce.Yet we need that Continue reading "Keyword Clustering: grouping thousands of…
Anyone who spends their days inside Search Console knows that little nagging feeling: a page sits steadily in third position, yet the clicks are few, a CTR that looks like it belongs at the bottom of the page.The question we usually ask is the wrong one: not how many clicks does it get? , but the Continue reading "Expected vs Actual CTR: finding the pages that earn fewer clicks than their position…
In the article on the multi-armed bandit we used Bayes to decide between variants: shifting traffic toward the one converting best while the test is still running. Now we take a step sideways, while staying within the same line of reasoning: instead of choosing between options, we want to classify, that is to attach to Continue reading "Naive Bayes: classifying search intent with Bayes theorem"
In the article on Bayesian A/B testing we compared two variants at a fixed sample size: we collect the data for the whole planned duration, compute the probability that B beats A, and decide. It is a solid method, but it carries a cost that usually goes unmentioned. That cost is the traffic that, for Continue reading "Multi-armed bandit: optimising the variants while the test is still running"
In the article on classic A/B testing we saw how to compare two variants with the two-proportion test: we compute a statistic, get a p-value, and decide whether to reject the null hypothesis. It works, and it is the daily bread of anyone running online experiments. But there is a subtle gap between what the Continue reading "Bayesian A/B Testing: not just whether B beats A, but by how much "
In the article on the foundations of Bayesian statistics, we saw how Bayesian updating works through simulation: generate samples from the prior, simulate data, filter. An intuitive method, but one that runs into a practical limit as soon as data becomes even slightly numerous. In this article we move to the elegant analytical solution that Continue reading "Bayesian Conversion Rate Estimation:…
On 21 January 2015 Optimizely — one of the most widely used A/B testing platforms in the world — switched on a completely new statistical engine for all of its customers, the New Stats Engine. It wasn t a technical whim: the old engine, built around a classic fixed-horizon t-test (Fixed Horizon) and developed with statisticians Continue reading "The peeking problem: why sneaking a look at an A/B…
In the Israeli Air Force, Daniel Kahneman recounts, the flight instructors were sure of one thing: praising a cadet after an excellent manoeuvre made him worse, scolding him after a terrible one made him better. They had seen it happen a thousand times in the field, so it had to be true: with pilots, severity Continue reading "Regression to the Mean: the SEO Fix That Worked… by Accident"
In this article: What an A/B test is Setting up an A/B test correctly Worked example: conversion rate of two landing pages The most common mistakes Frequentist vs Bayesian approach Practical SEO example: meta description A/B test FAQ Try it yourself Over the previous articles we have looked at how hypothesis testing works and how Continue reading "A/B Testing: How to Run Statistically Valid…
In this article: What PCA is The mathematical foundations Practical applications PCA in web marketing Implementing PCA in R Verification and interpretation FAQ SEO and web marketing analysis almost always presents us with the same problem: we have too many metrics and we don t know which ones really matter. Search volume, CPC, competition, CTR, bounce Continue reading "PCA (Principal Component…