Multi-armed bandit: optimising the variants while the test is still running

In the article on Bayesian A/B testing we compared two variants at a fixed sample size: we collect the data for the whole planned duration, compute the probability that B beats A, and decide. It is a solid method, but it carries a cost that usually goes unmentioned. That cost is the traffic that, for Continue reading "Multi-armed bandit: optimising the variants while the test is still running"