Imagine you run an online experiment with several versions of a webpage, several drug doses, or several ad creatives. Each option pays off at an unknown rate, and every trial costs you a chance to earn from the best option you already know. Play it safe and you may lock in a mediocre choice forever. Explore forever and you waste trials on options you already suspect are weak. This is the multi-armed bandit problem, and Thompson sampling is the Bayesian answer that has quietly become a default in industry experimentation, recommender systems, and adaptive trials.

The idea is disarmingly simple: keep a posterior belief about each option, sample one plausible world from that belief, and act as if the sample were true. Options that might be best get played often. Options that are almost certainly poor get played rarely, but never with probability exactly zero while doubt remains. Exploration is not bolted on as a random coin flip.…