The Coin That Knows
One of these three machines pays out 70% of the time. You have 10 pulls to figure out which.
Go ahead — pull each arm. You'll see coins on wins, nothing on losses. The magenta one looks good so far. But is it actually the best, or just lucky?
This is the exploration-exploitation tradeoff, and it's the entire job of any system that learns from feedback. Every pull gives you two things: a reward, and information. You can't maximize both at once. Pull the machine that looks best — you exploit what you know. Pull a random one — you explore what you don't. The optimal strategy isn't either extreme. It's knowing when to switch.
Below each arm, you'll see a regret curve climbing. Regret is the cost of not knowing the best arm from the start. Every pull that isn't the 70% machine adds to it. Your job is to keep that curve as flat as possible.
Many Minds, One Problem
One agent is a gamble. Five agents, each with a different philosophy, racing in parallel — that's a system.
On the left, you're watching five agents attack the exact same bandit problem. Same arms. Same probabilities. Completely different strategies.
The Random agent just pulls uniformly — it's the baseline, the control. Epsilon-Greedy exploits 90% of the time and explores 10% — simple, and surprisingly effective, but it never stops wasting pulls on bad arms even after it's sure. UCB1 tracks confidence intervals — it picks arms based on how uncertain it is about them, not how good they look. Thompson Sampling is Bayesian: it maintains a probability distribution over which arm is best and samples from it — theoretically optimal, but computationally heavier. Optimistic Initialization starts believing every arm is incredible, then lets reality correct it downward.
Watch the regret curves. Thompson settles fastest on hard problems. UCB is more consistent on easy ones. Epsilon-Greedy climbs forever because it wastes that 10% forever. No single strategy dominates every bandit — which is exactly why you'd never deploy just one.
The Wisdom of the Swarm
Five agents, five opinions. The visual on the left shows how they converge to one decision.
Each agent has an estimate of which arm is best, and a confidence level to go with it. The question: how do you combine five independent judgments into a single collective decision?
Simple majority voting is fast — but it ignores the fact that an agent with 200 pulls has a much better estimate than one with 10. Weighted voting by confidence fixes that: more data = more influence. Trust-weighted consensus goes further — it tracks each agent's historical accuracy and adjusts its weight accordingly. An agent that was wrong last round gets less influence next round.
This isn't just math. Ant colonies use the same mechanism to find the shortest path to food. Individual ants lay pheromone trails. The colony follows the strongest signal. No ant knows the full picture — but the swarm doesn't need one. It converges faster than any individual because it doesn't just share information. It shares confidence.
Watch the voting rings. Watch the confidence bars grow. The convergence animation shows the moment a group of uncertain individuals becomes a single certain decision.
Play With It
Toggle strategies, crank the dials, run the race. The sandbox is below — and it's yours.
Pick any combination of the five strategies. Turn epsilon up. Turn UCB's confidence parameter down. Give Thompson more steps. Hit Run and watch the regret curves unfurl in real time.
The best way to understand why Thompson Sampling crushes Epsilon-Greedy on one problem but ties it on another is to make them fight it out yourself. Each run is a new experiment. Each experiment teaches you something about the shape of the problem — the bandit's difficulty, the strategy's blind spots, the value of diversity in your swarm.
Reset. Change one variable. Run again. That's the scientific method — applied to the design of collective intelligence.