Menu

PBO, Explained in Plain English: The Probability Your Best Backtest Is a Fluke
📰

PBO, Explained in Plain English: The Probability Your Best Backtest Is a Fluke

DEV Community·Dron Pancholi·16 days ago
#XIeiTK6D
#dev#winner#sample#strategy#backtest
Reading 0:00
15s threshold

You tested 8 strategies and one looks brilliant. PBO asks the question most backtests never face: among everything you tried, how likely is the winner just the luckiest loser?

You ran 8 versions of a strategy. Seven were mediocre. One was brilliant. You ship the brilliant one. Question: did you find an edge, or did you run a lottery and crown the winning ticket?

That is exactly what PBO — the Probability of Backtest Overfitting — measures. And it is one of the least explained ideas in trading, because the people who understand it write papers, and the people who write blog posts mostly skip it. Here it is in plain English, with a worked example.

The core idea

PBO does not ask “is this strategy good?” It asks something sneakier: if you pick the best performer from a set of candidates, how likely is it to disappoint out of sample?…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.