Menu

Two checks before you trust a backtest: one-stock luck and regime change
📰

Two checks before you trust a backtest: one-stock luck and regime change

DEV Community: python·Quantooox·5 days ago
#ehEFRjtp
#devtopython#devto#dev#average#contrib
Reading 0:00
15s threshold

A backtest's headline number is an average. Averages hide two common problems. Both have cheap checks.

1. Is it just one stock?

A portfolio of 20 names can look great because one of them went up tenfold. That isn't a strategy. That's one lucky pick, and it won't repeat.

The check: report what share of the total gain came from the top three names.

contrib = ( weights * moves ) . sum ( axis = 0 ) # each stock's total contribution top3_share = contrib . nlargest ( 3 ) . sum ( ) / contrib [ contrib > 0 ] . sum ( )

Enter fullscreen mode

Exit fullscreen mode

If most of the result comes from three names, you're looking at concentration, not skill. Our portfolio page shows this number for each of our hypothetical portfolios.

2. Is it just one year?

What worked in 2021 failed in 2022. Interest rates, inflation and what investors fear all change, and a model can be tuned, without anyone meaning to, to one kind of market.

The check: show results year by year, not one average.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.