The overfitting problem in TradingView backtests

If you've ever published a TradingView strategy and seen a 400% return on a 6-month backtest, congratulations — you've probably built a curve-fitter, not a strategy.

This post is about why we built Deepwick. We'll start with the disease, not the cure.

What backtests actually measure

A TradingView backtest is a single-sample evaluation of your strategy on a specific historical window. It says: on this slice of the past, with these exact parameters, this is the equity curve you would have seen.

That's a fine statement of fact. The problem is what traders then read into it.

When you see Net Profit: 412%, Win Rate: 68%, Max Drawdown: 7%, your brain does the same thing it does with a fitted regression line: it sees signal. It assumes that if the past had looked similar enough, the future will too. And the TradingView Strategy Tester doesn't help you disabuse yourself of that notion — it shows you one number, not the distribution.

Three failure modes the Strategy Tester hides

1. Selection bias from parameter search

You ran the optimisation, picked the parameters with the highest Sharpe ratio, then reported that as your strategy's performance. The number you reported is biased upward — it was selected because it happened to fit that window. The expected out-of-sample performance of those same parameters is almost always worse than the in-sample number suggests, and often by a lot.

This is the textbook deflated Sharpe ratio problem. Bailey and López de Prado's 2014 paper showed that with even a modest number of parameter combinations tested against the same data, the probability that the best one was a true edge rather than noise drops dramatically.

2. Regime collapse

Your backtest window happens to contain a trending regime. Your exit logic is perfect for trends and lethal in ranges. In-sample, you got the trends. Out-of-sample, you got the chop. This isn't a parameter problem — it's a sample problem. The Strategy Tester shows you one window. It can't tell you how stable the strategy is across regimes.

3. Indicator-then-strategy data snooping

You spent six months coding an indicator, watched it work on historical charts, then built a strategy around it. You backtested the strategy. Now you're double-counting: the strategy is fitted to data the indicator was already inspected against. The Strategy Tester has no idea this happened.

Monte Carlo doesn't save you

The popular defence against these failure modes is Monte Carlo simulation. Trade shuffling, randomised paths, etc. Monte Carlo is great at estimating the expected distribution of returns under the null hypothesis that your strategy has no edge.

But Monte Carlo only tests the future you don't know. It tells you whether your in-sample number is plausible given random trade ordering. It does not tell you whether the strategy itself was cherry-picked from a search. That's a fundamentally different question, and it's the one that actually bites retail traders.

Deepwick doesn't optimise strategies yet; optimisation is on the roadmap, and the checks below are the ones it will have to report. What it does today is make a single backtest harder to fool yourself with: market orders fill at the next bar's open, stops and limits fill in a fixed order inside the bar, and every fill pays spread, slippage and commission.

What to do about it today

You don't need Deepwick to be less wrong tomorrow. Three habits help immediately:

1. Always report the number of parameter combinations tested. If you ran 500 parameter sets and picked the best, your Sharpe is inflated by a known amount. Subtract it before you decide anything.

1. Always split your data before you start. Decide on the train/test split before you write the strategy, then never look at the test set until you're done optimising. If you peeked, the number is contaminated.

1. Always check parameter plateau shape. Plot your out-of-sample metric across the full parameter grid. If the best parameter sits on a sharp peak with neighbours far worse, you almost certainly overfit. If the plateau is wide, the strategy is more robust.

None of this is novel. All of it is ignored. Deepwick exists because the tools traders use every day make ignoring it easy.

— Enrique