Scalping: Optimized backtesting, a successful strategy?

[u/turtlemaster1993]

I have optimized roughly 15 scalping strategies on the past 20 days worth of data for a stock, The backtesting is on those same days and I have selected the best performer. Obviously I can’t expect it to perform the same as the backtesting on the next week but should I expect it to fail altogether? Would a better approach be to save the last 5 days for backtesting and optimize on the 20 days prior to those? How do you guys separate your data for optimization and testing? What other approaches are there?

Edit: using 1-min data

[u/Skytwins14]

Everyone is telling you need a few years worth of data, but I disagree. I also am using the last 20-30 days worth for testing intraday strategies, since the market situation with all this votality coming from the actions of the president is not normal. A strategy needs to be able to cope with sudden price jumps and testing it on a broader timeframe could hide weakness in high votality environments.

When testing on a smaller timeframe you need a good amount of trades and filter outliers to have a meaningful statistical analysis. To test for overfitting what I like to do is randomly change some parameters a few percentage points. If your backtest suddenly changes drastically you know that these parameters were hyperoptimized.

[u/Aurelionelx → u/Skytwins14]

God damn it I should have kept scrolling before writing my comment. Absolutely, everyone is obsessed with time but what is really important is the number of trades and economic rationale.