TL;DR: A backtest applies a precisely defined rule to historical prices and records the hypothetical trades. Its most useful output is expectancy: across 100 hypothetical trades with a 45 percent win rate, average wins of 3 percent, and average losses of 1.5 percent, expectancy is 0.45 times 3 minus 0.55 times 1.5, which equals 1.35 minus 0.825, or about plus 0.53 percent per trade before costs. That number, honestly computed over enough trades and realistic costs, tells you whether a rule deserved to exist in the past. It cannot tell you the rule will work in the future, and the distance between those two statements is where most backtesting damage happens.
Key Takeaways
- A backtest requires rules precise enough to execute without judgment: entries, exits, sizing, and the universe of securities, all specified before the data is consulted.
- Costs are not a detail. Spreads, slippage, and commissions turn many elegant historical edges into round trips to nowhere, the fill realism problem from paper trading platforms in historical dress.
- The four classic deceptions: look ahead bias, survivorship bias, overfitting, and cherry picked periods. Each produces beautiful results from a rule with no live content.
- Out of sample testing is the minimal honesty standard: develop the rule on one period, judge it on a later, untouched period, and prefer rules whose logic you can explain in one sentence.
- Backtesting screens ideas for the forward test that actually counts, disciplined paper trading at real scale.
What does an honest backtest require?
Start with the rule written as code or as an unambiguous checklist, because any ambiguity will be resolved by the tester, consciously or not, in the rule's favor. Then the data: prices adjusted for splits and dividends, and a universe that includes companies that later failed or were acquired, because testing only on today's listed survivors builds in the conclusion that the rule avoided every disaster that actually happened to somebody. Then the execution model: orders filled at prices achievable at the time, typically the next day's open or a volume weighted price, never at the same close that generated the signal, which is look ahead bias in its purest form, using information at a moment it was not yet available to act on.
Sample size disciplines the conclusions. A small sample of trades cannot distinguish a rule from luck; how many trades a test needs depends on the size of the edge being claimed, its variability, and how independent the trades are. Larger samples across different market regimes can begin to, because a rule tested only across a bull market has been tested on the question of whether it buys, not on whether it works. Report the uncomfortable statistics beside the return: maximum drawdown, longest losing streak, and the result's dependence on the best five trades. A rule whose profit lives in five trades out of five hundred is a lottery record wearing a system's clothing.
How do the four deceptions operate?
Look ahead bias enters through data that was not knowable at the time: using restated financial figures rather than the figures first published, trading on a close that the signal itself required knowing, or selecting today's index members for a test set in the past. Survivorship bias is its cousin, quietly deleting every bankrupt and delisted name from history. Overfitting is the craftsman's failure: adjusting parameters, adding filters, until the rule fits the tested years like a tailored suit, at which point it describes those years rather than any law of markets. Every parameter tuned on the same tested years adds to the risk that the result is memorization with extra steps rather than research. Cherry picked periods complete the set: a rule that works from 2013 to 2019 and is presented without its behavior in 2008 or 2022 has been marketed, not tested.
The defenses are boring and effective. Hold out a final period you do not touch during development. Prefer fewer parameters with a stated economic reason: value, momentum, quality, and carry have stories about why they might persist; a rule requiring the third Tuesday and a 37 day average has a curve fit. And demand that the live forward test, in simulation at realistic fills and then at small real size, reproduce the backtest's character, its win rate, average win and loss, and drawdown shape, before capital scales. Divergence between the test and the forward record is information about which deception you committed, delivered at tuition rates of your choosing, which is the entire argument for practicing without real money first.
What is a backtest actually for, then?
Falsification and calibration, in that order. Most rules die in honest backtests, cheaply, which is a triumph of the method: every rule killed by history is a tuition bill the market does not get to present live. The survivors earn a forward test, not a fortune. Calibration is the second gift: the drawdown and streak statistics teach you the size at which you can actually operate the rule when it hurts, feeding directly into the risk budget of position sizing explained. An investor who has seen their rule's honest worst year in a spreadsheet is measurably harder to shake out of it in the live version, and that steadiness, more than any parameter, is what a backtest can legitimately give you.
FAQ
How much history should a backtest use? As much clean data as the rule's logic spans, ideally including at least one full bear market and one rate regime different from the recent one. More history with structural breaks in it teaches more than a long calm stretch, because the rule's behavior in stress is the behavior your capital will eventually meet.
Can I backtest a fundamental strategy, like buying low payout ratio dividend growers? Yes, with point in time fundamental data, meaning the figures as first reported, not as later restated, and with the tradable universe as it then existed. This is harder data to obtain than prices, which is why fundamental backtests are rarer and more error prone, and why their results deserve extra skepticism about look ahead bias.
What software do people use? Ranges from spreadsheets for slow rules with few trades, to dedicated platforms and code libraries for systematic tests. The tool matters less than the data hygiene and the execution model; an expensive platform fed survivorship biased data produces expensive fiction.
If a backtest is great, why not just trade it? Because the backtest is a hypothesis about the past's data generating process, and markets change participants, rules, and costs. Trade it as a hypothesis: small, monitored against its test statistics, with pre written conditions under which you conclude the hypothesis failed. That posture converts a backtest from a promise into an experiment, which is all it ever honestly was.
Sources
- Standard quantitative finance references on backtest methodology and bias, including treatments of look ahead and survivorship bias.
- Investor.gov (U.S. Securities and Exchange Commission), investor education material on investment strategies and risk, accessed October 2026.
- Every worked example in this post was computed directly from the stated inputs shown in the text.
Educational content, not financial advice. Any worked example uses hypothetical, illustrative figures chosen for round numbers. It is not a projection of any security's returns and not a recommendation to buy or sell any security.
