A backtest is an argument, and most backtests argue for more than the data supports. The rules below are how we keep an argument honest. We apply them to our own research and to every strategy a client asks us to validate.
How a test is run
- No flattering position sizing. A signal is first judged at a fixed size of one contract or one lot. Compounding, even at 1% fixed-fractional sizing, makes a weak signal look strong, so sizing is studied separately and later.
- Nothing that hides the signal. No take-profit or stop-loss orders, trailing stops or other exits that can be tuned until the history looks good. A signal is judged on its own exits and on time-based exits.
- Costs from the start. Spread, slippage and commission are in every test, set conservatively for the market and the size traded.
- Small search spaces. The fewer variations tried, the less the best one owes to luck. When many variations have been tried, the result is corrected for it.
- Data the strategy has never seen. A large share of the history is held back from all development and opened once, at the end.
How a result is judged
- A result has to hold across neighbouring parameter values, not only at the best one.
- A result has to hold on related instruments and in different periods, unless there is a clear reason why it should not.
- Live trading is compared with the research continuously. A strategy that drifts away from its research is flagged and, by rule, switched off.
What we do not publish
This site shows no performance figures. Client research stays with the client, and a chart of our own results would prove less than the method behind it. The library lists the published papers that shaped these rules.