Tested strategies
Study 1: 12 famous strategies tested the same way →
The Codeability Index: can a computer test the rule as it's published? →
Does It Backtest?
Popular trading rules, tested in QuantWright with costs on, every version counted, and one held-out test on years the rules never saw. The figures are the results as QuantWright stated them: statistics, never prices. Backtested results are hypothetical. Research software, not financial advice.
| # | Strategy | Market | Grade | Held-out test | Tested |
|---|---|---|---|---|---|
| 1 | RSI(2) mean reversion, with and without a 200-day filter | SPY | C | Run once | 26 October 2026 |
| 2 | RSI 30/70, the "87% win rate" rule | SPY | C | Run once | 28 October 2026 |
#1: RSI(2) mean reversion, with and without a 200-day filter
Rules. Version A: buy at the next open when the 2-day RSI closes below 10; sell when the close is above its 5-day simple moving average. Version B: the same, but buy only when the close is above its 200-day simple moving average.
Market. SPY, daily bars, long only, one trade at a time.
Data. US stocks and ETFs on daily bars, split-adjusted, with cash dividends credited on each ex-date (total returns). Run through the Agent API by Claude Code, on a research account used only for this series.
Costs. The defaults: commission 0.005% per round trip and slippage 0.01% per fill, with next-bar fills.
Versions tried. Two rules, each on two windows: four development runs. Version B on 1993–2024 was graded against the 2 distinct versions this account had tested on SPY in the last 30 days.
| Version | Window | Trades | Won | Grade (Honesty Score) |
|---|---|---|---|---|
| A | Jan 1993 – Dec 2024 | 384 | 71% | C (64/100) |
| B | Jan 1993 – Dec 2024 | 242 | 75% | C (64/100) |
| A | Jan 2009 – Dec 2024 | 177 | 69% | C (60/100) |
| B | Jan 2009 – Dec 2024 | 130 | 72% | C (62/100) |
Beat buy-and-hold over the same dates? No, on every run.
Held-out test. Run once, on Version B, over 2 January 2025 – 23 October 2026: 20 trades, 80% won, grade C (51/100), below buy-and-hold. Twenty trades is a small sample. This account's held-out test on SPY is now used.
Grade C (64/100) is the most a result can score before its held-out test. A high win rate is not an edge: neither version beat holding SPY over the same dates.
#2: RSI 30/70, the "87% win rate" rule
Rules. Buy at the next open when the 14-day RSI closes below 30; sell when it closes above 70. No stop.
Market. SPY, daily bars, long only, one trade at a time.
Data. US stocks and ETFs on daily bars, split-adjusted, with cash dividends credited on each ex-date (total returns). Run in the app, by its own agent, in one chat on the same research account as episode 1.
Costs. The defaults: commission 0.005% per round trip and slippage 0.01% per fill, with next-bar fills.
Versions tried. One rule, one development run. The app counted 5 distinct versions tested on SPY on this account in the last 30 days, episode 1's included.
| Version | Window | Trades | Won | Grade (Honesty Score) |
|---|---|---|---|---|
| The rule | Nov 2018 – Oct 2026 | 8 | 87.5% | C (50/100) |
Beat buy-and-hold over the same dates? No, on both runs.
Held-out test. Run once on 30 October 2026, the same code unchanged, over 29 January 1993 – 8 November 2018 (25.7 years): 20 trades, 85.0% won, grade C (50/100), below buy-and-hold. The 20th trade was still open on 8 November 2018 and is marked there. The development run warmed up its indicator on bars before November 2018 but took no trades there. Episode 1 used the Agent API's held-out test, which counts once per account and market; in the app it counts once per chat and market, so this chat's is now used.
A high win rate on few trades tells you little. The first run's own warning: "Only 8 trades — too few for the statistics to be reliable. Aim for 30+ before trusting the verdict." And on the held-out years: "Grade C: treat this as a first look, not proof."
Found a mistake?
Tell us and we will check it against the stored run. See the corrections policy.
See also: Legal & disclaimers · Terms · Privacy · Support