A big trade count can hide a narrow test. Many trades may come from one week or the same market move. The count matters, but it cannot tell you on its own whether you have enough evidence.
Copy this sample-review worksheet
Keep these eight notes beside the saved report. Use your own document or notebook; this is a review aid, not a claim that your app has these fields. Start with the question, then inspect the count.
1. The claim
What exact question should this test answer? A check that the rules run is different from a claim about many market conditions.
2. What one row means
Are you counting signals, orders, fills, open trades or closed trades? Use the report’s definition.
3. The count and dates
Write the total, first and last dates, market, time frame and saved rule version.
4. Shared market moves
Which trades came from the same session, event or move? Note closely related markets too.
5. What drives the result
Record the largest winner, largest loser and busiest period. Note what changes when you review them separately.
6. Versions tried
Record how many settings or rule versions you tried. Keep the failed tries, not just the chosen result.
7. Untouched data
Which dates were kept out of rule choices? If none were, say so. A new name does not make old data untouched.
8. The next check
What can this sample not answer? Name the missing evidence and one next check. Leave unknown fields unknown.
For the full record of rule changes and failed tries, use the out-of-sample test guide. For a count that looks wrong, first separate signals, orders and fills.
A made-up example: 200 trades, one busy week
Suppose a report has 200 closed trades. Of those, 160 came from one week and 40 from other dates. Four related markets were used. The researcher tried 24 rule versions before choosing this one. These figures are made up. They are not TradeLab test results.
160 in one week · 40 on other dates
The note should say: “Most trades came from one week. Review that week and the shared market moves. Keep all 24 versions in the trial log. Name which data was never used to pick the rules.” Do not turn these facts into a pass mark or guess a smaller “independent trade count.” That needs more evidence and a method that fits the data.
Why a magic minimum does not work
Strategies generate opportunities at different rates. A long-horizon trend system may trade only a few times per year, while an intraday system may generate hundreds of signals. Giving both the same minimum ignores how each strategy is designed to behave.
The outcome you are measuring matters too. Estimating an average trade is different from studying rare drawdowns, stop-loss behavior, or the chance of a long losing streak. Rare events need broader evidence than common ones.
Start with the claim you want the test to support
A backtest can answer a narrow question such as “did these exact rules generate valid signals in this window?” with relatively little data. A broad claim such as “this strategy is robust across market environments” needs observations from multiple environments, instruments, and time periods.
Write the intended conclusion before looking at the result. The broader the conclusion, the stronger and more varied the evidence must be.
Measure market coverage, not just rows in a report
Ask whether the sample includes the conditions the strategy is expected to face: rising and falling markets, quiet and volatile periods, ordinary sessions, gaps, and any relevant changes in liquidity. A high count drawn entirely from one regime may not support a claim about another.
This is also why the timeframe and lookback must match the strategy. A short sample can produce plenty of intraday trades while containing very little variation in the larger market backdrop.
Check whether the trades are truly distinct
Several signals can be driven by the same underlying move. Trades clustered in one session, one news event, or one trend are not as independent as the raw count suggests. Repeated entries in highly related instruments can create the same problem.
Group the results by time, instrument, direction, and market condition. If most of the outcome comes from one cluster, describe the evidence as concentrated instead of pretending every row is a separate confirmation.
Inspect uncertainty before interpreting averages
Small samples can swing sharply when one trade is added or removed. Recalculate the headline after excluding the largest winner and the largest loser. Review the median trade alongside the average, and compare separate periods instead of relying only on the full-period total.
If a reasonable change reverses the conclusion, the honest result is “not stable enough yet.” That is a useful research outcome, not a reason to search for a prettier setting.
A confidence interval can describe uncertainty in a proportion, such as a win rate, under a chosen model. It does not measure the size of wins, losses or costs. The NIST guide to intervals for proportionsexplains those methods. It does not set a trading pass mark.
A trade count becomes useful only when you understand what those trades represent.
Use a staged evidence process
- Validate the implementation. Confirm that the rules, data, warm-up, entries, and exits match the specification.
- Assess the sample. Review trade count, regime coverage, clustering, and concentration.
- Test stability. Compare logical neighboring periods or settings without searching until a result looks good.
- Observe forward. Preserve the rules and use paper trading to study unseen signals without risking capital.
Our backtest validation checklist can help you complete the first stage before sample-size questions distract from a configuration error.
How to describe an inconclusive sample
Do not round uncertainty into confidence. Record the exact rules, date range, timeframe, instrument, trade count, and known limitations. State which environments are missing and what evidence would be needed next.
A zero-trade or low-trade test can still reveal that the lookback is too short, the conditions are too restrictive, or the strategy behaves less frequently than expected. As our guide to useful losing and inconclusive backtests explains, research can succeed without producing a profitable headline.
The practical answer
There is no magic total. Save the claim, the count, the dates and the gaps. Review what drove the result. If the evidence cannot answer your question yet, call it preliminary and name the next check.
Why the record matters
Research by Bailey, Borwein, López de Prado and Zhustudies the risk of overfitting investment backtests. A held-out test is not a cure for every research-design problem. Keep the choices made before the final report visible. This worksheet does not calculate the paper’s probability of backtest overfitting.
Historical tests are hypothetical, depend on their data and assumptions, and do not predict future results. More trades do not eliminate model risk, execution differences, or uncertainty. Paper trading also differs from live execution. This material is educational and is not financial advice or a recommendation to trade.