Backtest accuracy: fees, slippage and honest limits

Backtest accuracy has several layers

Asking whether a backtest is “accurate” hides multiple questions. Separate them so a strong result in one layer does not excuse a weakness in another.

1. Rule accuracy: did the engine implement the strategy graph exactly as specified?
2. Data accuracy: were timestamps, prices, volume, and instrument history suitable for the test?
3. Execution accuracy: could simulated orders plausibly have filled at the assumed prices and sizes?
4. Research validity: were rules frozen before validation, or tuned repeatedly to the same history?
5. Inference quality: does the conclusion stay within what hypothetical history can support?

Fees are necessary, but not sufficient

Trading fees reduce results on every simulated transaction. They matter most when turnover is high or gross edge per trade is small. Use the fee schedule relevant to the venue, product, order type, and assumed account tier, then test a less favorable rate as a sensitivity case.

Fees are only one part of execution cost. A backtest that deducts fees can still be optimistic if it assumes every order fills at a convenient chart price.

Spread: crossing the market has a cost

The spread is the difference between available bid and ask prices. OHLCV candles usually do not preserve that full state. A market buy commonly interacts with the ask, while a market sell interacts with the bid; modelling both at a candle close can overstate performance.

Spread varies by instrument, venue, time, volatility, and liquidity. One constant assumption is better than zero only if its limitation is visible. Sensitivity tests should include wider conditions than the average.

Slippage: intended price is not necessarily fill price

Slippage is the difference between the price a strategy expects and the price at which simulated or actual execution occurs. It can result from price movement during delay, insufficient quantity at the best price, queue position, volatility, or an order large relative to available liquidity.

A single slippage percentage is an approximation, not a universal truth. Test several adverse assumptions and pay special attention to strategies that trade frequently, react to fast moves, or depend on small per-trade differences.

Partial fills and order-state ambiguity

A model may assume an order is either completely filled or not filled. Realistic order lifecycles can be messier: part of the quantity fills, the rest remains open, an order expires, or cancellation races with another fill. That changes exposure, fees, exit size, and subsequent decisions.

Bar data also hides the sequence of prices inside a candle. If a bar touches both a stop and a target, the bar alone may not prove which came first. The simulator needs a documented, conservative rule for ambiguous cases.

Timing errors and look-ahead bias

Look-ahead bias occurs when a decision uses information that was not available at the simulated decision time. Common forms include:

• reading a completed candle and filling earlier within that same candle;
• using an indicator value calculated with future observations;
• selecting assets based on information learned after the test date;
• using revised data that differs from what was originally observable;
• applying an end-of-period ranking before every member’s data exists.

Define signal time and earliest eligible fill time explicitly. Inspect edge trades around transitions, not just ordinary cases.

Data quality and selection bias

Missing bars, duplicated timestamps, wrong time zones, symbol changes, delisted assets, and inconsistent venue data can alter results. A dataset containing only assets that survived to the present can introduce survivorship bias. Choosing the best market after testing many markets is another form of selection.

Keep a record of datasets examined, quality checks performed, and discarded experiments. Without that ledger, the final chart hides how many opportunities the research process had to find a lucky outcome.

Overfitting and parameter cliffs

Every rule, threshold, indicator period, market, and date window adds a choice. Repeatedly selecting the best historical combination can fit noise rather than a repeatable behavior.

Warning signs include:

• a narrow parameter value is profitable while nearby values collapse;
• one small date change reverses the conclusion;
• many exceptions were added after inspecting losing trades;
• performance depends on one asset, one regime, or a few outlier trades;
• the final strategy cannot be explained without referring to the backtest chart.

Freeze the rule graph, reserve out-of-sample data, and prefer stable regions over the single best parameter. No statistical technique can rescue a test whose design kept peeking at its validation set.

How to interpret Koval results today

Koval replays historical OHLCV bars and applies an explicit fees-only cost model. It does not currently model slippage, spread, or partial fills. Therefore:

• treat net results as an optimistic bound under the stated fee assumption;
• inspect every simulated fill and its bar context;
• discount high-turnover or small-edge strategies more aggressively;
• do not describe the report as executable, expected, or forecast performance;
• use paper and sandbox stages to test behavior the historical model cannot cover.

This transparent limitation is more useful than false precision. It tells you which question the benchmark can answer and what evidence is still missing.

A conservative review protocol

1. Reproduce the run from the same frozen graph, data scope, and configuration.
2. Verify several entries and exits manually, including ambiguous bars.
3. Separate in-sample design data from untouched out-of-sample data.
4. Review results by regime, asset, and time period rather than only in aggregate.
5. Stress fees and estimate plausible spread and slippage outside the current model.
6. Check neighboring parameters and simpler versions of the same hypothesis.
7. Inspect concentration in a few trades and long underwater periods.
8. Write the conclusion with every material omission beside it.
9. Move to paper trading without changing the rules during the observation window.
10. Use a supported exchange sandbox to validate adapter behavior with test assets.

Regulatory warnings are relevant even without real-money execution

Koval itself does not place real-money orders, but clear research language still matters. The U.S. Securities and Exchange Commission’s Investor.gov explains thatbacktested performance is hypothetical and not actual performance. The U.S. Commodity Futures Trading Commission also warns readers toscrutinise hypothetical trading-system claims and their assumptions.

Those resources do not certify Koval or any strategy. They reinforce the right standard: simulations should be transparent, reproducible, and described without implying future results.

What a credible conclusion sounds like

“Under the frozen rules, selected historical OHLCV data, and stated fee model, this strategy produced the recorded hypothetical behavior. Spread, slippage, and partial fills are not modelled. The result requires out-of-sample, paper, and sandbox validation and does not predict future performance.”

That statement is less exciting than a promise. It is also much more useful for deciding what to test next.