Make a backtest investable in time

Learning goals

• Calculate net one-period return after both cost drags.
• Explain why survivorship bias applies to this case.
• Identify the case-specific assumption or failure mode before taking action.

Draft · approximately 30 minutes, excluding further revision.

Shared case and decision context

A strategy reports gross return 12%. Turnover implies modeled transaction costs 3% of beginning NAV, and fees are 1%. The dataset contains only surviving companies and uses financial statements revised after the trade dates. Treat costs as additive for this one-period illustration.

Economic or professional mechanism

Backtesting must reconstruct what was available and tradable at each decision time. Survivorship bias removes failed firms; look-ahead bias uses future or subsequently revised information. Repeatedly searching parameter combinations on the same test sample creates data-mining risk even if each individual trial appears statistically significant.

Apply, challenge and communicate

The simple net return is 8%, before correcting the data biases. Use point-in-time data, delisted assets, chronological walk-forward validation and realistic capacity and financing assumptions. Bootstrap methods should respect dependence; a random shuffle can destroy temporal structure. An impressive backtest remains a hypothesis, not proof of future performance.

Read backtest evidence and simulate without hindsight

Report compounded wealth, drawdowns, recovery time, turnover, exposures and out-of-sample performance alongside Sharpe. A highaverage return can hide long underwater periods or few concentrated winners. A historicalstress replay applies actual past shocks to today’s positions and may reveal vulnerability, but today’s market structure and portfolio reactions can differ. It is a scenario, not a frequency forecast.

Historicalsimulation resamples observed shocks and preserves only features represented by the sample; Monte Carlo draws from a specified jointmodel and can explore unobserved combinations but inherits parameter, distribution and dependence errors. Set horizon, frequency, number of paths, initial conditions, rebalancing, costs and stopping rules before inspecting results. Validate convergence and compare models rather than selecting the most favorable run.

Sensitivity changes one or a structured set of assumptions to reveal fragility; scenarioanalysis changes a coherent economic state. Ifgross 12% less modeled costs 3% andfees 1%=8%, raising costs to 5% reduces net to 6% under the same additive convention. This does not repair survivorship or revised-data leakage. Rebuild point-in-time data and run a genuinely held-out test before interpreting the result as investable.

Formula and assumptions

Simple one-period net return=gross−transaction cost drag−fee drag

Case analysis with explicit assumptions

1. A strategy reports gross return 12%. Turnover implies modeled transaction costs 3% of beginning NAV, and fees are 1%. The dataset contains only surviving companies and uses financial statements revised after the trade dates. Treat costs as additive for this one-period illustration.
2. Method: Simple one-period net return=gross−transaction cost drag−fee drag
3. Case result: 8.00%. The simple net return is 8%, before correcting the data biases. Use point-in-time data, delisted assets, chronological walk-forward validation and realistic capacity and financing assumptions. Bootstrap methods should respect dependence; a random shuffle can destroy temporal structure. An impressive backtest remains a hypothesis, not proof of future performance.

Errors to catch

• Do not replace the stated assumptions with an unstated shortcut. Reconcile units, timing and the claim being valued or duty being assessed.
• The absence of all model risk is not a defensible conclusion for these case facts.

Original knowledge check

Answer every question before checking. Review explanations after submitting; retry starts a new attempt.

Sources and coverage status

These original lessons are drafts. Public outlines are used to check scope; the official learning-outcome text and proprietary question banks are not reproduced here. Follow the source for exam-year requirements and updates.

• Reference · www.cfainstitute.org
• Reference · www.investor.gov