Backtesting Best Practices

Learn how to backtest effectively and avoid common pitfalls that lead to misleading results.

The Golden Rules

1. Never Trust a Single Backtest

Always validate with:

• Multiple Time Periods - Test across different market conditions
• Out-of-Sample Data - Reserve data the strategy hasn't seen
• Different Markets - Test on similar assets
• Paper Trading - Forward test before going live

2. Understand the Limitations

3. Be Skeptical of Great Results

If results seem too good:

Testing Methodology

Walk-Forward Analysis

Instead of optimizing on all data, use rolling windows:

┌────────────────────────────────────────────────────────┐
│                  WALK-FORWARD TESTING                   │
│                                                         │
│  Period 1:  [Optimize: Jan-Jun] → [Test: Jul-Aug]      │
│  Period 2:  [Optimize: Mar-Aug] → [Test: Sep-Oct]      │
│  Period 3:  [Optimize: May-Oct] → [Test: Nov-Dec]      │
│                                                         │
│  Final Result: Average of all test periods              │
└────────────────────────────────────────────────────────┘

Benefits:

• Prevents over-fitting
• Shows real-world degradation
• Tests adaptability

Note: Walk-Forward analysis is currently a manual process — run separate backtests for each window and compare results.

In-Sample vs Out-of-Sample

Process:

1. Optimize parameters on in-sample data
2. Lock parameters (no changes)
3. Test on out-of-sample data
4. Only proceed if out-of-sample results are acceptable

Monte Carlo Simulation

Randomize trade order to test robustness:

1. Take your trade results
2. Randomly shuffle the order
3. Calculate new equity curve
4. Repeat 1000+ times
5. Analyze distribution of outcomes

What to look for:

• Median outcome (50th percentile)
• Worst case (5th percentile)
• Best case (95th percentile)

Note: Monte Carlo simulation is currently a manual process — export your trade data via CSV (Pro+) and use external tools.

Parameter Optimization

Built-in Parameter Optimization (Whale Tier)

The platform includes a Parameter Optimization tool that performs grid search across parameter ranges:

1. Define parameter ranges (min, max, step) for each tunable parameter
2. The engine tests all combinations automatically
3. Results are ranked by your chosen metric (return, Sharpe, Sortino, etc.)
4. View the top-performing parameter sets

This replaces manual parameter sweeps and ensures systematic coverage of the parameter space.

Avoid Over-Fitting

Sensitivity Analysis

Test how results change with parameter variations:

RSI Period: 14 → Test: 12, 13, 14, 15, 16

Results:
12: +32% return, 18% drawdown
13: +35% return, 16% drawdown
14: +38% return, 15% drawdown  ← Selected
15: +36% return, 14% drawdown
16: +31% return, 17% drawdown

Good sign: Similar results across nearby values Bad sign: Results vary wildly with small changes

Parameter Stability

Backtest Comparison (Whale Tier)

The built-in Comparison Tool lets you view multiple backtest results side-by-side:

• Overlay equity curves on the same chart
• Compare key metrics across runs
• Identify which parameter changes had the most impact
• Track strategy performance across code versions

This is especially useful after running parameter optimization to compare the top results visually.

Realistic Assumptions

Account for Costs

Example Impact:

Gross Return: +50%
After Fees (0.1% x 200 trades): -20%
After Spread (0.02% x 200): -4%
After Slippage (0.1% x 200): -20%
Net Return: +6%

Account for Execution Reality

Position Sizing Reality

Testing Different Conditions

Market Regimes

Test across different market types:

Historical Events

Include periods with:

• Flash crashes
• Major news events
• Exchange outages
• High volatility spikes

Purpose: Ensure strategy doesn't blow up in extreme conditions

Statistical Significance

Minimum Trade Count

Confidence Intervals

Don't just look at point estimates:

Win Rate: 55%
95% Confidence Interval: 48% - 62%

Interpretation: True win rate likely between 48-62%

With 50 trades: Wide interval, uncertain With 500 trades: Narrow interval, more certain

Documentation

Record Everything

For each backtest, document:

Tip: Full Backtest mode auto-versions your strategy code, so every backtest links to the exact code version used.

Transitioning to Live Trading

Paper Trading First

After successful backtests:

1. Deploy to paper trading
2. Run for at least 2-4 weeks
3. Compare to backtest expectations
4. Identify discrepancies
5. Adjust and repeat if needed

Live Trading Checklist

Before going live:

• Multiple successful backtests
• Out-of-sample validation
• Paper trading confirmation
• Risk parameters set
• Position sizing appropriate
• Stop losses configured
• Monitoring plan in place

Start Small

Common Mistakes Summary

Next Steps

• Trading Bots Overview - Deploy your strategy
• TradeStaq Strategies - Explore available strategies
• Risk Management - Protect your capital
• AI Strategy Builder - Build strategies with AI assistance