Backtest vs live trading results: the walk-forward fix

Backtest vs live trading results diverge most often for a boring reason: the model saw a future it could not have known about. Finforge's Sophon-3 paper, version 1.0, published in July 2026, builds against that in its Forecast node with a walk-forward loop that trains on rolling windows and predicts one step ahead, never past the decision horizon.

The Forecast node is the step that turns the Screener's shortlist into a forward signal. It is an ensemble, which means several deep models blended into one forecast. The members are heterogeneous, of different kinds: transformer-style, interpolation and decomposition style, and recurrent or temporal-convolutional types. Member m emits its forecast of return for asset i at horizon h, and the ensemble is a weighted sum of those member forecasts.

Why a hidden trading cadence is how backtest and live results drift apart

The loop itself is simple. At a cursor t, each member trains on a rolling window that ends at t, then predicts the chunk right after it. Train on the window from t minus W to t. Predict from t plus 1 to t plus h. Then slide forward and repeat.

The horizon is one step at the configured cadence, day or week. Here is the part most systems hide. The rebalance cadence is induced by that horizon, so a daily forecast yields daily portfolio decisions and a weekly forecast yields weekly ones. The paper treats cadence as a stated property of the graph, not a hidden tuning parameter. Turnover, capacity and the member count are defined relative to it, and run-until and last-closed-bar caps make sure no fit or prediction reads beyond the decision horizon.

That matters for backtest vs live trading results because a hidden cadence is a classic quiet divergence. A strategy that rebalances more often in the backtest than it can in a live account flatters the paper version and under-delivers live.

The hard rule that kills the dead-live gap

The paper's strictest line is about validation. Validation is the held-out stretch of history a model is checked against, data it did not train on. The node requires a nonzero validation slice for any scorable training window. A silent fallback to zero validation is forbidden. Without a picked validation slice the loop cannot do early stopping, the rule that halts training when improvements stop, and it produces what the paper calls deterministic overfitting. Overfitting is memorising the past instead of learning from it. The paper names it a leading cause of strong-backtest, dead-live outcomes. Strong in the lab, silent with real money. Sparse assets are pruned rather than truncating the global window, and a configuration that cannot support a valid validation slice raises a run-level error instead of training unvalidated.

The honest part: retraining might not even help

The paper does not assume that continual retraining pays. Whether retraining adds tradable value over a strong frozen checkpoint is an empirical question, not an assumption. It is addressed directly by the Static-Forecast ablation, which compares the retraining forecast against one left frozen. That is the falsifiable move: state the doubt, then test it.

Determinism means the backtest is the live run

The part that closes the gap between a simulated and a live run is determinism. The paper enforces identical results across the batched and the live-resume paths, at the seed, the member and the execution level. The mechanics live in a companion parity paper. That is what lets a backtest and a live account be the same system run twice, rather than two programs that quietly drift apart.

Questions people ask

What makes backtest and live trading results diverge?

Usually a peek: the model used future data, a hidden cadence, or no validation. The Forecast node blocks all three with a one-step-ahead loop, a stated cadence, and a mandatory validation slice.

What is a walk-forward loop?

Train on a rolling window that ends at the decision day, predict the next chunk, slide forward, repeat. Every decision uses only history that existed before it.

Who is behind these numbers?

Finforge Research wrote the Sophon-3 paper and runs its trading agents in public, wins and losses included. Four Sophon agents have traded since 21 April 2026 on founder capital in Alpaca paper accounts, so no customer money trades before launch. Check the live results, benchmarks and drawdowns for every Sophon agent, and the Sophon-3 research summary for the full Forecast protocol. Past performance is not a guide to future returns.