Backtest vs live trading results: the 4 evidence tiers

Backtest vs live trading results usually do not agree, and the honest reason is that the numbers come from evidence of different quality. Finforge's Sophon-3 paper, Version 1.0 published in July 2026, refuses to blur that line. It stamps every performance figure with one of four evidence tiers, T1 to T4, so no number silently spans evidence that is not comparable.

That is the industry's real gap. A backtest can be tuned until it looks flattering, because the data was visible when the design was chosen. A live result cannot. The tiers exist to make that distinction visible instead of hiding it.

Why do backtest vs live trading results disagree?

Backtest vs live trading results can diverge for one simple reason: one was free to fit and the other was not. A result can be fitted in two ways. A model can overfit its own data, and a designer can shape a system to a period they already watched. The ladder from T1 to T4 removes one of those at each step, and the top rung removes both and adds real money.

What each of the 4 evidence tiers controls for

T1 is the in-sample backtest through 2025. It controls for nothing new, and the paper says so: it does not stop leakage, or model and design selection. T2 is walk-forward evidence from 1 January 2026, after the configuration boundary of 31 December 2025, when the system was tuned with data only through 2025. It controls for per-step model leakage, but not for design selection.

T3 starts at the architecture freeze on 21 April 2026. The evaluation procedure was locked that day, so nothing after it could be shaped by the data or the design. It controls for both. T4 adds real capital and real fills on top of T3, from 27 April 2026. Its one stated caveat is capacity, because it is a single small account.

The two dates that make the live window clean

Two dates anchor the timeline, which runs from January 2023 to 1 July 2026. On 31 December 2025 the configuration was frozen, so all of 2026 is out of sample at the data level. On 21 April 2026 the evaluation procedure was frozen, so everything after is also out of sample at the design level. A public hash commitment ties the freeze to configuration and checkpoint manifests, making it externally provable rather than asserted.

The reading rules matter too. A 126-bar rolling information ratio seen on 24 June 2026 reaches back to roughly December 2025, before the freeze, so it is a T2 object, not a design-frozen one. The paper reports it as such and does not conflate it with the post-freeze window.

What the Tier-4 real-money window showed

The paper's real-money account ran from 27 April to 24 June 2026, about 41 trading days, trading Sophon Core against a total-return Nasdaq-100 benchmark. It returned an active return of about +12.9%. The paper states plainly that a window this short is not statistically conclusive on its own. Its role is real-fill evidence that the system operates as designed, not a verdict on edge. Past performance is not a guide to future returns.

Keep two claims apart. That account is real money, one account. The four public Sophon agents trade founder capital in Alpaca paper accounts, and no customer money is traded before launch.

Questions people ask

What is the difference between T2 and T3? T2 sits after the data boundary but before the design freeze, so no model saw its own future, yet the architecture could still have been shaped by that period. T3 starts on 21 April 2026, when the design was locked, so neither data nor design could be fitted to it.

Why does evidence provenance matter to a retail investor? Because a performance figure without a tier is a figure you cannot evaluate. A T1 backtest, a T2 walk-forward and a T4 real-money run carry very different weight, and Finforge prints the label next to each one.

Finforge Research wrote the Sophon-3 paper and runs its trading agents in public, wins and losses included. You can check the live results, benchmarks and drawdowns for every Sophon agent, and the Sophon-3 research summary carries the evidence tiers in full.

Read the paper. Then check the agents.