Backtest vs live trading results: the data protocol

Backtest vs live trading results diverge for two reasons that hide inside the data, not the model: what a backtest is allowed to see, and how its returns are counted. Finforge's Sophon-3 paper, Version 1.0, published July 2026, attacks both. Its Data and Universe Protocol runs each decision on point-in-time data, the numbers exactly as they stood on the decision day, and measures active return on a symmetric dividend basis, so a simulated run and a live account are judged on the same terms. Active return is portfolio return minus the return of the benchmark universe the agent trades.

Start with the most common quiet flatterer: survivorship bias. A backtest that admits a stock only after it has succeeded looks sharp in hindsight. The protocol stops that. At any decision moment t, eligible assets are restricted to the constituents of the index at exactly t, written U^I_t = {i: i is an element of It}. A name can be chosen only if it was an index member when the decision was made, not because it happened to survive.

The same discipline runs through every data point. A record enters the working set only if its observable timestamp is at or before the decision moment t. A fundamental enters at its filing or release time. News enters at publication. A price bar enters only after it closes. Timestamp semantics are part of the protocol and are stored with every run manifest, so a skeptic can reconstruct what the system could have known on any given day.

How Sophon-3 keeps a backtest from cheating

Prices get the same care. The warehouse keeps both raw and split-adjusted fields. Model-facing features use forward split-adjusted prices, so a stock split does not create a fake jump in the series, while the display maps back to the familiar prices a reader recognizes.

Why backtest and live trading results diverge on dividends

Here is a divergence that has nothing to do with the model: dividends. The paper calls the benchmark convention the most common source of illusory active return in long-only equity research. A simulated backtest does not credit dividends to the portfolio, so it is compared to the price index, ex-dividend on both sides. The live brokerage account does receive dividends, so its net asset value is total return and is compared to the total-return index. Symmetry, not a particular index, is the requirement.

Because the treatment is symmetric, a price-basis result and a total-return-basis result differ only by (dp - db), the held book's dividend yield minus the index yield. For the growth-tilted books evaluated here, whose yield runs at or below the Nasdaq-100's roughly 0.7% to 0.9% a year, a price-basis figure overstates active return by at most that gap. The paper calls the overstatement bounded and immaterial next to the effects it reports, and the holdings-weighted differential is measured in the August 2026 revision.

That is why a Sophon backtest and its live account can be compared at all. Ablation deltas are unaffected on any basis, because the benchmark cancels in the paired difference. The model is the same system, and the discipline lives in the data around it.

Questions people ask

What is survivorship bias in a backtest?

It is the error of judging a strategy using only the assets that happened to survive. The point-in-time universe rule blocks it by restricting names to the index members at the decision time, so the backtest cannot reach back and pick winners it would not have known about.

Why do dividends change backtest vs live trading results?

Because the two settings get treated differently if you are not careful. The paper keeps both sides symmetric: the price index against the simulation that ignores dividends, and the total-return index against the live account that receives them. Same treatment on each side is what makes the comparison fair.

Finforge Research wrote the Sophon-3 paper and runs its trading agents in public, wins and losses included. Four Sophon agents have traded since 21 April 2026 on founder capital in Alpaca paper accounts, so no customer money is traded before launch. Check the live results, benchmarks and drawdowns for every Sophon agent, and the Sophon-3 research summary for the full Data and Universe Protocol. These figures describe past research methodology and simulated comparisons, not a promise of future returns; past performance is not a guide to future returns.