Why do backtested trading strategies fail live? Three causes

Backtested trading strategies fail live for three reasons, and the Sophon-3 paper from Finforge Research names all three in its opening lines: predictive relationships decay, investable universes change, and research backtests diverge from live execution. The paper is Version 1.0, published July 2026. It was built around that problem instead of around one clever predictor.

Why do backtested trading strategies fail live in the first place?

Markets are non-stationary, which in plain words means the thing you fitted the model to has already moved. A ranking rule that sorted stocks well in 2023 is not sorting the same market in 2026. The list of names inside the index has changed under it too.

Then comes the third failure, the awkward one. The backtest ran in a clean loop on a research machine. The live account ran against a broker, at a real price, at a real minute of the day. Two different systems, one set of numbers, and the chart on the website is usually the research machine.

Finforge's answer is structural. Sophon-3 is not a single static predictor. It is a graph of decision nodes that each keep learning, wired into a causal chain: Asset Warehouse, then Screener, then Forecast, then Make Portfolio. Different parts update on different clocks.

What is an architecture freeze, and why 21 April 2026?

The paper defines evaluation as a frozen procedure with adaptive state, measured out-of-sample against a dated architecture freeze. Finforge set that date at 21 April 2026. The models keep learning after it. The rules governing how they learn cannot be touched.

That single date is what separates a research result from a sales chart. If a strategy can still be adjusted after its history is visible, the backtest describes the editing, not the market. Freeze the design, publish the date, then let the record run.

Which parts of the system survived the test?

Finforge measured by removal. Three live agents, two trading the Nasdaq-100 and one the S&P 500, were each compared against a copy of themselves with one component taken out. The comparison uses the paired daily difference in active return, which is portfolio return minus benchmark return, pooled across the three agents before any statistics are run.

Removing the Screener cost a pooled 45% of active return a year, with a t-value of 4.6, and it was significant in every agent on its own. Removing forecast-magnitude position sizing, which means sizing each holding by how large the predicted move is rather than splitting money evenly, cost 36% a year at t = 3.8. Re-searching the Screener's rules every month beat rules fixed once at inception by 23% a year, t = 3.4. A t-value is a signal-to-noise score, and a higher one means the gap is less likely to be chance.

Two mechanisms came out empty, and the paper says so in its abstract rather than in a footnote. Retraining the forecaster continually scored +4% of active return a year against a single fixed checkpoint, with a range running from -5% to +13%, so zero sits inside it. Adaptive ensemble weighting scored -0.0% a year against a plain equal average, give or take 0.6%. Finforge built both features. Finforge published both as unproven at this sample size.

What happened when real money traded the same frozen design?

One real brokerage account returned an active return of about +12.9% against a total-return Nasdaq-100 benchmark over roughly 41 trading days, from 27 April to 24 June 2026, entirely inside the frozen window. The paper states plainly that a window this short is not statistically conclusive on its own. Its role is real-fill evidence that the system operates as designed, not a verdict on edge. Past performance is not a guide to future returns.

Keep two claims apart. The paper's account is real money, one account, trading Sophon Core. The four Sophon agents on the website trade founder capital in Alpaca paper accounts, and no customer money is traded before launch. You can check the live results, benchmarks and drawdowns for every Sophon agent yourself.

Does a good backtest prove a trading strategy works live?

No, and the paper does not claim it. Finforge's own framing is that positive backtests do not prove future profitability, and that continual learning does not remove market risk. What a dated freeze buys you is a cleaner question. Everything recorded after 21 April 2026 ran against a design nobody could edit afterwards.

Who produced these numbers?

Hans Dahlström, Adam Tittenberger and Ted Björling at Finforge Research, in a manuscript titled Sophon-3: Continual-Learning Deep Models for Non-Stationary Financial Markets. Every figure in it is computed from frozen internal runs, never estimated. It is the first of three companion papers, and one of the others covers deterministic live-versus-backtest parity, which is the direct sequel to this question. A revision was scheduled for August 2026. The Sophon-3 research summary carries the tables.

Published as a research paper. Coming to your phone.