Current status: complete, negative, and understood. 14-fold walk-forward failed on out-of-sample return; a turnover-matched random-genome null then showed the search itself is not the problem — the objective is.
Sections 1–8 below are the earlier fixed train/holdout design (two runs, 16 hand-picked training windows, one holdout read per run). That design was replaced by a walk-forward: train 18 months, test the 3 months immediately after, step 6 months, repeat — 14 folds spanning 2018–2026, each test window scored exactly once and never fed back into any design decision. This section is the current, statistically stronger read; §1–8 are kept for the historical record, not as an active result.
Out-of-sample headline, all 14 folds:
| Mean test Sharpe | −0.92 |
| Median test Sharpe | −0.31 |
| Folds with positive test Sharpe | 7 / 14 |
| Mean test return per 3-month fold | −3.76% |
| First-half-of-history mean Sharpe | +0.10 |
| Second-half-of-history mean Sharpe | −1.93 |
Fails every pre-registered walk-forward criterion. The first-half vs. second-half split is the important line: whatever the GA is finding decays across the sample rather than holding steady, which is what an edge that's merely being fit to a fading historical regularity looks like.
Every fold (test period Sharpe / return / max drawdown / Calmar / trades per year — Sharpe and return are for the 3-month test window only, not annualized):
| fold | test start | Sharpe | return | max DD | Calmar | trades/yr |
|---|---|---|---|---|---|---|
| 0 | 2019-08-18 | +2.61 | +38.9% | −6.6% | 41.7 | 100 |
| 1 | 2020-02-16 | −2.49 | −32.0% | −34.6% | −2.3 | 271 |
| 2 | 2020-08-17 | +0.64 | +7.5% | −13.3% | 2.6 | 449 |
| 3 | 2021-02-15 | +2.25 | +32.7% | −20.4% | 10.3 | 303 |
| 4 | 2021-08-17 | +1.35 | +20.3% | −36.7% | 3.0 | 703 |
| 5 | 2022-02-16 | −4.18 | −42.3% | −45.0% | −2.0 | 189 |
| 6 | 2022-08-17 | +0.48 | +2.8% | −43.6% | 0.3 | 1016 |
| 7 | 2023-02-16 | −1.36 | −14.2% | −22.3% | −2.1 | 550 |
| 8 | 2023-08-18 | −5.13 | −7.3% | −7.3% | −3.6 | 76 |
| 9 | 2024-02-16 | −0.65 | −26.6% | −47.4% | −1.5 | 734 |
| 10 | 2024-08-17 | +0.77 | +5.3% | −12.5% | 1.9 | 277 |
| 11 | 2025-02-16 | +0.03 | −2.0% | −12.7% | −0.6 | 152 |
| 12 | 2025-08-17 | −2.04 | −10.1% | −14.4% | −2.4 | 103 |
| 13 | 2026-02-16 | −5.09 | −25.6% | −28.2% | −2.5 | 273 |
The worst folds are ugly on every axis at once, not just Sharpe — fold 5 and fold 9 both draw down more than 45% in a single 3-month test window.
Why, separately from whether it works: is the search itself broken? A turnover-matched random-genome null (10,000 random genomes per fold, same scoring path, no evolution, restricted to genomes trading within 1.4× the GA's own trade rate so the comparison isn't just "beats a genome that churns itself to death on fees") was run against all 14 folds. The GA beat the random-search maximum in 13 of 14 folds — only fold 4 lost. So the search is a competent optimizer; it reliably outperforms 10,000 random draws at fitting its own training objective. The failure is entirely in what that objective rewards: p25-of-Sharpe over 6 quarterly sub-windows, with a 100-trades/yr floor, over an 8-hour-max-hold search space that structurally excludes the one edge this project has actually validated (a daily EMA trend rule held for weeks). Full breakdown of the five ranked design errors behind that objective is in HANDOFF.md in the btc-ga-trader repo.
Bottom line: a competent search, reliably fed a broken objective, produces a strategy that loses out-of-sample slightly more than half the time and loses badly the rest. That is a different, better-supported version of the same conclusion §1–8 reached with a single holdout read — this time backed by 14 independent-ish out-of-sample windows and an explicit noise-floor control instead of one run-1-vs-run-2 comparison.
Two runs, both failed. The second failed worse, and that is the useful part.
Fixing the genetic algorithm's search made the out-of-sample result substantially worse. That is strong evidence the bottleneck was never search quality — it is generalisation, and a better search simply finds a better fit to noise.
Criteria were fixed in DESIGN.md §8 before any engine was written, so the bar did not move to meet the result.
| run 1 (broken mutation) | run 2 (fixed mutation) | |
|---|---|---|
| Population × generations | 300 × 60 | 120 × 103 (stopped early) |
| Train Sharpe | +0.893 | +2.087 |
| Holdout Sharpe | −0.289 | −2.887 |
| Train → holdout swing | −1.18 | −4.97 |
| Train profit / window | +6.2% | +43.7% |
| Holdout profit / window | −0.4% | −9.3% |
| vs. 200 random genomes | passed, 100th pct | failed, 93.5th pct |
| Criteria met | 2 / 5 | 1 / 5 |
Run 2's champion performs worse out of sample than a meaningful fraction of randomly-initialised networks.
Run 1 had three structural defects (§5). The most serious pinned every bias at zero, so the policy could not represent a threshold at all. Run 2 fixed them. The fix demonstrably worked — the champion improved 16 times instead of effectively twice, and the population median rose from −6.7 to −0.75 rather than stalling at −4.4.
And the out-of-sample result got five Sharpe points worse.
That is the whole result. A search that explores better, against a fixed training set with no generalisation pressure, finds a better fit to noise. Train Sharpe more than doubled while holdout Sharpe fell from roughly zero to decisively negative. There was no robust edge for the better search to find, so it found a better illusion instead.
The corollary matters for what comes next: more exploration — more randomness, more generations, bigger populations — is expected to make this worse, not better. That experiment has effectively already been run.
Champion at generation 103, each holdout window vs. buy-and-hold on the same bars:
| regime | Sharpe | return | max DD | trades | buy & hold |
|---|---|---|---|---|---|
| BULL | −1.71 | −1.7% | −1.9% | 11 | +39.1% |
| BEAR | −5.05 | −20.7% | −20.7% | 49 | −27.3% |
| CHOP | −2.26 | −10.0% | −18.0% | 44 | −0.9% |
| FLAT | −2.71 | −5.1% | −6.9% | 34 | +5.9% |
Negative in all four. Run 1 at least managed positive Sharpe in CHOP (+0.35) and FLAT (+1.02); run 2 lost that too. In the BULL window buy-and-hold returned +39.1% while the champion returned −1.7%.
Cost sensitivity is no longer the deciding factor — run 2 is unprofitable even at 2 bps (−7.7%/window), where run 1 was still positive (+1.4%/window). It is not a fee-fragility problem any more; the strategy is simply wrong out of sample.
Run 2's champion improved at generations 2, 4, 5, 8, 11, 14, 15, 19, 20, 21, 22, 24, 31, 41, 46, 48 — then held +1.95 for 55 generations before breaking out to +2.09 at generation 103, which is where the run was stopped.
Two things follow. The long flat stretch was a plateau, not convergence — a patient run does still escape. And the escape at generation 103 went straight to a worse out-of-sample champion, so escaping plateaus was not worth much here.
Also worth recording: trades/year fell to 106, hard against the 100/yr floor, before jumping back to 221 at the breakout. Evolution pushing to the minimum allowed trade count is the classic way to inflate Sharpe by shedding fee drag — the floor was doing real work.
Mutation sigma was derived from arr.std(). Biases initialise to zeros, so std was 0, sigma collapsed to a 1e-3 floor — ~180× smaller than weights received — and it was self-reinforcing. After 60 generations 15 of 16 hidden biases were still exactly 0.0. With no biases the tanh units are pinned symmetric about the origin and the policy cannot express "act only when the signal exceeds X".
Fixed: sigma now derives from a fixed per-tensor init scale, and biases initialise with real spread. Verified b1 std 0.121 and growing.
300 × 60 is 18,000 evaluations but only 60 steps through a 1,196-dimensional space. Changed to 120 × 300.
Uniform crossover of two networks hits the competing-conventions problem. Cut from 100% of children to 20%, the rest mutation-only clones.
Each input zeroed and the champion re-run across all 16 training windows:
| input | Δ Sharpe when removed |
|---|---|
| [conv price-window stream] | −13.02 |
| close_over_ema50 | −1.11 |
| close_over_sma_1h20 | −0.66 |
| macd_line | −0.66 |
| lr_slope_20 | −0.62 |
The raw 200-bar price window dominates every engineered indicator combined — the policy is essentially a price-shape reader. That vindicates giving the model raw prices rather than only summary statistics.
Caveat on the magnitude: zeroing the conv block removes 16 of 61 inputs at once versus 1 of 39 for an indicator, and pushes inputs far outside anything the network evolved against, so −13.02 is not directly comparable to the indicator rows. Direction is solid; scale is not.
No features were removed on the strength of this. The ablation describes one under-evolved genome that was frozen for 56 of 60 generations with dead biases — it measures what that particular accident happened to use, not what the feature set is worth. Cutting features on that basis would bake a broken search's arbitrary preferences into the design.
Established:
bit-identical to the reference on 47 of 48 genomes (Spearman rank correlation 1.000), and a holdout read once per champion.
generalisation, not optimisation.
search overfits decisively.
Not established:
pressure at all: no validation set, no early stopping, no capacity control.
The levers that address the observed failure, in order of expected value:
training windows 12/4 and pick the generation with the best validation Sharpe. Run 2's generation-103 champion would very likely not have been selected under this rule.
and filters 8→4 gives roughly 600.
training windows (holdout kept strictly disjoint) directly improves the data-to-parameter ratio.
Explicitly not recommended: more randomness, larger populations, or longer runs. Run 2 is the evidence for why.
Holdout reads so far: three (run 1 control, run 1 light-mutation, run 2). Each is disclosed above. Further reads should follow a genuine design change — validation-based selection plus a capacity cut would qualify; a parameter tweak would not.