A genetic algorithm evolving a small neural policy that trades 5-minute BTC across four deliberately selected market regimes, judged on Sharpe ratio, and validated once on a held-out period it never influenced.
Status: design frozen, engine under construction. Windows are already frozen in windows.json and must not be re-selected.
A policy with access to the raw recent price path and standard quantitative indicators can learn regime-appropriate behaviour — trend-following in BULL/BEAR, restraint or mean-reversion in CHOP/FLAT — that survives out-of-sample on a holdout drawn from all four regimes, after realistic fees.
The honest null: it cannot, and any in-sample Sharpe is the product of a 1,212-parameter search over 415k bars finding noise.
| Source | C:/Users/jared/market-data-archive/data/btc/btcusdt_5m.parquet |
| Coverage | 949,488 bars, 2017-08-17 04:00 → 2026-09-01 22:50 UTC |
| Integrity | 0 duplicates, 0 NaNs, 910 zero-volume bars |
| Gaps | 34 gaps ≠ 300s, four of them ≥ 8h (33.6h, 10.1h, 10.1h, 8.1h) |
The ≥8h gaps collide directly with the 8-hour max hold. Rule: any open position is force-closed at the last bar before a gap > 15 minutes, at that bar's close. Without this a position "held" across the Feb-2018 hole would teleport 33 hours.
windows.json)Labels are computed from each block's own realised behaviour, never a trailing indicator. A first attempt used trailing 90-day return and produced a "BEAR" window that returned +20.6%, a "CHOP" window that returned +81.2%, and a "FLAT" window that returned +42%, while failing to flag 2022 at all. That approach was discarded.
Labelling rule: 90-day blocks, stepped 30 days (overlap used only to find rare chop). BULL if total return > +20%; BEAR if < −20%; otherwise non-trending, split on realised volatility against the median of all candidates — CHOP above (moves a lot, arrives nowhere), FLAT below (quiet drift).
Selection rule: rarest regime picks first; holdout chosen before training; every selected block mutually disjoint across regimes with a 1-day leak buffer (wider than the 200-bar / 16.7h lookback); training blocks spread by farthest-point selection on dates only, never on returns.
| Regime | Training windows | Holdout |
|---|---|---|
| BULL | 2017, 2019, 2020-21, 2024 | 2025-04-07 → 07-05 (+36.7%) |
| BEAR | 2018, 2019, 2021, 2022 | 2025-10-04 → 2026-01-01 (−27.4%) |
| CHOP | 2018, 2020, 2021, 2022 | 2026-02-01 → 05-01 (+1.6%, ER 0.01) |
| FLAT | 2020, 2022-23, 2023, 2024-25 | 2026-06-01 → 08-29 (+9.6%) |
Training: 1,440 days (~415k bars). Holdout: 360 days (~104k bars). 4:1.
Emergent property worth keeping: all training ends 2025-03-07 and the earliest holdout begins 2025-04-07, so the holdout is simultaneously regime-balanced and strictly in the future relative to every training bar.
| Rule | Value |
|---|---|
| Actions | buy (open/add long), sell (open/add short), hold, close |
| Leverage cap | 2× equity notional |
| Pyramiding | adds allowed while leverage headroom remains |
| Max hold | 8 hours = 96 bars, from first entry (adds do not reset the clock) |
| Fees | 5 bps/side, swept at 2 / 5 / 10 / 20 |
| Start equity | $10,000 |
| Wipeout | equity < −$10,000 → dead |
| Force close | max hold reached, or a data gap > 15 min |
Shorts are enabled at your instruction. Noting for the record that this project has twice killed short mechanics on evidence (trend_study test 2: short leg negative in 6/6 up years; research_loop D-000a). This is a deliberate re-test on a different timeframe and a different decision process, not an oversight.
Three streams, 62 inputs total.
Stream 1 — raw 200-bar price window (16.7h), normalised. z_i = (c_i / c_last − 1) / σ_window — scale-free, so it behaves identically at $2,820 and $126,011, and preserves chart shape (slope, compression, levels) which bar-to-bar differencing destroys. Reaches the network through convolution (§6), not 200 independent weights.
Stream 2 — 39 indicators. Trend (EMA/SMA ratios, LR slope 20/100), momentum (RSI 14/50, MACD triple, ROC 1h/4h/24h), volatility (ATR/close, Bollinger %b and bandwidth, realised vol 20/100), volume (z-scores, OBV slope, VWAP distance), regime (Kaufman efficiency ratio 20/100, ADX, consecutive-bar run), bar structure, and multi-timeframe context (1h/4h/daily SMA distance, daily RSI, 30-day range position). The last group exists because 200×5min is only 16.7 hours — this project's standing rule requires the model have genuine multi-day context.
Stream 3 — 6 position-state values. In-position, side, unrealised PnL, hold fraction, leverage used, adds so far.
Standardisation (measured necessity, not cosmetic). Raw indicator scales span ~500×: close_over_ema20 has sd ≈ 0.002 while vol_z_288 has sd ≈ 1.07. Eleven of thirty-nine features had sd < 0.01 and would be invisible to a linear layer. All features are standardised by median/IQR fitted on training bars only, then clipped to ±5. Holdout statistics never inform the model.
Feature warmup (50 days, driven by the daily SMA50) is taken from bars preceding each window, so it never consumes the window itself.
200-bar window ──> conv: 8 filters × 16 taps, stride 8 ──> max+mean pool ──> 16
│
39 indicators ───────────────────────────────────────────────────────────────┤──> 62 ──> tanh(16) ──> 4 actions
│
6 position state ────────────────────────────────────────────────────────────┘
| Component | Params |
|---|---|
| Conv filters (8 × 16) + biases | 136 |
| Dense 62 → 16 | 1,008 |
| Output 16 → 4 | 68 |
| Total | ~1,212 |
Why convolution rather than 200 dense weights. Wiring the window straight into a 24-unit hidden layer costs 6,004 parameters. At a realistic budget of 300 × 60 = 18,000 genome evaluations that is ~3 samples per free dimension — a lottery, not a search. The conv stream preserves genuine raw-window access at 136 params instead of 3,920, giving ~15 evals per dimension. For reference the live ga_trader net is 447 params.
The convolution runs over the raw close series once per genome, then each window's outputs are rescaled by 1/(σ_t·c_t) — algebraically identical to convolving every normalised window separately, at a tiny fraction of the cost.
Fitness = annualised Sharpe on daily-aggregated returns of the genome's equity curve, pooled across all 16 training windows.
Validity floor: ≥ 100 trades/year (~25 per 90-day window, ~394 across training). Below this, Sharpe is not treated as a score — the genome goes to the trade-count cull bucket. This is the guard against a genome taking three lucky trades and posting a spectacular Sharpe; the existing engine's floor of 5 trades is far too low to do that job.
Per generation:
with per-layer sigma
Population 300. Generations set from measured throughput (§9).
The holdout is scored exactly once, at the end, on the frozen champion. It is never used for culling, ranking, early stopping, or architecture choice. This project has already been burned precisely here — an earlier experiment culled on its holdout every generation, which made ~1,500 selection decisions using supposedly unseen data.
Pre-registered success criteria. The experiment "worked" only if the champion, on the holdout, satisfies all of:
the same initialisation distribution and scored on the same holdout
Failing any of these is a negative result and will be reported as one. Per-regime holdout Sharpe is reported separately — a champion that works only in BULL is a different and much weaker claim than one that works in all four.
Measured on this machine, population 300, 62 inputs, 16 hidden:
| Stage | Cost |
|---|---|
| Batched forward pass (all 300 genomes) | 170 µs/bar |
| Position/equity state machine | 95 µs/bar |
| Total | 265 µs/bar |
| One generation (415k training bars) | 110 s |
| 60 generations, single core | ~1.8 hours |
| 60 generations, windows parallelised over 8 cores | ~30–45 min |
Why not GPU. The workload is 415,000 sequential steps — position state feeds back into the next decision, so bars cannot be parallelised. Each step does only ~300k MACs, far too little to amortise GPU kernel-launch latency (~5–10 µs × ~40 ops per bar × 415k bars ≈ 80+ seconds of pure launch overhead per generation, before any arithmetic). A GPU would help only if the entire 415k-step loop were fused into one kernel via CUDA/Numba/JAX scan — a substantial rewrite for a workload that already completes in under two hours on CPU.
The real CPU lever is that the 16 training windows are independent and embarrassingly parallel across the 8 available cores.
Note: the research_loop experiment is still running on this machine and competes for cores; worker count should leave it headroom.
dimension. Better than the naive 3, still sparse. A negative result cannot fully distinguish "no edge exists" from "the GA could not find it."
overlapping search, versus 38 BULL. Chop conclusions rest on the least data.
could flip class under a small change to the rule.
behaviour. This is acceptable because labels are mechanical and outcome-blind (the strategy never sees the label, and windows were never re-picked after seeing a result), but it is not the same as trading a regime detector live.
high; the 2/5/10/20 bps sweep is mandatory reporting, not a footnote.
disproportionate share of bear and chop history.
Every performance report, dashboard table and results summary in this project must display all of:
| metric | why it is mandatory |
|---|---|
| Sharpe | risk-adjusted return, the headline |
| Return | the actual profit, per window |
| Max drawdown | Sharpe hides this entirely |
| Calmar | the only one of these that reacts to leverage |
| Win rate | distinguishes many small wins from one lucky window |
| Trades / year | catches a GA drifting to the minimum trade count to shed fee drag |
Sharpe and return alone are not acceptable. A run-4 champion posted a respectable Sharpe while carrying an -85% drawdown, and that was invisible until drawdown and Calmar were added to the dashboard. Iteration 4 of the earlier study measured the same point directly: DEV Calmar moved only 0.09 across a 3x leverage change while P(drawdown >= 50%) went from 9.5% to 99.8%. A promotion rule decided on Sharpe cannot see the axis that decides whether a book survives.
Implementation rule: evaluation functions must return drawdown and Calmar alongside Sharpe, not have them retrofitted. walkforward.py's eval_pop was written as a slim copy of ga.py's evaluate() and silently dropped both, which had to be added back after the omission was spotted in the live dashboard.