Validation-based champion selection

Lever #1 from the report's ยง8 Next list, run for real: at every walk-forward fold, the champion the report already shows was picked purely by re-scoring on the full training span at the end (the exact objective being maximized -- zero independent check). Here, each fold's 18-month training span is split 75% train-sub / 25% validation, chronologically, with the scaler fit on train-sub only. Every generation, the genome that's best on training is also scored, read-only, on the validation slice -- one look per generation, never fed back into breeding. Whichever generation's snapshot scored best on validation across the whole run becomes the "validation-selected" champion. Both that champion AND the original training-selected one are then scored on the fold's real, never-touched test window, so the two methods are directly comparable on the same out-of-sample data. See the discussion of why this doesn't fully escape the same multiple-comparisons risk in the chat that requested this page -- validation selection reduces bias, it doesn't eliminate it.

Live

Loading…

Completed folds

Loading…