NFL Prediction System

Problem

NFL prediction is a temporal-data problem dressed up in a leakage hazard. The standard tutorial shape (scrape some play-by-play, train on a random train/test split, report a number) teaches the scikit-learn API and nothing else. Random cross-validation silently lets the model see Thursday’s closing line before predicting Wednesday’s game, and the reported accuracy is fiction. The second hazard is the market-edge floor: NFL spreads and totals are sharp, so any edge has to come from features the book has not already priced in, or from mathematically honest blending of the two signals. I wanted a pipeline where future-data leakage was structurally impossible, with every forecast reproducible from a single input: a Friday 18:00 ET snapshot of games, odds, and weather.

Approach & Architecture

The system is a Bronze / Silver / Gold data lakehouse with Pydantic v2 quality gates at every layer boundary (one bad row fails the whole batch, because silent row-skipping causes downstream corruption that is harder to debug than a hard failure). Bronze stores append-only timestamped Parquet snapshots from three external sources: games via nflreadpy, odds via The Odds API, and weather via Open-Meteo. Silver stores schema-validated, latest-wins tables Hive-partitioned by season and snapshot timestamp, so the Friday 18:00 ET snapshot is a first-class storage attribute. Gold stores one feature matrix per target: features_wp.parquet, features_ats.parquet, features_ou.parquet.

Temporal safety is enforced at three independent layers. Every builder in features/ satisfies a @runtime_checkable FeatureBuilder Protocol that mandates an as_of_datetime cutoff. A LeakageGate validator runs a per-builder time fence, a keyword scan over the combined feature matrix (anything matching closing_*, final_*, post-game variables), and a chronological Elo ordering check. Violations raise LeakageViolation and write a JSON diagnostic under outputs/diagnostics/. TemporalSplitConfig.validate() hard-fails if train, HP-validation, and holdout seasons overlap or violate ordering; a WalkForwardSplitter does expanding-window splits within the holdout. No random CV anywhere.

I trained three target-specific models using Optuna (TPE + Hyperband, resumable SQLite studies, pinned exactly to version 4.8.0): a calibrated LogisticRegression + IsotonicRegression pipeline for Win Probability, an XGBRegressor routed through a ResidualDistributionConverter for ATS cover probabilities, and a second XGBRegressor routed through a TotalDistributionConverter for Over/Under. The market blender combines model output with the Friday-snapshot lines in the mathematically correct space (log-odds for probabilities, point space for spreads and totals) with weights tuned strictly on pre-2018 seasons, so the 2021-2024 backtest window is temporally disjoint from weight selection. A DynamicBlendWeights schedule with per-target gating is implemented and tested; it activates the moment a future retrain clears gating.

The API is architecturally quarantined from the ML layer. A pytest AST-walking import guard (tests/api/test_import_guard.py) fails CI the moment api/ imports from models/, features/, or ratings/. The FastAPI app opens exactly one shared read-only DuckDB connection against a 6 MB web cache rebuilt by scripts/populate_cache.py, renders Jinja2 templates through jinja2-fragments’s Jinja2Blocks so full-page and HTMX fragment requests share the same template tree, and runs under a single-worker uvicorn envelope that the module docstring documents plainly.

Tradeoffs

Walk-forward validation is slower than k-fold (one retrain per season) but eliminates an entire class of leakage bugs I cannot catch by eye. Market blending caps upside (I cannot beat a sharp book by much) but the alternative is unblended model output that underperforms the market on Brier score and CLV, so calibration wins. HTMX over a JavaScript SPA keeps the payload tiny and the server single-sourced at the cost of client-side state I do not need; the data only changes once per week. The single-worker uvicorn envelope is a deliberate concurrency ceiling imposed by the shared read-only DuckDB connection and the module-level cachetools.TTLCache; multi-worker gunicorn is available in deployment/gunicorn.conf.py but would invalidate those invariants. Pinning Optuna to ==4.8.0 costs me minor-version bug fixes; Optuna study state persists to SQLite across runs and minor changes have historically altered search behaviour enough to drift results, which mattered more.

Outcome

The v1.0 MVP shipped on 2026-03-28 with the data pipeline, feature engineering, model training, walk-forward backtest, market blending, and the web dashboard all in production. The backtest covers expanding-window walk-forward training over the 2021-2024 holdout seasons with per-season retraining. The lakehouse holds 2002-2024 seasons fully ingested across Bronze, Silver, and Gold. A unified FridayPipeline orchestrator runs 18 sequential steps (8 data, 10 predictions) with tenacity-scoped retry on transient exception types and an atomic JSON execution log at logs/friday_pipeline.json. The test suite is 70 files (45 unit, 16 integration, 9 API) including the UIAP-01 import guard and seven Hypothesis-driven property tests.

Learnings

The hardest problem was discovering how easy temporal leakage is to ship by accident. My first Elo implementation processed games in dataset-order, which was not strictly chronological. A Monday Night Football result sometimes influenced the same-week Sunday predictions. The fix was a check_elo_ordering gate inside LeakageGate, but the deeper lesson was to add independent guards at every layer (Protocol, validator, splitter) so any single oversight gets caught by the next one. A close second was feature normalization: z-scoring across the full dataset leaks the distribution of future games, so I replaced it with expanding-window normalization grouped by season with min_periods=4 and prior-season bootstrap for the cold-start weeks. The architectural lesson I am most proud of is UIAP-01: quarantining api/ from models/, features/, and ratings/ behind an AST-walking test means the API stays millisecond-fast at startup no matter what the ML layer does.