Sports Analytics Pipeline Python & Data Engineering 117 Mathematical Models

Project Oracle — Turning Football Data Into Probabilities

If you've ever watched a football match with friends, you know how everyone argues over who's going to win based on gut feeling, recent highlights, or team loyalty. But bookmakers aren't guessing—they're using math. I built Project Oracle to see if I could build my own professional-grade quantitative prediction system from scratch in Python. It connects to live match feeds, crunches 117 mathematical formulas (from shot quality and pitch control down to player fatigue and weather), and spots where the market has mispriced a game—all backed by strict money management so you never bet recklessly.

My Role Creator · Data Engineering & Mathematical Modeling
Tech Stack Python 3.11 · NumPy · SciPy · AsyncIO · PostgreSQL · Redis
Scope 117 Formulas Across 10 Practical Angles
Key Focus Automated Data Pipeline, Probability Modeling & Risk Control

Why I Built This: The Math Behind the Beautiful Game

Football is pure emotion. A lucky deflection in the 90th minute can steal a win, and a bad referee call can turn a season upside down. Because the game is so chaotic, many people assume it's impossible to model with math.

I didn't build Oracle to predict the future—no algorithm can tell you if a star striker is going to slip on wet grass. What math can do, however, is strip away the hype and answer three honest questions:

  • 1. What is the true likelihood of each outcome?

    If these two teams played this exact match 100 times, how often would Team A actually win based on how well they control the pitch and create high-quality chances?

  • 2. Are the bookmaker odds fair?

    Is the public betting market underestimating or overestimating a team simply because of a temporary run of bad luck or an unlucky bounce?

  • 3. How much should you risk to stay safe?

    When the math shows the odds are in favor of an edge, exactly how much capital should be risked so that a normal streak of bad luck never wipes out your bankroll?

Answering those three questions turned into a deep dive across probability, sports physics, and financial risk modeling.

How the Pipeline Works: From Raw Match Data to Predictions

The platform is structured into four clean, automated stages that run without manual intervention:

  • 1. Gathering Match Data While You Sleep

    The pipeline runs automatically in the background every 30 minutes, pulling match statistics, team lineups, player pitch coordinates, and live odds from across the web—without requiring anyone to click a button or copy numbers.

  • 2. Examining the Game from 10 Different Angles

    Instead of throwing data into a black-box AI model and hoping for the best, Oracle passes the numbers through 117 specific mathematical formulas. It asks: How dangerous are their shots? Who controls the center of the pitch? Are key players tired? Is the pitch soaked in heavy rain?

  • 3. Finding the Edge and Staying Disciplined

    The engine compares its calculated win probability against public bookmaker odds. If the system calculates a 55% chance to win but the market priced it like a 45% chance, that's a statistical edge. It then uses a classic Wall Street formula (the fractional Kelly Criterion) to recommend a tiny, safe wager—usually just 1% to 2% of the total bankroll.

  • 4. Testing Against History Without Cheating

    In data science, it's dangerously easy to fool yourself by testing a model on past games it already 'knows' the scores of. I built a strict walk-forward backtester that only tests predictions using information that was available before kickoff. If the math doesn't work on past seasons, it doesn't get used.

How the Pipeline Fits Together

Cleanly separated into ingestion, vectorized mathematical calculation, and risk evaluation:

┌─────────────────────────────────────────────────────────┐
│              1. Collecting the Match Data               │
│  Pulls live stats, pitch tracking & odds automatically  │
│  - Cleans up messy data from multiple sources           │
│  - Queues everything in Redis for fast processing       │
└────────────────────────────┬────────────────────────────┘
                             │ Clean match statistics
                             ▼
┌─────────────────────────────────────────────────────────┐
│            2. The Math Engine (117 Formulas)            │
│  Examines the match across 10 realistic angles:         │
│  - True goal likelihood (Poisson & Dixon-Coles)         │
│  - Team form & rating updates (Elo & Glicko-2)          │
│  - Who controls the pitch (Voronoi spatial math)        │
│  - Real-life factors (fatigue, red cards, bad weather)  │
└────────────────────────────┬────────────────────────────┘
                             │ Calculated match probabilities
                             ▼
┌─────────────────────────────────────────────────────────┐
│            3. Finding Value & Staying Safe              │
│  - Compares model probabilities against bookmaker odds  │
│  - Only flags matches where I find a clear edge (>3%)   │
│  - Recommends tiny, safe stake sizes (Kelly Criterion)  │
│  - Saves every prediction in PostgreSQL for review      │
└─────────────────────────────────────────────────────────┘

What the 117 Formulas Actually Calculate

Rather than relying on a mysterious black box, Oracle groups its mathematics into 10 practical, real-world angles:

Angle & Scope What It Measures In Plain English Key Formulas Used
1. Goal Distributions (5 Formulas) Predicts expected scorelines. Corrects for the fact that low-scoring draws (0-0, 1-1) occur more often in football than standard probability predicts. Poisson, Skellam, Dixon-Coles (ρ correction for low scores), Zero-Inflated Poisson, Weibull Count
2. Dynamic Team Strength (5 Formulas) Updates team ratings based on how they played, not just the scoreline. A lucky 1-0 penalty win gets treated very differently from a 4-0 demolition. Margin-Adjusted Elo, Pythagorean Expectation, Log5 Win Probability, Bradley-Terry, Glicko-2
3. Value & Bankroll Protection (5 Formulas) Calculates whether market odds are mispriced in favor of value, and uses Wall Street portfolio math to recommend safe, disciplined stake sizing. Fractional Kelly Criterion, Implied Probability, Brier Score, Asian Handicap, Closing Line Value (CLV)
4. Trends & Form Momentum (20 Formulas) Spots long-term trend shifts, detects sudden statistical anomalies, and monitors risk-adjusted returns so you know when a model needs tuning. Log Loss, Gradient Descent + Momentum, Gini Impurity, Z-Score Anomaly, Sharpe Ratio, Max Drawdown, Weighted Ensemble
5. True Shot Quality (5 Formulas) Goes far beyond basic shot counts by evaluating distance, angle, defender pressure, and post-shot placement to measure real finishing ability. Expected Goals (xG), Post-Shot Expected Goals (PSxG), xG Chain, xG Buildup, Shot Clarity
6. Territory & Passing (5 Formulas) Measures 'Field Tilt' (how much possession occurs in the opponent's final third) and counts passes that cut cleanly through defensive lines. Field Tilt, Expected Assists (xA), Deep Completions, Progressive Passes, Packing Metric
7. Defensive Work Rate (8 Formulas) Evaluates how aggressively a team presses to win the ball back and how compact their defensive shape remains during counter-attacks. PPDA (Passes Allowed Per Defensive Action), Possession-Adjusted Tackles (PAdj), Rest Defense Compactness
8. Pitch Physics & Control (10 Formulas) Uses Voronoi spatial geometry to draw dynamic boundaries around players, calculating exactly what percentage of the pitch each team controls at any second. Voronoi Pitch Control Surfaces, Effective Playing Time, Convex Hull Compactness / Compression
9. Real-World Match Factors (10 Formulas) Adjusts predictions for real-world context: true home-field advantage, harsh weather (high wind and rain), travel fatigue, red cards, and new-manager bounce. Home Advantage Venue Coefficient, Manager Bounce, Red Card Decay, Weather Impact adjustments
10. League Patterns & Slumps (7 Formulas) Uses statistical regime models to tell whether a team is in an uncharacteristic slump or whether an entire league's scoring trend is shifting. Hidden Markov Model (HMM) Regimes, Hurst Exponent (trend vs mean-reversion), Shannon League Entropy

What a Match Prediction Looks Like

When all 117 formulas finish calculating, the system outputs clean, structured data for the match:

{
  "match_id": "EPL_001",
  "predicted_home_goals": 2.1,
  "predicted_away_goals": 1.6,
  "most_likely_score": [2, 1],
  "prob_home_win": 0.51,
  "prob_draw": 0.24,
  "prob_away_win": 0.25,
  "value_edge_home": 0.04,
  "kelly_stake_home": 0.018,
  "confidence_level": 0.72,
  "regime_status": "trending"
}

What this means in plain English: The engine expects an open game, forecasting about 2.1 goals for the home team and 1.6 for the away team, with 2-1 being the single most probable score. It calculates that the home team has a 51% chance of winning. Comparing that against current bookmaker odds reveals a 4% value edge for the bettor. But rather than placing a reckless bet, the Kelly formula advises wagering just 1.8% of the total bankroll to keep risk strictly under control.

Engineering Highlights I'm Proud Of

  • Fast and Asynchronous in Python

    Powered by Python 3.11 with asyncio and asyncpg. It can scrape telemetry feeds, query 14 PostgreSQL tables, and run math across dozens of games concurrently without freezing the server.

  • Zero-Cheating Historical Backtesting

    The most common trap in sports data science is 'lookahead bias'—accidentally using stats from after a match kicked off. I wrote a strict walk-forward backtester that only tests predictions using data that was knowable before kickoff.

  • 14 Structured PostgreSQL Tables

    Stores everything from raw player tracking coordinates to odds histories and model results, managed cleanly with connection pooling so any game in history can be audited.

  • Reliable Automated Testing

    Built with modern developer tooling including automated pytest test suites, fast linting with ruff, and static type safety with mypy running on GitHub Actions CI.