Files
stefan-jansen--machine-lear…/envs

Docker Environments

Pre-built images are available on Docker Hub (docker.io/ml4t/). Most readers need only the main ml4t image.

Images

Image Docker Hub Python Platforms Size
ml4t ml4t/ml4t:latest 3.14 amd64 + arm64 ~12 GB / ~3 GB
py312 ml4t/ml4t-py312:latest 3.12 amd64 only ~9.6 GB
benchmark ml4t/ml4t-benchmark:latest 3.14 amd64 + arm64 ~1.7 GB
rapids (build locally) 3.12 amd64 + NVIDIA GPU ~15 GB

ml4t (Main)

Covers all 27 chapters and 9 case studies. Includes PyTorch with CUDA 12.8 support, LightGBM, scikit-learn, Polars, Plotly, and all ML4T libraries.

docker compose pull ml4t
docker compose up ml4t                    # Jupyter Lab at http://localhost:8888
docker compose run --rm ml4t python nb.py # Run a notebook directly

GPU passthrough (same image, NVIDIA runtime required):

docker compose --profile gpu run --rm ml4t-gpu python notebook.py

py312 (Python 3.12 Dependencies)

For notebooks requiring libraries without Python 3.14 wheels:

Notebook Library
Ch05 03_sigcwgan_signatures signatory
Ch09 06_path_signatures, 12_wasserstein_regimes signatory, esig
Ch10 01_word2vec, 02_asset_embeddings, 03_sentiment_evolution gensim
Ch15 06_fed_announcement_bsts tfcausalimpact (TFP BSTS)
Ch21 05_deep_hedging_pfhedge pfhedge
docker compose --profile py312 pull py312
docker compose --profile py312 run --rm py312 python 05_synthetic_data/03_sigcwgan_signatures.py

Not available on Apple Silicon — view pre-executed .ipynb files instead.

benchmark (Storage Benchmarks)

For Chapter 2 storage benchmarks comparing file formats and databases (TimescaleDB, ClickHouse, QuestDB, InfluxDB).

docker compose pull benchmark

# Start database services
docker compose --profile benchmark up -d timescaledb clickhouse questdb influxdb

# Run benchmark
docker compose --profile benchmark run --rm benchmark \
  python 02_financial_data_universe/21_storage_benchmark_database.py

# Stop databases
docker compose --profile benchmark down

rapids (GPU Benchmarks)

For Chapter 12 GBM GPU benchmark with RAPIDS cuML and LightGBM CUDA. Requires NVIDIA GPU. Must be built locally:

docker compose --profile rapids build rapids
docker compose --profile rapids run --rm rapids python 12_gradient_boosting/02_gbm_comparison.py

Directory Structure

envs/
├── README.md                  # This file
├── ml4t/Dockerfile            # Main image (Python 3.14)
├── py312/
│   ├── Dockerfile             # Python 3.12 for signatory/esig/gensim/pfhedge/tfcausalimpact
│   └── pyproject.toml         # py312-specific dependencies
├── benchmark/
│   ├── Dockerfile             # Benchmark image with DB clients
│   └── pyproject.toml         # Benchmark-specific dependencies
├── rapids/
│   └── Dockerfile             # RAPIDS cuML + LightGBM CUDA
└── test_all_imports.py        # Import verification script (63 packages)

Import Verification

Each Docker image has a self-test that verifies all packages needed by its notebooks are importable. Run it after pulling or building to confirm the environment is healthy:

# ml4t image (baked-in command)
docker compose run --rm ml4t ml4t-test-imports

# Or explicitly
docker compose run --rm ml4t python envs/test_all_imports.py

# py312 image
docker compose --profile py312 run --rm py312 python envs/test_all_imports.py --image py312

# Benchmark image
docker compose --profile benchmark run --rm benchmark python envs/test_all_imports.py --image benchmark

# Test a specific chapter
docker compose run --rm ml4t python envs/test_all_imports.py --chapter 15 --verbose

What's tested per image:

Image Packages ML4T Libs Utils Modules Chapters
ml4t 50 third-party 6 (data, engineer, models, diagnostic, backtest, live) 22 (4 repo + 18 case study) Ch01-Ch26
py312 5 (signatory, esig, gensim, pfhedge, tfcausalimpact) 1 (diagnostic) Ch05, Ch09, Ch10, Ch12, Ch14, Ch15, Ch21
benchmark 5 (duckdb, tables, DB clients) 0 Ch02

The test groups packages by chapter, so failures map directly to which notebooks are affected. Exit code is 0 (all pass) or 1 (failures). py312-only packages are shown as informational when running the ml4t test.

Building Locally

If you prefer to build from source instead of pulling from Docker Hub:

docker compose build ml4t                       # ~45 min on x86, ~15 min on ARM64
docker compose --profile py312 build py312      # ~30 min
docker compose --profile benchmark build benchmark  # ~10 min