wehub-resource-sync
593b94c120
pytest / Unit Tests (push) Has been cancelled
pytest / Integration (integration_tests_a) (push) Has been cancelled
pytest / Integration (integration_tests_b) (push) Has been cancelled
pytest / Integration (integration_tests_c) (push) Has been cancelled
pytest / Integration (integration_tests_d) (push) Has been cancelled
pytest / Integration (integration_tests_e) (push) Has been cancelled
pytest / Integration (integration_tests_f) (push) Has been cancelled
pytest / Integration (integration_tests_g) (push) Has been cancelled
pytest / Integration (integration_tests_h) (push) Has been cancelled
pytest / Integration (integration_tests_i) (push) Has been cancelled
pytest / Integration (integration_tests_j) (push) Has been cancelled
pytest / Distributed (distributed_a) (push) Has been cancelled
pytest / Distributed (distributed_b) (push) Has been cancelled
pytest / Distributed (distributed_c) (push) Has been cancelled
pytest / Distributed (distributed_d) (push) Has been cancelled
pytest / Distributed (distributed_e) (push) Has been cancelled
pytest / Distributed (distributed_f) (push) Has been cancelled
pytest / Minimal Install (push) Has been cancelled
pytest / Event File (push) Has been cancelled
pytest (slow) / py-slow (push) Has been cancelled
Publish JSON Schema / publish-schema (push) Has been cancelled
Ludwig Serving + Prometheus + Grafana Monitoring
This directory contains a Docker Compose stack for running a Ludwig vLLM server alongside Prometheus and Grafana so you can monitor inference metrics in real time.
Prerequisites
- Docker 24+ and Docker Compose v2
- NVIDIA Container Toolkit (for GPU support)
- A trained Ludwig LLM model directory on the host
Quick start
-
Set the model path (the directory Ludwig will serve):
export MODEL_PATH=/absolute/path/to/your/ludwig_model -
Start the stack:
docker compose up -d -
Check services are healthy:
docker compose ps -
Send a test prediction:
curl http://localhost:8000/predict -X POST -F 'text=Hello world'
Services
| Service | URL | Description |
|---|---|---|
ludwig-serve |
http://localhost:8000 | Ludwig vLLM inference server |
prometheus |
http://localhost:9090 | Prometheus metrics scraper |
grafana |
http://localhost:3000 | Grafana dashboards (admin/admin) |
Prometheus metrics
Ludwig exposes a /metrics endpoint in Prometheus exposition format.
Prometheus is configured to scrape it every 15 seconds (see prometheus.yml).
Key metrics to watch:
ludwig_request_latency_seconds— per-request inference latency histogramludwig_requests_total— total requests served (labelled by endpoint)ludwig_batch_size— distribution of batch sizes received by/batch_predict
Grafana
Open http://localhost:3000 and log in with admin / admin.
The Prometheus datasource is pre-provisioned. To import a ready-made dashboard:
- Click Dashboards → Import
- Paste the dashboard ID or upload a JSON file
- Select the Prometheus datasource
Stopping the stack
docker compose down
To also remove persistent volumes (Prometheus TSDB and Grafana data):
docker compose down -v
Configuration
prometheus.yml— scrape interval and target configurationdocker-compose.yml— service definitions, port mappings, GPU allocation- Set
GF_SECURITY_ADMIN_PASSWORDindocker-compose.ymlbefore deploying to production