# Provider Doctor `jcode provider-doctor` is a user-facing diagnostic that answers one question: > Why isn't my provider/model (or the model picker) working? It walks the same strict end-to-end checkpoints that the live coverage ledger tracks (`jcode provider-test-coverage`), but as an interactive command you can run yourself, with clear pass/fail output and a "what to try next" hint on the first failure. It works with **OpenAI-compatible providers** (cerebras, fpt, nvidia-nim, comtegra, deepseek, groq, openrouter, and other `openai-compatible` profiles). ## Quick start ```bash # Validate jcode's own wiring for a provider, no API key, no spend: jcode provider-doctor cerebras --tier offline # Validate the key + live model catalog (needs a key, negligible spend): jcode provider-doctor cerebras --tier catalog # Full readiness, including real chat, streaming, and tool calls (spends balance): jcode provider-doctor cerebras --tier full # Pin a specific model and emit JSON for scripting/CI: jcode provider-doctor cerebras --model gpt-oss-120b --tier full --json ``` The model defaults to the provider's default model (or the first live catalog model). Use the global `--model` flag to pin a specific one. ## Tiers Pick how much to exercise. Each tier validates as much as is possible given its constraints, so you can debug cheaply and escalate only when needed. | Tier | Needs key? | Spends balance? | What it adds | Catches | | --- | --- | --- | --- | --- | | `offline` | no | no | jcode-side wiring against a synthetic catalog | catalog reload, picker rendering, fallback labeling, and model-switch routing bugs for this provider | | `catalog` (default) | yes | ~none | live `GET /models` | bad/missing key, dead endpoint, model not in the live catalog | | `full` | yes | yes | non-streaming chat, streaming, tool-call loop | the model actually chats, streams, and supports tool-calling | Only the `full` tier can earn strict ("READY") coverage. The lighter tiers intentionally record the API-dependent checkpoints as skipped, so nothing is over-credited in the coverage ledger. ## Checkpoints Every run reports these strict checkpoints in order. A pair is fully ready only when all of them pass on the `full` tier. 1. `auth_credential_loaded` - a credential was found for the provider 2. `model_catalog_live_endpoint` - the live `/models` endpoint returned models 3. `catalog_hot_reload_current_session` - the catalog reloaded into the session 4. `picker_live_models` - the picker shows the live models, including the selected one 5. `picker_fallback_labeling` - routes are live-catalog backed, not static fallback 6. `model_switch_route` - switching models produces a provider-explicit route 7. `non_streaming_chat_completion` - a basic chat reply came back (full tier) 8. `streaming_chat_completion` - a streamed reply came back (full tier) 9. `tool_call_parse` - the model emitted a parseable tool call (full tier) 10. `tool_execution_loop` - the tool-call loop ran (full tier) 11. `tool_result_followup` - the tool result was fed back (full tier) 12. `real_jcode_tool_smoke` - an end-to-end tool smoke passed (full tier) (Checkpoints 1-2 plus the auth-lifecycle stages are pre-flight; 7-12 are the API-dependent ones gated behind `--tier full`.) ## Reading the output ``` Provider doctor: Cerebras / gpt-oss-120b Tier: catalog (API key, ~no spend: adds live catalog fetch) ... [ PASS] Credential loaded Loaded credential from CEREBRAS_API_KEY [ PASS] Live model catalog endpoint 2 live model(s) returned [ PASS] Catalog hot reload in current session 2 catalog route(s) reloaded [ PASS] Picker shows live models 2 model(s) in picker, selected `gpt-oss-120b` [ PASS] Picker fallback labeling all routes backed by live catalog (no static fallback) [ PASS] Model switch route switch request `cerebras:...` routed via `openai-compatible:cerebras` [ skip] Non-streaming chat completion catalog tier: requires --tier full (spends balance) ... Verdict: tier `catalog` passed. Run `--tier full` to confirm full readiness (spends balance). ``` - `PASS` / `FAIL` - the checkpoint ran and passed/failed. - `skip` - the current tier does not run this checkpoint (use `--tier full`). - The verdict line tells you whether the tier passed, fully passed (`READY`), or failed, and on failure points at the first failing checkpoint with a next step. The command exits non-zero when the chosen tier did not fully pass, so it can be used as a CI/scripting gate. ## Spend tracking (how much does a run cost?) Balance-spending tiers (`catalog` makes a catalog call, `full` makes several chat/stream/tool calls) report exactly what they consumed so you can budget: ``` Spend this run: 3 billable API calls, 554 tokens (289 in + 265 out), cost not reported by provider ``` - **billable API calls** - how many requests actually hit the provider. - **tokens** - prompt + completion totals summed across those calls, when the provider returns a `usage` block. Streaming probes request `stream_options.include_usage` so streamed calls are counted too. - **cost** - shown as a USD figure only when the provider reports a `cost` field; many providers (e.g. cerebras) only return tokens, so you'll see "cost not reported by provider" and can multiply tokens by your plan's rate. A full cerebras run is roughly 550-620 tokens (about $0.0003). `--json` includes the same data under a `spend` object (`billable_calls`, `prompt_tokens`, `completion_tokens`, `total_tokens`, `has_token_data`, `reported_cost_usd`). This spend is **persisted** into the coverage ledger alongside the run, so `jcode provider-test-coverage` shows a cumulative "Recorded spend" footer summing the latest run per pair. That gives you a durable, at-a-glance answer to "how much has exercising this coverage cost me so far?" ## Typical debugging flow 1. **"My picker is broken / shows the wrong models."** Run `--tier offline`. If `picker_live_models`, `picker_fallback_labeling`, or `model_switch_route` fail, it's a jcode-side routing bug for that provider: capture the output and file an issue. 2. **"It won't connect / says auth failed."** Run `--tier catalog`. If `auth_credential_loaded` or `model_catalog_live_endpoint` fail, the key/endpoint is the problem. Run `jcode login --provider `. 3. **"It connects but the model behaves badly."** Run `--tier full`. If `non_streaming_chat_completion` / `streaming_chat_completion` / the `tool_*` checkpoints fail, the model itself is the issue; try another model from the live catalog. ## Relationship to coverage Every doctor run records a live-verification event into the coverage ledger, tagged with the tier (`doctor_tier`). A `full`-tier pass that clears all 11 strict checkpoints flips the pair to strict ("READY") in `jcode provider-test-coverage`. Lighter tiers record the API-dependent checkpoints as skipped, so they never over-credit a pair. `jcode provider-test-coverage` renders the same 11 checkpoints as an 11-stage pipeline. Each observed pair gets one compact line: a status token (`READY`, or `N/11` = how many stages it cleared) followed by `provider / model`, and then, for any pair that is not yet READY, the first blocker plus the exact `provider-doctor` command to push it past that blocker. So the two commands are two views of one pipeline: the coverage report shows where every pair is stuck and hands you the doctor command to advance it. Each line ends with a freshness note, e.g.: ``` READY cerebras / gpt-oss-120b last tested 9 minutes ago (2026-05-30) by developer (dev build) 6/11 nvidia-nim / gemma-4-31b failed at `streaming reply`; run `jcode provider-doctor nvidia-nim --model gemma-4-31b --tier full`; last tested 2 days ago ... ``` - **how long ago** the most recent run was, in plain English plus the absolute date, so you can tell at a glance whether the evidence is stale. - **who ran it**: a clean release build is labeled `user (release build)` (real user evidence), a dirty/dev build is `developer (dev build)`. This is derived durably from the build flag recorded with each run, not guessed.