tracer-cloud--opensre/integrations/llm_cli/AGENTS.md

LLM CLI providers (subprocess)

Use this package when adding a new non-interactive LLM that shells out to a vendor CLI (like OpenAI Codex), instead of HTTP APIs.

Layout

File	Role
base.py	LLMCLIAdapter protocol, CLIProbe, CLIInvocation.
constants.py	Shared runner/adapter constants (probe-cache TTL, tempfail retry knobs, common timeout defaults).
registry.py	CLI_PROVIDER_REGISTRY: maps LLM_PROVIDER → adapter_factory + optional model_env_key. opensre doctor uses get_cli_provider_registration() so any key in this registry is treated as CLI-backed—do not hardcode provider IDs in doctor.py.
subprocess_env.py	Filtered env passed to CLI subprocesses (build_cli_subprocess_env). Extend _SAFE_SUBPROCESS_ENV_PREFIXES here when a CLI needs vendor-specific env prefixes.
env_overrides.py	Optional explicit HTTP/API keys merged into CLIInvocation.env when build_cli_subprocess_env would drop them (shared tuples + nonempty_env_values).
timeout_utils.py	Shared timeout env parsing (resolve_timeout_from_env for default + clamp behavior).
probe_utils.py	Shared subprocess probe helpers (currently run_version_probe for <binary> --version).
semver_utils.py	Shared semver helpers (parse_semver_three_part, semver_to_tuple).
binary_resolver.py	Shared executable resolution helpers (env -> PATH -> fallback paths).
runner.py	CLIBackedLLMClient: guardrails, detect(), subprocess.run, ANSI strip, LLMResponse.
text.py	flatten_messages_to_prompt for stdin from chat-style payloads.
codex.py	Reference adapter: binary resolution, codex exec, --version, and opt-in-only login status probing.
opencode.py	Multi-provider CLI: --version, then opencode auth list (see _parse_opencode_auth_list_output).
kimi.py	kimi --print path: --version, kimi login status, then env/config.toml fallback (KIMI_API_KEY).
copilot.py	copilot -p path: --version, then env tokens, then gh auth status (and --hostname when COPILOT_GH_HOST / GH_HOST targets a non-default host); otherwise logged_in=None. Plaintext $COPILOT_HOME/config.json is not inspected. No OS keychain probes.
pi_cli.py	Pi CLI (pi.dev, BYOK multi-provider) pi -p print mode: --version, then state-based auth (provider API key in env, else ~/.pi/agent/auth.json from /login) — Pi has no non-interactive auth-status command. Provider keys forwarded via CLIInvocation.env (PI_PROVIDER_ENV_KEYS), never the blanket prefix.
grok_cli.py	xAI Grok Build CLI (grok -p --output-format plain): --version, then grok models (~0.5 s, no LLM call) to classify auth — looks for "You are logged in" / "logged in with" in output. XAI_API_KEY env promotes to authenticated as a headless/CI fallback even when the probe result is unclear. XAI_API_KEY forwarded only via CLIInvocation.env (XAI_CLI_ENV_KEYS), never the blanket prefix. Never passes --always-approve.

Wiring a new provider

Before merging, read Subprocess environment allowlist below: if your CLI reads vendor-specific env vars, you must extend _SAFE_SUBPROCESS_ENV_PREFIXES in subprocess_env.py or the subprocess will not see them (auth and config will break silently).

1. Adapter — Implement LLMCLIAdapter: detect() must not raise; build() returns argv + optional stdin; parse / explain_failure for success and non-zero exits. Put prompt text on stdin and/or in argv as appropriate — the runner does not branch on a separate “delivery mode”; CLIInvocation carries what build() produced.
2. Registry — Add an entry to CLI_PROVIDER_REGISTRY in registry.py (adapter_factory, model_env_key). The dict key must match LLM_PROVIDER / ProviderOption.value / LLMProvider in config/config.py. resolve_llm_route in core/llm/factory.py resolves registered CLI providers into a cli_provider_registration, and the builders in core/llm/client_builders.py construct the CLI-backed client automatically (no new branch for normal cases).
3. Config — Add the provider literal to LLMProvider and validators in config/config.py (same string as the registry key).
4. Wizard (optional) — If onboarding should offer the CLI: add a ProviderOption in surfaces/cli/wizard/config.py with credential_kind="cli" and adapter_factory. flow.py already runs _run_cli_llm_onboarding for CLI providers and builds the saved-summary credential line from provider.label + adapter.auth_hint.
5. Typing — Prefer adapter_factory: Callable[[], LLMCLIAdapter] on ProviderOption so wizard and client stay aligned.

Binary resolution (recommended pattern)

Use binary_resolver.resolve_cli_binary(...) so all adapters share the same behavior.

Resolution order:

1. Explicit binary env var (<PROVIDER>_BIN, e.g. Codex CODEX_BIN) only if it points to a runnable file.
2. shutil.which(...) lookups for platform-specific binary names.
3. Fallback install locations from default_cli_fallback_paths(...).

Notes:

• Binary env vars are optional by default.
• Blank/invalid explicit paths are ignored; PATH/fallback resolution still runs.
• For Codex, keep this behavior: users can run with no CODEX_BIN.

Conventions

• No TTY: invocation must be suitable for subprocess.run without an interactive session.
• Probe vs run: detect() is cheap; CLIBackedLLMClient.invoke probes again before exec so missing auth fails fast with a clear error.
• Structured output: CLIBackedLLMClient.with_structured_output delegates to StructuredOutputClient (JSON-in-prompt), same pattern as API clients.
• Optional model envs: use <PROVIDER>_MODEL (see Per-provider env vars); always optional—if unset, rely on vendor CLI defaults.

Per-provider env vars (required for every new CLI)

Codex is the reference. Every subprocess LLM must expose the same shape of knobs:

Env var	Role
<PROVIDER>_BIN	Optional explicit path to the vendor executable. Pass the same name as explicit_env_key to resolve_cli_binary(...). Missing, blank, or invalid paths are ignored; PATH + fallbacks still run.
<PROVIDER>_MODEL	Optional model override. Register as model_env_key on CLIProviderRegistration in registry.py. Empty or unset → runner omits the flag and the CLI uses its default.

Naming: derive <PROVIDER> from the registry / LLM_PROVIDER string: uppercase, then _BIN / _MODEL. Examples: codex → CODEX_BIN, CODEX_MODEL; a future gemini → GEMINI_BIN, GEMINI_MODEL.

Document both vars in the adapter module docstring or a one-line comment near binary_env_key / the registration entry so users and wizard copy stay aligned.

Auth probe pattern

detect() must return a CLIProbe with logged_in: bool | None — three states:

Value	Meaning	Wizard behaviour
True	Binary found and auth confirmed.	Proceeds immediately.
False	Binary found but definitely not authenticated.	Prompts user to run the login command (auth_hint).
None	Binary found but auth status is unclear (network error, unexpected output, etc.).	Asks user to retry or repick provider.

Recommended probe sequence for CLIs with safe non-interactive auth-status commands:

1. Run <binary> --version — if it fails, return installed=False immediately.
2. Run <binary> <auth-status-command> — parse stdout/stderr to classify logged_in.
3. Write a _classify_<name>_auth(returncode, stdout, stderr) -> tuple[bool | None, str] helper. Check negative phrases first (e.g. "not logged in" before "logged in") to avoid substring false-positives.
4. Map network/timeout errors to None, not False — the user may be on a flaky connection and shouldn't be forced to re-authenticate.

See _classify_codex_auth in codex.py for a complete reference implementation.

Codex exception: do not run codex login status by default. Some Codex CLI versions can open browser OAuth while checking a session, so the Codex adapter only runs that command when OPENSRE_CODEX_AUTH_STATUS_PROBE=1 is explicitly set. The normal prompt-safe status path reports logged_in=None and lets codex exec surface request-time auth failures.

OpenCode is multi-provider: users may rely on auth.json, environment API keys, or both. Run opencode auth list after --version and parse the reported credential/environment counts so detection matches the CLI (do not infer auth from the JSON file alone).

Kimi uses kimi login status after --version, then _check_kimi_auth_fallback() when the CLI did not positively confirm auth (logged_in not True): check KIMI_API_KEY, then API keys in ~/.kimi/config.toml (or KIMI_SHARE_DIR). API-key-only installs may omit “logged in” phrasing in login status, so the fallback mirrors real usage. The same fallback runs when login status times out or fails to spawn (logged_in=None), so a configured API key still counts as authenticated.

Subprocess environment allowlist

CLIBackedLLMClient passes only a safe subset of env vars to the subprocess via build_cli_subprocess_env in subprocess_env.py (_SAFE_SUBPROCESS_ENV_KEYS + _SAFE_SUBPROCESS_ENV_PREFIXES).

Shared HTTP/API overrides live in env_overrides.py: use nonempty_env_values(...) with OPENAI_PLATFORM_ENV_KEYS (Codex), HTTP_LLM_PROVIDER_ENV_KEYS (OpenCode), ANTHROPIC_CLI_ENV_KEYS (Claude Code), CURSOR_CLI_ENV_KEYS (Cursor Agent headless API key), or COPILOT_CLI_ENV_KEYS (Copilot CLI credential envs: COPILOT_GITHUB_TOKEN, GH_TOKEN, GITHUB_TOKEN) plus COPILOT_CLI_CONFIG_ENV_KEYS (COPILOT_HOME, COPILOT_MODEL, COPILOT_GH_HOST, GH_HOST). Extend those tuples when you add a matching API-key env to LLMSettings.

Kimi does not use those tuples today: OAuth/API material is covered by forwarding any KIMI_* keys via _SAFE_SUBPROCESS_ENV_PREFIXES; KimiAdapter.build() uses CLIInvocation(env=None) and relies on that allowlist.

The current prefix allowlist includes CODEX_, CURSOR_, CLAUDE_, OPENCODE_, KIMI_, and locale keys (LC_). COPILOT_ is deliberately NOT a prefix entry — COPILOT_GITHUB_TOKEN is a GitHub PAT, and a blanket prefix would leak it into every other CLI subprocess; the Copilot adapter forwards every Copilot-scoped env via its own CLIInvocation.env instead (see COPILOT_CLI_ENV_KEYS / COPILOT_CLI_CONFIG_ENV_KEYS in env_overrides.py).

If your CLI reads custom env vars (e.g. GEMINI_*) you must add the relevant prefix to _SAFE_SUBPROCESS_ENV_PREFIXES in subprocess_env.py, otherwise the subprocess will not receive those vars and authentication or configuration will silently fail. Add a test that asserts the required keys are forwarded.

Codex binary resolution (reference)

Order in CodexAdapter._resolve_binary (now delegated to shared resolver helpers):

1. CODEX_BIN if set and path is runnable (explicit override).
2. shutil.which("codex") (and Windows codex.cmd / codex.ps1).
3. _fallback_codex_paths() — conventional install locations; invalid or blank CODEX_BIN is ignored so PATH/fallbacks still apply.

Codex env quick reference (instance of the convention above)

All optional:

CODEX_MODEL=
CODEX_BIN=

• If CODEX_MODEL is unset, codex exec uses its default model behavior.
• If CODEX_BIN is unset, adapter resolution falls back to PATH + known install locations.

Provider checklist (copy/paste)

• Add adapter in integrations/llm_cli/.
• Define <PROVIDER>_BIN + <PROVIDER>_MODEL per Per-provider env vars; reuse resolve_cli_binary(..., explicit_env_key=...) for _resolve_binary.
• Implement detect() with --version + auth status checks; follow the three-state logged_in pattern above.
• Write _classify_<name>_auth — test against a real logged-in and logged-out session before merging.
• If the CLI reads custom env vars (e.g. GEMINI_*), add the prefix to _SAFE_SUBPROCESS_ENV_PREFIXES in subprocess_env.py.
• Register the provider in registry.py and add the same LLM_PROVIDER value in config/config.py.
• (Optional) Add wizard onboarding option in surfaces/cli/wizard/config.py.
• Add tests under tests/integrations/llm_cli/ for detect/build/failure paths, including env forwarding.

Tests

• tests/integrations/llm_cli/ — adapter and runner unit tests; mock subprocess / shutil.which as needed.
• Platform-specific assertions must patch integrations.llm_cli.binary_resolver.sys.platform (not codex.sys.platform), because resolution lives in binary_resolver.py.
• npm_prefix_bin_dirs is @lru_cached; tests that vary env or platform should call npm_prefix_bin_dirs.cache_clear() before each case (or use a shared autouse fixture) to avoid stale cache across tests.