learningcircuit--local-deep-research/docs/release_notes/1.6.3.md

1.6.3 — release notes

LLM provider rework shipped by #3670. 1.6.3 was tagged without release notes; this file documents its breaking changes retroactively.

BREAKING — llm.model no longer auto-fills gemma3:12b

Before this release, leaving the Language Model field empty in Settings caused fresh installs to silently download a 7–12 GB Ollama binary (gemma3:12b) the user had not asked for. The default is now an empty string (""), and get_llm() raises ValueError("LLM model not configured...") when the field is unset for any provider that requires a model name.

Impact

• Fresh installs: open Settings → LLM, pick a provider, and choose a model name (e.g. llama3.1:8b for Ollama, gpt-4o-mini for OpenAI, claude-3-5-sonnet-20241022 for Anthropic). The model field shows a yellow inline warning while empty so the requirement is discoverable.
• Existing installs: any user who already had llm.model saved (including gemma3:12b) keeps their value — import_settings(overwrite=False) at settings/manager.py:1129 preserves any non-None DB value, so only fresh DBs and never-written rows pick up the new empty default.
• The corresponding HTTP endpoint /api/check/ollama_model now returns HTTP 400 with error_type: "model_not_configured" when neither the query parameter nor llm.model is set, instead of silently substituting gemma3:12b.

BREAKING — llamacpp provider switched to HTTP

The llamacpp provider no longer loads .gguf files in-process via langchain_community.llms.LlamaCpp. It now talks HTTP to llama.cpp's OpenAI-compatible llama-server, mirroring the LM Studio integration.

Migration steps for existing llamacpp users

1. Install llama.cpp (https://github.com/ggerganov/llama.cpp) if you do not already have it.
2. Start the server:

   llama-server -m /path/to/your/model.gguf
   

   Default port is 8080.
3. Open Settings → LLM and confirm llm.llamacpp.url is http://localhost:8080/v1 (or wherever your server runs). Set llm.llamacpp.api_key only if your server is behind an auth proxy.
4. Set llm.model to the model name llama-server is hosting (e.g. llama-3.1-8b-instruct).

Removed settings

The four old in-process settings are gone:

• llm.llamacpp_model_path
• llm.llamacpp_n_gpu_layers
• llm.llamacpp_n_batch
• llm.llamacpp_f16_kv

Existing rows for these keys remain in the per-user DB but are no longer read by any code path; they will be silently ignored. Tuning that previously lived in these settings now belongs to the llama-server command line.

New settings

• llm.llamacpp.url — HTTP endpoint URL (default http://localhost:8080/v1).
• llm.llamacpp.api_key — optional, only needed if you put an auth proxy in front of llama-server.

Cross-provider default-model cleanup

All built-in providers (anthropic, google, openrouter, xai, ionos, openai, ollama, lmstudio, llamacpp, openai_endpoint) now have default_model = "" and refuse to silently substitute. Where a user-specified model is missing, the central check in get_llm() raises a ValueError with an actionable message naming the offending setting key. 73 tests were updated to pass model_name=... explicitly rather than relying on the old defaults.

Better error message when local LLM servers are not running

When a user selects llamacpp, lmstudio, openai_endpoint (or another OpenAI-compatible provider) and the configured server is not reachable, the raw openai/httpx connection error is rewritten — by the friendly-error layer added in #4027 — into a message that names the provider and base URL, e.g. Cannot reach <provider> at <url>. Check that the server is running and the URL is correct. so the user sees something actionable instead of a stack trace.

Optional API keys for lmstudio and llamacpp

Both providers now support an optional api_key setting (llm.lmstudio.api_key, llm.llamacpp.api_key) for users who put their local server behind an auth proxy. If left blank, the providers use a placeholder key that bare llama-server / LM Studio ignore. No action required for default setups.

Settings UI

The LLM Provider dropdown now lists llama.cpp (Local) as a selectable option (it was reachable only by manual config-edit in 1.6.2). The empty-llm.model warning live-updates on both keystroke and dropdown selection, mirroring the API-key-not-configured pattern in openai_base.py.