elizaos--eliza/plugins/plugin-lmstudio/AGENTS.md

@elizaos/plugin-lmstudio

Provides local LLM inference for Eliza agents via LM Studio's OpenAI-compatible API.

Purpose / role

This plugin wires LM Studio (a local model server at http://localhost:1234/v1 by default) into the elizaOS model dispatch system. It registers model handlers for every text-generation tier and for embeddings. The plugin auto-enables when LMSTUDIO_BASE_URL is set or when the default endpoint responds to a /v1/models probe — no explicit plugin list entry is required in those cases. It is opt-in when neither condition is true.

Plugin surface

This plugin registers model handlers only — no actions, providers, evaluators, services, routes, or events beyond MODEL_USED emissions.

Model type	Handler
ModelType.TEXT_NANO	handleTextNano — maps small/nano calls to the small model tier
ModelType.TEXT_SMALL	handleTextSmall — primary small-tier handler
ModelType.TEXT_MEDIUM	handleTextMedium — maps medium calls to the small model tier
ModelType.TEXT_LARGE	handleTextLarge — primary large-tier handler
ModelType.TEXT_MEGA	handleTextMega — maps mega calls to the large model tier
ModelType.RESPONSE_HANDLER	handleResponseHandler — response generation
ModelType.ACTION_PLANNER	handleActionPlanner — action planning (routed to large model)
ModelType.TEXT_EMBEDDING	handleTextEmbedding — vector embeddings via /v1/embeddings

All text handlers share handleTextWithModelType in models/text.ts, which supports streaming, structured output (Output.object), native tool calls, and per-tier model resolution. Embeddings fall back to a zero vector (length 1536) when LMSTUDIO_EMBEDDING_MODEL is not set, matching plugin-ollama behavior.

Layout

plugins/plugin-lmstudio/
  plugin.ts              Plugin object — model handler wiring + init probe + autoEnable predicate
  index.ts               Package entry — re-exports plugin, types, and config/detect utilities
  index.node.ts          Node/Bun build entry
  index.browser.ts       Browser build entry
  auto-enable.ts         Lightweight manifest entry-point for the autoEnableModule check (env-only)
  build.ts               Bun.build script (node + browser + cjs bundles, then tsc declarations)
  models/
    text.ts              All text generation handlers; resolveModelForType; streaming + structured output
    embedding.ts         handleTextEmbedding via @ai-sdk/openai-compatible textEmbeddingModel
    index.ts             Re-exports all handlers
  utils/
    client.ts            createLMStudioClient — @ai-sdk/openai-compatible provider factory
    config.ts            getSetting, getBaseURL, getApiKey, getSmallModel, getLargeModel, getEmbeddingModel, shouldAutoDetect
    detect.ts            detectLMStudio — probes GET /v1/models; parseModelsResponse; DetectionResult
    model-usage.ts       normalizeTokenUsage, estimateUsage, estimateEmbeddingUsage, emitModelUsed (MODEL_USED event)
  types/
    index.ts             LMStudioConfig, LMStudioModelInfo, LMStudioModelsResponse
  __tests__/
    config.test.ts       Unit tests for config resolution
    detect.test.ts       Unit tests for detectLMStudio with deterministic fetch
    embedding.test.ts    Unit tests for handleTextEmbedding
    text.shape.test.ts   Unit tests for normalizeNativeTools, normalizeToolChoice, normalizeNativeMessages
    integration.test.ts  Integration tests (requires live LM Studio)

Commands

These are the scripts available in this package:

bun run --cwd plugins/plugin-lmstudio build          # Bun.build via build.ts (node + browser + cjs) + tsc declarations
bun run --cwd plugins/plugin-lmstudio dev            # watch build
bun run --cwd plugins/plugin-lmstudio test           # unit tests (vitest)
bun run --cwd plugins/plugin-lmstudio test:unit      # same as test
bun run --cwd plugins/plugin-lmstudio lint           # biome check --write --unsafe
bun run --cwd plugins/plugin-lmstudio lint:check     # biome check (read-only)
bun run --cwd plugins/plugin-lmstudio format         # biome format --write
bun run --cwd plugins/plugin-lmstudio format:check   # biome format (read-only)
bun run --cwd plugins/plugin-lmstudio typecheck      # tsc --noEmit --noCheck
bun run --cwd plugins/plugin-lmstudio clean          # rm -rf dist .turbo .turbo-tsconfig.json tsconfig.tsbuildinfo

Config / env vars

All vars are optional. Resolution order: runtime.getSetting(key) → process.env[key] → default.

Var	Default	Notes
LMSTUDIO_BASE_URL	http://localhost:1234/v1	Base URL; /v1 is appended automatically if absent
LMSTUDIO_API_KEY	(none)	Bearer token; LM Studio does not require one by default
LMSTUDIO_SMALL_MODEL	(auto)	Model id for small/nano/medium tiers; falls back to SMALL_MODEL then first /v1/models entry
LMSTUDIO_LARGE_MODEL	(auto)	Model id for large/mega/action-planner tiers; falls back to LARGE_MODEL then first /v1/models entry
LMSTUDIO_EMBEDDING_MODEL	(none)	Model id for embeddings; zero vector returned if unset
LMSTUDIO_AUTO_DETECT	true	Set to 0/false to skip the init-time /v1/models probe

How to extend

Add a new model handler:

1. Add the handler function in models/text.ts (or a new file under models/), calling handleTextWithModelType with the desired ModelTypeName.
2. Export it from models/index.ts.
3. Register it in the models map in plugin.ts.

Add a new utility:

• Place it in utils/ as a named export. Import from there; do not put utility logic directly in plugin.ts.

Add config vars:

• Add resolution in utils/config.ts (follow the getSetting pattern).
• Add the var to agentConfig.pluginParameters in package.json so the runtime surfaces it.

Conventions / gotchas

• Model resolution is cached per runtime instance. The first call to GET /v1/models is stored in a WeakMap<IAgentRuntime, Promise<string | null>>. Tests that use multiple runtimes are unaffected; tests that reuse the same runtime instance will see cached results.
• Streaming + structured output are mutually exclusive. When stream: true is set alongside a responseSchema, handleTextWithModelType keeps the structured output and routes through generateText instead of streaming, to avoid LM Studio model engine inconsistencies. (Note: if both tools and responseSchema are present, the structured output is dropped instead — see the tools && outputSpec branch.)
• Embeddings do not throw on missing model. When LMSTUDIO_EMBEDDING_MODEL is unset, a zero vector is returned. This matches plugin-ollama but means embedding quality silently degrades — always set this var when using memory/recall features.
• Browser build is available (dist/browser/index.browser.js) but LM Studio itself is a local desktop app; browser use only makes sense when LM Studio is behind a CORS-permissive reverse proxy.
• Dependencies: @ai-sdk/openai-compatible (provider factory) and ai (Vercel AI SDK core — generateText, streamText, embed, Output). Both are runtime deps; @elizaos/core is a peer dep.
• No actions/providers/services. If you need an action that calls a local model, implement it in the agent layer using runtime.generateText — the plugin just wires the transport.
• See /AGENTS.md (repo root) for repo-wide architecture rules, logger conventions, and git workflow.

⛔ NON-NEGOTIABLE — evidence, trajectories & real end-to-end tests

The binding, repo-wide standard is AGENTS.md. Read it. Nothing in this package is done until it is proven done — a reviewer must confirm it works without reading the code, from the artifacts you attach. This applies to every feature, fix, refactor, and chore here. "Tests pass" is not proof; "CI is green" is not proof.

• Record AND read model trajectories. Capture the actual inputs and outputs of the model from a live LLM — not the deterministic proxy, not a mock: the prompt, the providers/context, the raw model output, every tool/action call, and the result. Then open the trajectory and review it by hand. A captured-but-unread trajectory is not evidence (packages/scenario-runner/bin/eliza-scenarios run <scenario> --report <out>).
• Real, full-featured E2E — no larp. Every feature ships detailed end-to-end tests that drive the real path end to end. Not the happy "front door" only: cover error paths, edge/empty/invalid input, concurrency, roles/permissions, and adversarial input. A test that asserts against a mock/stub/fixture standing in for the thing under test does not count. If the real model/device/chain/connector/account is hard to reach, make it reachable — that is the work, not an excuse to mock. If the existing tests here are shallow or mocked, fixing them is part of your change.
• Screenshots + logs at every phase, plus a complete walkthrough video/run-through of the entire feature or view, start to finish (bun run test:e2e:record).
• Manually review every artifact the change touches — never just the green check: client logs (console + network), server logs ([ClassName] …), the model trajectories in and out, before/after full-page screenshots, and the domain artifacts listed below for this package.
• No residuals. No shortcuts. The goal is not "done" — it is everything done. Clear every blocker by the hard path: build the real architecture, stand up the real model/device/service, actually test it. Never leave a TODO, a stub, a stepping-stone, or a "follow-up." When unsure, research thoroughly, weigh the options, and ship the best, highest-effort, production-ready version. Keep going until every possibility is exhausted.

Artifacts → attached inline in the PR (MP4 video, JPG screenshots, logs in <details>); attach each evidence type or explicitly mark it N/A with a reason — never leave it blank. If develop moved and changed behavior, re-capture evidence; stale proof is worse than none.

Capture & manually review for this package — model provider:

• A trajectory from a live call to this provider (not the proxy, not a mock): full request, raw response, token usage, finish reason, and streamed chunks.
• Proof of tool/function-calling and structured-output parsing against the real model.
• The error paths exercised: bad key, model-not-found, oversized context, timeout, rate-limit, mid-stream disconnect — plus latency and cost from the real call.
• If no key is available in CI, attach the documented live-run transcript as evidence — never a mocked client passed off as a pass.