Files
yvgude--lean-ctx/docs/user-journeys.md
T
wehub-resource-sync 26382a7ac6
CI / Clippy (push) Failing after 15m13s
CI / Test (ubuntu-latest) (push) Failing after 16m1s
CI / Test (macos-latest) (push) Has been cancelled
CI / Test (windows-latest) (push) Has been cancelled
CI / Build (no embeddings / no ORT) (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Cookbook (Node) (push) Has been cancelled
CI / Pi Extension (Node) (push) Has been cancelled
CI / Rust SDK (lean-ctx-client) (push) Has been cancelled
CI / Embed SDK (lean-ctx-sdk) (push) Has been cancelled
CI / Python SDK (leanctx) (push) Has been cancelled
CI / Hermes Plugin (Python) (push) Has been cancelled
CI / SDK Conformance Matrix (push) Has been cancelled
CI / Coverage (push) Has been cancelled
CI / cargo-deny (push) Has been cancelled
CI / Adversarial Safety (push) Has been cancelled
CI / Benchmarks (push) Has been cancelled
CI / Output-Quality Gate (eval A/B) (push) Has been cancelled
CI / Documentation (push) Has been cancelled
CI / CI Green (push) Has been cancelled
JetBrains Plugin / Actionlint (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (rust) (push) Has been cancelled
JetBrains Plugin / Validation (push) Has been cancelled
JetBrains Plugin / Build (push) Has been cancelled
JetBrains Plugin / Test (push) Has been cancelled
Security Check / Security Scan (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 12:35:30 +08:00

598 lines
27 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# lean-ctx User Journeys — The Governed, Scalable Context OS
> **Audience:** website / product narrative. Each journey is a persona-driven
> story: *who* hits a wall, *what* they do with lean-ctx, *what runs under the
> hood*, and the *payoff*. Every command and config key below is real and shipped
> — copy-paste them.
lean-ctx started as a way to **compress what your agent reads**. It has grown into
a **Context OS**: a local-first, governed context runtime that *any* developer can
build their own agent on — coding or not — and roll out to a team without ever
gating what a single developer gets locally.
This page tells that story as journeys, in two acts. **Act I** governs and scales
the context window. **Act II** turns lean-ctx into infrastructure you build *on*
stable SDKs, domain personas, universal ingestion, and a sandboxed extension
surface — monetized by coordination, never by subtracting local power (the
**Local-Free Invariant**).
The through-line: **one pre-prompt choke point**. Everything an agent is about to
see — native tool output, downstream MCP results, knowledge facts — passes through
the same pipeline, so the firewall, the sensitivity floor, and the gateway all
compose instead of fighting each other.
### Act I — Govern & scale the context window
| Feature | Persona it unblocks | One-line value | Surface |
|---|---|---|---|
| **MCP Tool-Catalog Gateway** | Agent with 5+ MCP servers | Unlimited downstream tools at constant context cost | `ctx_tools`, `[gateway]` |
| **Context Firewall** | Anyone running shell/search through an agent | Runaway outputs become a digest + retrieval ref | `[archive].ephemeral` |
| **Per-item Sensitivity Floor** | Regulated / security-conscious teams | Secrets & PII are redacted or dropped *before* the model | `[sensitivity]` |
| **Reproducible Scorecard** | Buyers, maintainers, CI | Self-verifying proof of savings + retrieval quality | `lean-ctx benchmark scorecard` |
### Act II — Build your own agent on lean-ctx
| Feature | Persona it unblocks | One-line value | Surface |
|---|---|---|---|
| **Open Door: `/v1` API + SDKs** | Developers in any language | Embed lean-ctx in your own harness over a stable contract | `lean-ctx serve`, `/v1/*`, `lean-ctx-client` |
| **Context Personas** | Non-coding agents (sales/research/support) | One runtime, many verticals — reshape the whole surface | `LEAN_CTX_PERSONA`, `persona` |
| **Universal Intake** | Research / data / support | Index PDF, CSV, email, HTML, JSON — not just code | format extractors, `ctx_index` |
| **Open Core: plugins + WASM** | Platform engineers | Add tools / compressors / providers without forking | `plugin.toml`, `LEAN_CTX_WASM_DIR`, `/v1/capabilities` |
| **Commercial Plane (Local-Free)** | Teams & buyers | Team RBAC, real plans & reproducible ROI — local stays free | `lean-ctx team`, `billing`, `savings roi` |
---
## Journey 1 — "My agent is drowning in tools" → MCP Tool-Catalog Gateway
**Persona:** Maya, a platform engineer. Her agent is wired to filesystem, GitHub,
Linear, Postgres and two internal MCP servers. Every request now ships *dozens* of
tool schemas in the system prompt. The agent has gotten **slower, pricier, and
worse at picking the right tool** — the well-documented "more tools → less
adoption" curve.
**The wall:** every connected MCP server injects its *entire* catalog, on every
request, whether or not the task needs it. lean-ctx used to shrink only its *own*
surface; Maya's pain is the *external* surface.
**The journey:**
1. Maya points lean-ctx at her servers — once, globally (this is privileged: it
can spawn processes and open connections, so it is **never** read from a
project-local config):
```toml
# ~/.lean-ctx/config.toml
[gateway]
enabled = true
top_n = 5
cache_ttl_secs = 300
[[gateway.servers]]
name = "linear"
transport = "http"
url = "https://mcp.linear.app/mcp"
headers = { Authorization = "Bearer ${LINEAR_TOKEN}" }
[[gateway.servers]]
name = "fs"
transport = "stdio" # spawned as a child process
command = "mcp-server-filesystem"
args = ["/srv/project"]
```
2. Her agent now sees **one** tool, `ctx_tools`, instead of the whole union. It
describes the task in natural language:
```jsonc
ctx_tools {"action":"find","query":"open an issue with a title and assignee"}
```
3. lean-ctx returns a ranked shortlist — the few tools that actually matter, plus
a count of everything it shielded:
```text
gateway: 3 tool(s) for "open an issue" (catalog: 47 tool(s) across 5 server(s))
1. linear::create_issue — Create a Linear issue params: title*, assignee, team
2. github::create_issue — Open a GitHub issue params: repo*, title*, body
3. linear::update_issue — Update an existing issue params: id*, state
```
4. The agent invokes the chosen handle; lean-ctx **proxies** the call to the
owning server and streams back the result:
```jsonc
ctx_tools {"action":"call","tool":"linear::create_issue",
"arguments":{"title":"Fix login","assignee":"maya"}}
```
**Under the hood** (`rust/src/core/gateway/`):
- `client.rs` — a real MCP client on the official `rmcp` SDK. `stdio` spawns the
server; `http` uses streamable-HTTP with custom headers. Every connect/list/call
is bounded by `call_timeout_secs`; sessions open per-operation and shut down
cleanly (no orphaned child processes).
- `catalog.rs` — aggregates each server's tools into a namespaced `server::tool`
catalog behind a **TTL cache**. Per-server errors are surfaced, never hidden.
- `router.rs` — builds an **ephemeral BM25 index** over the catalog per query (the
same engine as `ctx_search`) and returns the top-N, deterministically.
- `ctx_tools.rs` — gates on config, routes the action, and proxies the call;
downstream results flow back through the *same* firewall + sensitivity floor as
native tools.
**Payoff:** Maya can connect *as many* MCP servers as she likes. The model's
per-request tool surface stays flat at one meta-tool, tool-selection accuracy
recovers, and the catalog refreshes itself on a TTL. Full reference:
[Journey 5 §10](reference/05-advanced.md).
---
## Journey 2 — "One `grep` blew up my context window" → Context Firewall
**Persona:** Sam, who lets the agent run `ctx_shell`, `ctx_search` and `ctx_tree`
freely. One `rg` across a monorepo, one noisy build log, and **30k tokens of
output** lands in the window — pushing out the code the agent was actually editing.
**The journey:** Sam does nothing. The firewall is **on by default**. When a
firewallable tool's output crosses the token threshold, lean-ctx stores the full
output out-of-band and returns a compact, deterministic **digest** instead:
```text
[ctx_search output: 31,402 tokens stored]
… head (20 lines) …
… tail (8 lines) …
Retrieve in full: ctx_expand(id="a1b2c3", search="TODO", start_line=…, end_line=…)
```
The agent keeps a small, navigable footprint and can drill into the *exact* slice
it needs with `ctx_expand` — by line range or full-text search across the archive.
**Under the hood** (`rust/src/core/firewall.rs`):
- Scope is deliberately narrow: `ctx_shell`, `ctx_execute`, `ctx_search`,
`ctx_tree`. **Explicit file reads are never firewalled**
`is_protected_read()` makes `ctx_read` / `ctx_multi_read` / `ctx_smart_read`
the single source of truth for "a read always returns content the agent can
edit against," honoured by both the firewall and the `reference_results` path.
- The digest is built without an LLM (head/tail or char-bounded excerpt for single
giant lines) so it is reproducible and cheap.
**Config:**
```toml
[archive]
ephemeral = true # default on. Env: LEAN_CTX_EPHEMERAL
ephemeral_min_tokens = 4000 # threshold. Env: LEAN_CTX_EPHEMERAL_MIN_TOKENS
```
**Payoff:** runaway outputs can no longer evict the working set, with **zero loss**
— the raw output is one `ctx_expand` away.
---
## Journey 3 — "We can't let secrets reach the model" → Per-item Sensitivity Floor
**Persona:** Dana, security lead at a fintech. Policy is non-negotiable:
credentials and customer PII must never leave the building inside an LLM prompt —
even by accident, even in a stack trace an agent happened to `cat`.
**The journey:** Dana sets a **policy floor** once, globally:
```toml
[sensitivity]
enabled = true # no-op until set. Env: LEAN_CTX_SENSITIVITY
policy_floor = "confidential" # public < internal < confidential < secret
action = "redact" # redact (mask spans) | drop (withhold whole item)
```
From then on, every item heading to the model is classified and enforced at the
pre-prompt choke point. With `redact`, a leaked AWS key or card number is masked in
place; with `drop`, the offending item is withheld entirely.
**Under the hood** (`rust/src/core/sensitivity/`):
- Ordered levels `Public < Internal < Confidential < Secret` drive a single
`level >= floor` comparison.
- **Honest classification only** — no speculative heuristics. Secret-like paths and
detected secrets → `Secret`; **Luhn-validated** card numbers and **ISO-7064**
IBANs → `Confidential`. This keeps false positives from silently degrading good
context.
- One `enforce_text()` entry point is applied uniformly to **tool outputs** and
**knowledge injection** — including downstream results coming back through the
gateway (Journey 1).
**Payoff:** a uniform, auditable guarantee that sensitive data is handled *before*
it reaches the model — off by default, so nothing changes for users who don't opt
in. Full reference: [Security & Governance](reference/13-security-and-governance.md).
---
## Journey 4 — "Prove the savings are real" → Reproducible Scorecard
**Persona:** Priya, an engineering manager evaluating lean-ctx. Marketing numbers
don't survive procurement. She wants a measurement she can **re-run and get the
same answer** — on her laptop and in CI.
**The journey:**
```bash
lean-ctx benchmark scorecard # human-readable
lean-ctx benchmark scorecard --json # machine-readable artifact
```
She gets compression savings (per mode), retrieval **recall@5 / recall@10 / MRR**,
and latency over a fixed scenario matrix — plus a `determinism_digest`:
```jsonc
{
"schema_version": 1,
"tokenizer": "…",
"determinism_digest": "…", // fingerprint of the latency-free metrics
"scenarios": [ /* per-scenario savings + recall + mrr */ ],
"aggregate": { "avg_savings_pct": , "avg_recall_at_5": , "avg_mrr": }
}
```
**Under the hood** (`rust/src/core/scorecard/`): the corpus is generated
deterministically and retrieval is pure BM25, so the **quality** metrics are
identical run-to-run and machine-to-machine. Latency is reported but deliberately
**excluded** from the digest (it's wall-clock). Two runs of the same code anywhere
produce the same `determinism_digest` — the artifact is **self-verifying**, and CI
uploads it on every build.
**Payoff:** Priya can independently reproduce the headline numbers and diff them
across versions — trust by construction, not by claim.
---
## Act II — Build your own agent on lean-ctx
Act I made the context window safe and scalable. Act II opens the runtime itself:
embed it from any language, point it at any corpus, reshape it for any domain, and
extend it without forking — then take it to a team while every local feature stays
free. Full developer map: [The Context OS Guide](context-os/guide.md).
---
## Journey 5 — "I want to build my *own* agent — in any language" → Open Door
**Persona:** Leo, building an **outbound-sales agent** (not a coding agent) in
Python. He wants lean-ctx's compression, retrieval and memory — but driven from
*his* loop, in *his* language, against a contract that won't break under him.
**The wall:** lean-ctx looked coding-agent shaped (stdio MCP, IDE configs). Leo
needs a stable, language-neutral way to call it from his own harness — and a way
to *discover* what a given server can do instead of trial-and-error.
**The journey:**
1. Leo runs the local HTTP server (REST + SSE + MCP on one port):
```bash
lean-ctx serve # → http://127.0.0.1:8080 (prints a loopback bearer token)
```
2. He **discovers capabilities** instead of guessing — contract version, active
persona, transports, presets, the live tool surface, and every extension:
```bash
curl -s --oauth2-bearer "$TOKEN" http://127.0.0.1:8080/v1/capabilities
```
3. He installs an SDK — same wire contract in every language — and calls tools:
```python
# pip install lean-ctx-client
from leanctx import LeanCtxClient
client = LeanCtxClient("http://127.0.0.1:8080", bearer_token=TOKEN)
caps = client.capabilities() # branch on real features
text = client.call_tool_text("ctx_read", {"path": "notes/acme.md"})
```
4. He wires lean-ctx tools straight into his existing LLM loop via a **framework
adapter** — no glue code:
```python
from leanctx.adapters import to_openai_tools, run_openai_tool_call
tools = to_openai_tools(client) # pass to your OpenAI call
result = run_openai_tool_call(client, tool_call) # when the model picks one
```
**Under the hood:**
- **Stable `/v1` contract** (`rust/src/http_server/`): `GET /v1/capabilities`
([`capabilities-contract-v1`](contracts/capabilities-contract-v1.md)) and
`GET /v1/openapi.json` are the SSOT, generated from code and drift-tested —
generate a typed client in any language.
- **Three first-party SDKs**, all thin wire clients (no engine linking): Python
[`clients/python`](../clients/python), TypeScript [`lean-ctx-client`](../cookbook/sdk),
Rust [`clients/rust/lean-ctx-client`](../clients/rust/lean-ctx-client).
- **Adapters** for OpenAI, LangChain, LlamaIndex and CrewAI — present when the
framework is installed, with a helpful `ImportError` when it isn't.
**Payoff:** lean-ctx becomes a **service any developer embeds**, in any language,
behind a versioned contract — verified by a shared conformance kit
(`run_conformance`) before you ship.
---
## Journey 6 — "My agent isn't about code" → Context Personas
**Persona:** the same sales team. The defaults are tuned for software work — the
full power tool surface, a code-oriented intent taxonomy, identity compression.
Leo wants a surface shaped for **prospecting**, not refactoring.
**The wall:** one-size-fits-code defaults bury a non-coding agent in irrelevant
tools and the wrong compression/chunking for prose.
**The journey:** Leo selects a **persona** — one switch that reshapes the whole
context surface (tool set + read-mode + compressor + chunker + intent taxonomy +
sensitivity floor):
```bash
export LEAN_CTX_PERSONA=lead-gen # or: research · support · data-analysis · coding
```
Each persona is a real, shipped bundle — the surface, compressor, chunker, intents
and sensitivity floor all change together:
```text
coding → Power profile · identity · lines · auto · floor=public
research → Standard profile · markdown · paragraph · map · floor=public
support · data-analysis → Standard profile · prose/identity · floor=internal
lead-gen (alias: sales) → Custom 6-tool surface · prose · paragraph · floor=confidential
tools: ctx_read · ctx_search · ctx_url_read · ctx_knowledge · ctx_semantic_search · ctx_session
```
The lead-gen surface is genuinely **narrowed** to those six prospecting tools — the
refactor/code tools are gone — while `coding` keeps the full Power surface.
Not one of the built-ins? Ship a declarative persona file and select it — no fork:
```toml
# <personas_dir>/compliance.toml (default <OS-config-dir>/lean-ctx/personas; override via LEAN_CTX_PERSONAS_DIR)
name = "compliance"
tool_profile = "custom"
tools = ["ctx_read", "ctx_search", "ctx_semantic_search", "ctx_knowledge"]
default_read_mode = "map"
compressor = "prose"
chunker = "paragraph"
intent_taxonomy = ["scan", "flag", "cite", "report"]
sensitivity_floor = "confidential"
```
**Under the hood** (`rust/src/core/persona.rs`,
[`persona-spec-v1`](contracts/persona-spec-v1.md)): `Persona::resolve` reads
`LEAN_CTX_PERSONA` `config.persona` default `coding`; an unknown name falls
back to `coding`, never an error. The resolved profile drives `list_tools`, so the
surface genuinely shrinks/grows per persona. **Backward-compatible:** an explicit
tool-profile always wins, so existing coding installs are untouched.
**Payoff:** **one runtime, many verticals.** A sales, research, support or data
agent gets a surface built for its domain — and the `coding` default behaves
exactly as before.
---
## Journey 7 — "Feed it my PDFs, CRM exports and emails" → Universal Intake
**Persona:** Nadia, a research analyst. Her corpus is reports (PDF), web captures
(HTML), CRM exports (CSV/JSON) and a mailbox (EML) — **not source code**.
**The wall:** lean-ctx historically indexed *code*. Nadia's documents never reached
the BM25 / semantic / knowledge stores, so retrieval couldn't see them.
**The journey:** Nadia points the index at a mixed-format directory. The
**ingestion front-door** admits documents and data, and a **format extractor**
picks the right reader per file — no per-format flags:
```python
client.call_tool_text("ctx_index", {"action": "build", "project_root": "./reports"})
hits = client.call_tool_text("ctx_semantic_search", {"query": "Q3 churn drivers"})
```
| Format | What the extractor does |
|---|---|
| **PDF** | local text extraction → paragraph chunks (no upload) |
| **HTML** | rendered to clean Markdown, paragraph-chunked |
| **CSV/TSV** | RFC-4180 parse (quoted fields/embedded delimiters) → labeled record chunks |
| **EML** | salient header summary (From/To/Subject/Date) + `text/plain` body, MIME stripped |
| **JSON/NDJSON** | chunked per array element / object entry |
**Under the hood:** `rust/src/core/ingestion.rs` decides *whether* a file is
indexable (code · document · data · text); `rust/src/core/extractors/`
([`extractors-v1`](contracts/extractors-v1.md)) decides *how* to read it. Each
text format also registers as a named **chunker** in the extension registry, so
it shows up in `/v1/capabilities` and is conformance-checked for determinism and
coverage.
**Payoff:** **any corpus reaches the same engine.** The retrieval, knowledge and
compression that powered coding agents now power a research, support or data agent
over the documents that domain actually uses.
---
## Journey 8 — "Extend it — without forking the engine" → Open Core
**Persona:** Priya, a platform engineer. She needs a **domain tool** (an internal
lookup) and a **custom compressor** for her data shape. Forking and maintaining a
patched build is a non-starter.
**The wall:** historically, adding a tool or a transform meant forking
`build_registry()`. That doesn't scale to a team or survive upgrades.
**The journey — three escalating options, no fork:**
1. **A tool the agent can call** — declare it in a plugin manifest; lean-ctx
registers it as a native MCP tool at startup and advertises it in
`/v1/capabilities`:
```toml
# plugin.toml
[plugin]
name = "crm"
version = "0.1.0"
[[tools]]
name = "crm_lookup"
description = "Look up an account in the CRM"
command = "crm-bin" # gets JSON args on stdin, returns text on stdout
timeout_ms = 8000
input_schema = { type = "object", properties = { account = { type = "string" } }, required = ["account"] }
[trust]
permissions = ["network"] # declared + surfaced for consent
```
2. **React to lifecycle events** — a hook command per event
(`pre_read`, `post_compress`, `on_knowledge_update`, `on_session_*`); zero-cost
when nothing listens.
3. **A custom compressor / read-mode / chunker** — compile it to a sandboxed
**WASM** module (any language) and drop it in a directory; lean-ctx discovers
it by file stem and registers it as a first-class extension:
```bash
export LEAN_CTX_WASM_DIR=~/.lean-ctx/wasm # *.wasm → registered compressors
lean-ctx conformance # your extension is checked like a built-in
```
```text
/v1/capabilities → extensions.compressors: ["identity","markdown","prose","whitespace","my_ext"]
conformance scorecard → [ok] extensions/compressor:my_ext
```
**Under the hood:** plugins live in `rust/src/core/plugins/` (manifest tools +
fired hooks), the WASM host in `rust/src/core/wasm_ext.rs`
(`wasm-abi-v1`: `alloc` + `lctx_compress`/`lctx_provider_fetch`, **host-enforced**
byte budget so a faulty guest can never overrun). Everything is governed by the
[extension trust model](contracts/extension-trust-v1.md) (scrubbed env + cwd jail
+ timeout, declared permissions surfaced for consent) and proven by
[`conformance-v1`](contracts/conformance-v1.md).
**Payoff:** the engine is **extensible in any language, sandboxed, discoverable
and conformance-checked** — your tools and transforms are first-class without ever
touching lean-ctx's source.
---
## Journey 9 — "Roll it out to my team & prove ROI — without gating my devs" → Commercial Plane
**Persona:** Marco, an engineering manager. He wants shared coordination, RBAC and
a procurement-grade ROI number — **without** taking away anything his developers
get for free locally.
**The wall:** most tools monetize by gating features. Marco needs the opposite:
local stays best-in-class and ungated; only **team coordination** is paid.
**The journey:**
1. **Local stays free.** Billing is informational only — it *describes* plans and
*meters* local savings; it never gates a local capability:
```bash
lean-ctx billing plans # free · supporter · pro · team · business · enterprise
lean-ctx billing usage --json # metered from the signed ledger, read-only
```
2. **Team coordination, with real RBAC.** Issue role-scoped tokens and serve a
shared, audited team endpoint:
```bash
lean-ctx team token create --config team.json --id alice --role viewer # viewer·member·admin·owner
lean-ctx team serve --config team.json
```
A `viewer` may search but is denied mutations/audit (`403 scope_denied`); an
`admin` has the full scope set — every decision written to an audit log.
3. **Prove the value** with a reproducible, signed ROI artifact:
```bash
lean-ctx savings roi # net tokens, USD, top tools — SHA-256 chain + Ed25519 signature
```
**Under the hood:** the Team/Org plane is `rust/src/http_server/team/` (bearer
auth, `TeamRole` → scope expansion in `roles.rs`, per-request audit). Billing is
`rust/src/core/billing/``entitlement_allows` returns **`true` for every local
feature on every plan**, the billing-layer expression of the **Local-Free
Invariant**. The savings ledger (`rust/src/core/savings_ledger/`) is the metering
substrate. The invariant is not a promise but a **CI gate**:
[`local-free-invariant-v1`](contracts/local-free-invariant-v1.md) fails the build
if any local capability is ever put behind an account, license or plan.
**Payoff:** a genuinely monetizable **Team/Cloud plane** that adds coordination,
governance and scale — while the local engine every developer runs stays **free,
ungated and best-in-class, forever**.
---
## How it all connects
```mermaid
flowchart LR
subgraph Sources
N[Native tools<br/>read / search / shell]
G[Downstream MCP servers<br/>via ctx_tools gateway]
K[Knowledge facts]
end
N --> P
G --> P
K --> P
P{{Pre-prompt pipeline}}
P --> F[Context Firewall<br/>large output → digest + ref]
F --> S[Sensitivity Floor<br/>redact / drop ≥ policy_floor]
S --> M[(Model context window)]
SC[benchmark scorecard] -. measures .-> P
```
- The **gateway** widens what can *enter* the pipeline (unbounded external tools)
without widening the window.
- The **firewall** caps the *size* of anything that enters.
- The **sensitivity floor** caps the *sensitivity* of anything that enters.
- The **scorecard** measures the whole pipeline, reproducibly.
Because they share one choke point, a downstream gateway result is firewalled and
sensitivity-checked exactly like a native one — no feature can be bypassed by
routing around it.
**Act II wraps this same pipeline.** SDKs and the `/v1` API are the *door* into it;
personas *shape* it per domain; ingestion + extractors *widen what can enter* from
any corpus; plugins and WASM *extend* the transforms inside it — all discoverable
via `/v1/capabilities` and conformance-checked. The Team/Cloud plane adds
coordination *around* the runtime without ever reaching in to gate a local feature.
---
## What changed under the hood (engineering summary)
| Feature | New / changed code | Tests | Config / surface |
|---|---|---|---|
| **MCP Gateway** | `core/gateway/{config,client,catalog,router,mod}.rs`, `tools/ctx_tools.rs`, `tools/registered/ctx_tools.rs` | `tests/gateway_e2e.rs` (in-process `rmcp` echo server), gateway unit tests | `[gateway]`, `[[gateway.servers]]`; tool `ctx_tools` (granular surface → **72**) |
| **Context Firewall** | `core/firewall.rs` (`is_protected_read` SSOT, digest builder) | firewall + `archive_expand_tests` | `[archive].ephemeral`, `ephemeral_min_tokens` |
| **Sensitivity Floor** | `core/sensitivity/{mod,classify}.rs`, `enforce_text` choke point | `tests/sensitivity_floor.rs` (8) | `[sensitivity]` (`enabled`, `policy_floor`, `action`) |
| **Scorecard** | `core/scorecard/{mod,scenarios}.rs`, `benchmark scorecard` CLI | `tests/scorecard_determinism.rs` (2) | `lean-ctx benchmark scorecard [--json]` |
| **Open Door (SDKs + `/v1`)** | `http_server/` (`/v1/capabilities`, `/v1/openapi.json`), `clients/{python,rust}`, `cookbook/sdk` | SDK conformance kits; `tests/capabilities_*`, OpenAPI drift | `lean-ctx serve`; `lean-ctx-client` |
| **Context Personas** | `core/persona.rs`, `core/config` resolution | persona resolution + tool-surface tests | `LEAN_CTX_PERSONA`, `config.persona`; `<personas_dir>/*.toml` |
| **Universal Intake** | `core/ingestion.rs`, `core/extractors/{json,csv,eml,html,pdf,text}.rs` | extractor + chunker conformance | `ctx_index`; auto per-format |
| **Open Core (plugins + WASM)** | `core/plugins/`, `core/wasm_ext.rs` (`wasm-abi-v1`) | plugin/hook + WASM host tests; `conformance` | `plugin.toml` `[[tools]]`, hooks; `LEAN_CTX_WASM_DIR` |
| **Commercial Plane** | `http_server/team/`, `core/billing/`, `core/savings_ledger/` | RBAC + billing + `local-free-invariant` CI gate | `lean-ctx team`, `billing`, `savings roi` |
**Cross-cutting consistency (this pass):** every "tool count" reference across
README, `ARCHITECTURE.md`, `VISION.md`, guides, comparisons, Discord FAQ,
marketing and skills was reconciled to the runtime SSOT of **72** tools — enforced
by `tests/docs_tool_counts_up_to_date.rs`, which fails CI on drift. The generated
appendices ([MCP tools](reference/generated/mcp-tools.md),
[config keys](reference/generated/config-keys.md)) and the website manifest are
regenerated from code.
---
## Where to go next
- **Build your own tool (developer map):** [The Context OS Guide](context-os/guide.md)
- **Non-coding recipes (lead-gen / research / support / data):** [Non-Coding Cookbook](context-os/cookbook-non-coding.md)
- **The vision & rationale:** [Context OS RFC v1](context-os/rfc-v1.md)
- **The contracts you build against:** [docs/contracts/](contracts/) — capabilities · persona-spec · extractors · extension-trust · wasm-abi · conformance · local-free-invariant
- **Full feature reference, as journeys:** [docs/reference/README.md](reference/README.md)
- **The gateway in depth:** [Advanced & Integrations §10](reference/05-advanced.md)
- **Security surface:** [Security & Governance](reference/13-security-and-governance.md)
- **Always-current tool list:** [generated MCP tools](reference/generated/mcp-tools.md)