wehub-resource-sync
26382a7ac6
CI / Clippy (push) Failing after 15m13s
CI / Test (ubuntu-latest) (push) Failing after 16m1s
CI / Test (macos-latest) (push) Has been cancelled
CI / Test (windows-latest) (push) Has been cancelled
CI / Build (no embeddings / no ORT) (push) Has been cancelled
CI / Format (push) Has been cancelled
CI / Cookbook (Node) (push) Has been cancelled
CI / Pi Extension (Node) (push) Has been cancelled
CI / Rust SDK (lean-ctx-client) (push) Has been cancelled
CI / Embed SDK (lean-ctx-sdk) (push) Has been cancelled
CI / Python SDK (leanctx) (push) Has been cancelled
CI / Hermes Plugin (Python) (push) Has been cancelled
CI / SDK Conformance Matrix (push) Has been cancelled
CI / Coverage (push) Has been cancelled
CI / cargo-deny (push) Has been cancelled
CI / Adversarial Safety (push) Has been cancelled
CI / Benchmarks (push) Has been cancelled
CI / Output-Quality Gate (eval A/B) (push) Has been cancelled
CI / Documentation (push) Has been cancelled
CI / CI Green (push) Has been cancelled
JetBrains Plugin / Actionlint (push) Has been cancelled
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (rust) (push) Has been cancelled
JetBrains Plugin / Validation (push) Has been cancelled
JetBrains Plugin / Build (push) Has been cancelled
JetBrains Plugin / Test (push) Has been cancelled
Security Check / Security Scan (push) Has been cancelled
30 lines
1.2 KiB
Markdown
30 lines
1.2 KiB
Markdown
# Evaluation / Quality Gates (local-first)
|
||
|
||
This folder contains **small, deterministic** evaluation tests that act as CI quality gates.
|
||
|
||
## What’s covered
|
||
|
||
- **Graph-driven context**: `core::graph_context` must include the expected related files for a minimal fixture project.
|
||
- **Knowledge embeddings**: `core::knowledge_embedding` semantic search must return the expected top hits (feature-gated).
|
||
|
||
## Where the gates live
|
||
|
||
- `p2_graph_embeddings_eval.rs`
|
||
|
||
## How to extend fixtures (rules)
|
||
|
||
- **Deterministic**: avoid timestamps in assertions; don’t depend on OS-specific paths.
|
||
- **Small**: keep fixture projects to a handful of files and minimal content.
|
||
- **Local-first**: do not require network access or external services.
|
||
- **Must-include assertions**: prefer “top‑k contains X” over brittle full ordering checks.
|
||
- **Isolation**: set `LEAN_CTX_DATA_DIR` to a temp directory inside the test (and unset afterwards).
|
||
|
||
## Adding a new gate
|
||
|
||
Add a new `#[test]` in `p2_graph_embeddings_eval.rs`:
|
||
|
||
- Create a tiny fixture project in a temp dir.
|
||
- Build/run the component under test.
|
||
- Assert a bounded invariant (e.g. “must include file A” / “top hit must be key=K”).
|
||
|