feat: surface telemetry drift in skill atlas
This commit is contained in:
yaojingang
@@ -32,6 +32,7 @@ It turns rough workflows, transcripts, prompts, notes, and runbooks into reusabl
|
||||
- three high-value next iteration directions after the first package is created
|
||||
- a lightweight feedback log that does not require a full promotion cycle
|
||||
- a local-first metadata-only adoption and drift report that turns real usage signals into next iteration candidates
|
||||
- a Skill Atlas drift layer that reads aggregate adoption reports and surfaces portfolio-level drift signals without packaging raw telemetry logs
|
||||
- a baseline compare report for with-skill vs baseline review
|
||||
- a conversation-style, archetype-aware quickstart that steers new packages toward scaffold, production, library, or governed fits
|
||||
- Skill IR as the platform-neutral semantic contract, plus compiler reports and client-specific adapters
|
||||
@@ -406,7 +407,7 @@ Utility scripts that make the meta-skill operational:
|
||||
- `render_review_annotations.py`: records reviewer annotations tied to Review Studio gates, source/report paths, and optional line numbers, with open blocker annotations reflected in Review Studio decisions
|
||||
- `run_conformance_suite.py`: verifies runtime conformance for OpenAI, Claude, Agent Skills, VS Code/Copilot-style, and generic targets
|
||||
- `trust_check.py`: generates the trust/security report for scripts, dependencies, secret risk, bounded network host policy, execution-level `--help` smoke checks, permission inputs, trust metadata, and stable source-contract integrity
|
||||
- `build_skill_atlas.py`: builds the Skill Atlas catalog, route-overlap matrix, dependency graph, stale report, owner gaps, and HTML overview for a multi-skill workspace
|
||||
- `build_skill_atlas.py`: builds the Skill Atlas catalog, route-overlap matrix, dependency graph, stale report, owner gaps, aggregate drift signals, and HTML overview for a multi-skill workspace
|
||||
- `registry_audit.py`: builds registry package metadata and audits version, owner, license, checksum, Skill IR source, and compatibility matrix
|
||||
- `verify_package.py`: verifies generated package manifests, target adapters, zip archive safety, archive checksum, and registry parity
|
||||
- `simulate_install.py`: extracts a generated zip into a temporary skill root and verifies entrypoint, manifest, interface, reports, and adapters can be loaded
|
||||
|
||||
@@ -41,6 +41,8 @@ This document tracks the first migration path from Yao Meta Skill 1.x to the Ski
|
||||
python3 scripts/build_skill_atlas.py --workspace-root . --output-dir skill_atlas --report-html reports/skill_atlas.html --report-json reports/skill_atlas.json
|
||||
```
|
||||
|
||||
This also writes `skill_atlas/drift_signals.json` from aggregate `reports/adoption_drift_report.json` files. Do not migrate or publish raw `reports/telemetry_events.jsonl` logs.
|
||||
|
||||
5. Keep legacy trigger eval gates intact while output eval coverage grows.
|
||||
|
||||
6. Move adapter-specific logic toward future compiler commands after IR fields are stable.
|
||||
|
||||
@@ -16,6 +16,7 @@ Single-skill quality is not enough for a team library. A skill portfolio also ne
|
||||
- Flag missing owner or review metadata.
|
||||
- Flag stale skills based on `updated_at` and `review_cadence`.
|
||||
- Extract no-route opportunities from failure notes.
|
||||
- Read aggregate adoption drift reports and flag telemetry drift without reading raw telemetry logs.
|
||||
|
||||
## Scope Policy
|
||||
|
||||
@@ -23,6 +24,12 @@ Atlas keeps a full catalog, but release gates should distinguish actionable libr
|
||||
|
||||
Use `skill_atlas/policy.json` to mark path prefixes as non-actionable when they are intentionally retained as examples, evolution snapshots, embedded generated skills, or validator fixtures. Non-actionable items still appear in the full report, route matrix, stale list, and owner gap list, but Review Studio should use the actionable counts for release readiness.
|
||||
|
||||
## Telemetry Link
|
||||
|
||||
Atlas may read each skill's aggregate `reports/adoption_drift_report.json` to surface portfolio drift signals such as no telemetry for production/library/governed skills, missed triggers, bad outputs, missing resources, script errors, and review-overdue counts. It must not read or package `reports/telemetry_events.jsonl`; raw telemetry remains local-only evidence owned by the skill.
|
||||
|
||||
Write drift output to `skill_atlas/drift_signals.json`. Non-actionable scopes stay visible in that file and in the HTML report, but only actionable drift signals should affect release readiness.
|
||||
|
||||
## Reviewer Gate
|
||||
|
||||
Use Atlas before promoting a single skill into a shared library. If an actionable route collision, missing owner, or stale governed skill appears, fix the portfolio boundary before adding more local complexity to one skill. Non-actionable issues should stay visible as evidence, not as release blockers.
|
||||
Use Atlas before promoting a single skill into a shared library. If an actionable route collision, missing owner, stale governed skill, or telemetry drift signal appears, fix the portfolio boundary before adding more local complexity to one skill. Non-actionable issues should stay visible as evidence, not as release blockers.
|
||||
|
||||
@@ -54,7 +54,12 @@ Package builders should exclude `reports/telemetry_events.jsonl`. The root repos
|
||||
4. Convert bad outputs into Output Eval assertions and failure taxonomy entries.
|
||||
5. Convert script errors into non-interactive smoke tests.
|
||||
6. Feed review-overdue signals back into Skill Atlas and owner review.
|
||||
7. Let Skill Atlas read only `reports/adoption_drift_report.json` and publish portfolio-level `skill_atlas/drift_signals.json`.
|
||||
|
||||
## Review Studio Role
|
||||
|
||||
Review Studio should show the aggregate telemetry gate as an operating loop, not as raw logs. A blocker means the telemetry contract was violated. A warning means the evidence is absent or the drift signal needs a follow-up case.
|
||||
|
||||
## Skill Atlas Role
|
||||
|
||||
Skill Atlas uses aggregate adoption drift reports to rank portfolio work. It should surface no-data warnings for actionable production/library/governed skills, and drift warnings for missed triggers, wrong triggers, bad outputs, missing resources, script errors, and review-overdue counts. It must not inspect raw JSONL telemetry or use non-actionable example/fixture signals as release blockers.
|
||||
|
||||
+1
-1
@@ -16,7 +16,7 @@
|
||||
"vscode"
|
||||
],
|
||||
"package_metadata": "registry/packages/yao-meta-skill.json",
|
||||
"package_sha256": "7a8dab0d542d2994adec2045b54559612db36f14db8b7b66e4138fb0435ae3d3"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -16,8 +16,8 @@
|
||||
"trust_level": "local",
|
||||
"license": "MIT",
|
||||
"checksums": {
|
||||
"package_sha256": "7a8dab0d542d2994adec2045b54559612db36f14db8b7b66e4138fb0435ae3d3",
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb",
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20"
|
||||
},
|
||||
"compatibility": {
|
||||
"openai": "pass",
|
||||
@@ -48,7 +48,7 @@
|
||||
},
|
||||
"distribution": {
|
||||
"archive_verified": true,
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060",
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20",
|
||||
"package_verification": "reports/package_verification.json",
|
||||
"install_simulated": true,
|
||||
"install_simulation": "reports/install_simulation.json"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"ok": true,
|
||||
"schema_version": "2.0",
|
||||
"generated_at": "2026-06-13T12:15:07Z",
|
||||
"generated_at": "2026-06-13T12:26:25Z",
|
||||
"skill_dir": ".",
|
||||
"privacy_contract": {
|
||||
"storage": "local-first",
|
||||
|
||||
@@ -6,10 +6,10 @@
|
||||
"context_budget_tier": "production",
|
||||
"context_budget_limit": 1000,
|
||||
"skill_body_tokens": 751,
|
||||
"other_text_tokens": 905782,
|
||||
"other_text_tokens": 910665,
|
||||
"estimated_initial_load_tokens": 944,
|
||||
"estimated_total_text_tokens": 906533,
|
||||
"relevant_file_count": 386,
|
||||
"estimated_total_text_tokens": 911416,
|
||||
"relevant_file_count": 388,
|
||||
"unused_resource_dirs": [],
|
||||
"quality_signal_points": 130,
|
||||
"quality_density": 137.7
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
"installed_skill_dir": "dist/install-simulation/simulate-yao-meta-skill/yao-meta-skill",
|
||||
"summary": {
|
||||
"archive_present": true,
|
||||
"archive_entry_count": 493,
|
||||
"archive_entry_count": 494,
|
||||
"archive_extracted": true,
|
||||
"entrypoint_loaded": true,
|
||||
"manifest_loaded": true,
|
||||
|
||||
@@ -34,7 +34,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 28.86,
|
||||
"duration_ms": 31.8,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -62,7 +62,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 28.72,
|
||||
"duration_ms": 31.63,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -85,7 +85,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 29.59,
|
||||
"duration_ms": 30.68,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -113,7 +113,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 30.19,
|
||||
"duration_ms": 32.69,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -136,7 +136,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 31.03,
|
||||
"duration_ms": 32.73,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -164,7 +164,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 29.71,
|
||||
"duration_ms": 34.03,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -187,7 +187,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 32.23,
|
||||
"duration_ms": 34.95,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -214,7 +214,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 31.55,
|
||||
"duration_ms": 34.6,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -237,7 +237,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 32.21,
|
||||
"duration_ms": 33.14,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -266,7 +266,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 31.89,
|
||||
"duration_ms": 36.79,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
|
||||
@@ -23,16 +23,16 @@ Command runner evidence is present. This proves the eval harness executed an ext
|
||||
|
||||
| Case | Variant | Mode | Model | Duration ms | Tokens | Score | Status |
|
||||
| --- | --- | --- | --- | ---: | ---: | ---: | --- |
|
||||
| skill-package-contract | baseline | command | local-output-eval-runner | 28.86 | 33 | 0.0 | pass |
|
||||
| skill-package-contract | with_skill | command | local-output-eval-runner | 28.72 | 73 | 100.0 | pass |
|
||||
| output-eval-expectation | baseline | command | local-output-eval-runner | 29.59 | 36 | 0.0 | pass |
|
||||
| output-eval-expectation | with_skill | command | local-output-eval-runner | 30.19 | 80 | 100.0 | pass |
|
||||
| ir-before-packaging | baseline | command | local-output-eval-runner | 31.03 | 33 | 0.0 | pass |
|
||||
| ir-before-packaging | with_skill | command | local-output-eval-runner | 29.71 | 80 | 100.0 | pass |
|
||||
| near-neighbor-boundary | baseline | command | local-output-eval-runner | 32.23 | 36 | 0.0 | pass |
|
||||
| near-neighbor-boundary | with_skill | command | local-output-eval-runner | 31.55 | 65 | 100.0 | pass |
|
||||
| file-backed-governed-package | baseline | command | local-output-eval-runner | 32.21 | 37 | 0.0 | pass |
|
||||
| file-backed-governed-package | with_skill | command | local-output-eval-runner | 31.89 | 98 | 100.0 | pass |
|
||||
| skill-package-contract | baseline | command | local-output-eval-runner | 31.8 | 33 | 0.0 | pass |
|
||||
| skill-package-contract | with_skill | command | local-output-eval-runner | 31.63 | 73 | 100.0 | pass |
|
||||
| output-eval-expectation | baseline | command | local-output-eval-runner | 30.68 | 36 | 0.0 | pass |
|
||||
| output-eval-expectation | with_skill | command | local-output-eval-runner | 32.69 | 80 | 100.0 | pass |
|
||||
| ir-before-packaging | baseline | command | local-output-eval-runner | 32.73 | 33 | 0.0 | pass |
|
||||
| ir-before-packaging | with_skill | command | local-output-eval-runner | 34.03 | 80 | 100.0 | pass |
|
||||
| near-neighbor-boundary | baseline | command | local-output-eval-runner | 34.95 | 36 | 0.0 | pass |
|
||||
| near-neighbor-boundary | with_skill | command | local-output-eval-runner | 34.6 | 65 | 100.0 | pass |
|
||||
| file-backed-governed-package | baseline | command | local-output-eval-runner | 33.14 | 37 | 0.0 | pass |
|
||||
| file-backed-governed-package | with_skill | command | local-output-eval-runner | 36.79 | 98 | 100.0 | pass |
|
||||
|
||||
## Next Fixes
|
||||
|
||||
|
||||
@@ -8,8 +8,8 @@
|
||||
"target_count": 4,
|
||||
"adapter_count": 4,
|
||||
"archive_present": true,
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060",
|
||||
"archive_entry_count": 493,
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20",
|
||||
"archive_entry_count": 494,
|
||||
"failure_count": 0,
|
||||
"warning_count": 0
|
||||
},
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
- Package directory: `dist`
|
||||
- Targets: `4 / 4` adapters present
|
||||
- Archive present: `True`
|
||||
- Archive SHA256: `176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060`
|
||||
- Archive SHA256: `23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20`
|
||||
- Failures: `0`
|
||||
- Warnings: `0`
|
||||
|
||||
|
||||
@@ -21,8 +21,8 @@
|
||||
"trust_level": "local",
|
||||
"license": "MIT",
|
||||
"checksums": {
|
||||
"package_sha256": "7a8dab0d542d2994adec2045b54559612db36f14db8b7b66e4138fb0435ae3d3",
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb",
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20"
|
||||
},
|
||||
"compatibility": {
|
||||
"openai": "pass",
|
||||
@@ -53,7 +53,7 @@
|
||||
},
|
||||
"distribution": {
|
||||
"archive_verified": true,
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060",
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20",
|
||||
"package_verification": "reports/package_verification.json",
|
||||
"install_simulated": true,
|
||||
"install_simulation": "reports/install_simulation.json"
|
||||
@@ -78,7 +78,7 @@
|
||||
"vscode"
|
||||
],
|
||||
"package_metadata": "registry/packages/yao-meta-skill.json",
|
||||
"package_sha256": "7a8dab0d542d2994adec2045b54559612db36f14db8b7b66e4138fb0435ae3d3"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -6,8 +6,8 @@
|
||||
- Maturity: `governed`
|
||||
- Owner: `Yao Team`
|
||||
- License: `MIT`
|
||||
- Package SHA256: `7a8dab0d542d2994adec2045b54559612db36f14db8b7b66e4138fb0435ae3d3`
|
||||
- Archive SHA256: `176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060`
|
||||
- Package SHA256: `ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb`
|
||||
- Archive SHA256: `23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20`
|
||||
- Install simulated: `True`
|
||||
|
||||
## Compatibility
|
||||
|
||||
@@ -242,12 +242,12 @@
|
||||
|
||||
<section>
|
||||
<h2>核心指标</h2>
|
||||
<div class="metrics"><article class='metric'><span>Skill IR</span><strong>2.0.0</strong><p>5 targets in platform-neutral contract</p></article><article class='metric'><span>Compiler</span><strong>5/5</strong><p>target contracts compiled from Skill IR</p></article><article class='metric'><span>Output Delta</span><strong>100.0</strong><p>5 cases; 1 file-backed</p></article><article class='metric'><span>Exec Runs</span><strong>10</strong><p>command 10; model 0; recorded 0</p></article><article class='metric'><span>Blind A/B</span><strong>5</strong><p>review pairs hide baseline vs with-skill labels</p></article><article class='metric'><span>Review A/B</span><strong>0/5</strong><p>adjudication decisions; pending 5</p></article><article class='metric'><span>Runtime</span><strong>5/5</strong><p>target conformance pass rate</p></article><article class='metric'><span>Perm Probe</span><strong>4/4</strong><p>0 native enforcement targets</p></article><article class='metric'><span>Trust</span><strong>0</strong><p>69 scripts scanned; secrets found</p></article><article class='metric'><span>Atlas</span><strong>5</strong><p>12 scanned skills; route collisions</p></article><article class='metric'><span>Drift</span><strong>low</strong><p>1 metadata events; 0 missed triggers</p></article><article class='metric'><span>Waivers</span><strong>0</strong><p>0 gates covered; human risk decisions</p></article><article class='metric'><span>Notes</span><strong>0/0</strong><p>0 open blocker annotations</p></article><article class='metric'><span>Registry</span><strong>1.1.0</strong><p>5 targets; MIT license</p></article><article class='metric'><span>Archive</span><strong>pass</strong><p>493 zip entries; package verification</p></article><article class='metric'><span>Install</span><strong>pass</strong><p>4 adapters readable; local install simulation</p></article><article class='metric'><span>Upgrade</span><strong>minor</strong><p>declared minor; 0 breaking changes</p></article></div>
|
||||
<div class="metrics"><article class='metric'><span>Skill IR</span><strong>2.0.0</strong><p>5 targets in platform-neutral contract</p></article><article class='metric'><span>Compiler</span><strong>5/5</strong><p>target contracts compiled from Skill IR</p></article><article class='metric'><span>Output Delta</span><strong>100.0</strong><p>5 cases; 1 file-backed</p></article><article class='metric'><span>Exec Runs</span><strong>10</strong><p>command 10; model 0; recorded 0</p></article><article class='metric'><span>Blind A/B</span><strong>5</strong><p>review pairs hide baseline vs with-skill labels</p></article><article class='metric'><span>Review A/B</span><strong>0/5</strong><p>adjudication decisions; pending 5</p></article><article class='metric'><span>Runtime</span><strong>5/5</strong><p>target conformance pass rate</p></article><article class='metric'><span>Perm Probe</span><strong>4/4</strong><p>0 native enforcement targets</p></article><article class='metric'><span>Trust</span><strong>0</strong><p>69 scripts scanned; secrets found</p></article><article class='metric'><span>Atlas</span><strong>5</strong><p>12 scanned skills; route collisions</p></article><article class='metric'><span>Drift</span><strong>low</strong><p>1 metadata events; 0 missed triggers</p></article><article class='metric'><span>Waivers</span><strong>0</strong><p>0 gates covered; human risk decisions</p></article><article class='metric'><span>Notes</span><strong>0/0</strong><p>0 open blocker annotations</p></article><article class='metric'><span>Registry</span><strong>1.1.0</strong><p>5 targets; MIT license</p></article><article class='metric'><span>Archive</span><strong>pass</strong><p>494 zip entries; package verification</p></article><article class='metric'><span>Install</span><strong>pass</strong><p>4 adapters readable; local install simulation</p></article><article class='metric'><span>Upgrade</span><strong>minor</strong><p>declared minor; 0 breaking changes</p></article></div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>审查闸门</h2>
|
||||
<div class="gates"><article class='gate pass'><div><span>通过</span><h3>意图画布</h3></div><p>intent confidence 100/100; Intent is clear enough to package the first routeable version.</p><footer>reports/intent-confidence.json <a href='intent-confidence.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>触发实验</h3></div><p>13 trigger cases; 0 misroutes; 0 ambiguous</p><footer>reports/route_scorecard.json <a href='route_scorecard.md'>证据</a></footer></article><article class='gate warn'><div><span>关注</span><h3>输出实验</h3></div><p>5/5 cases; with-skill 100.0; baseline 0.0; file-backed 1; near-neighbor 1; blind A/B 5; exec 10; command 10; model 0; recorded 0; reviewed 0/5; review pending 5</p><footer>reports/output_quality_scorecard.json <a href='output_quality_scorecard.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>上下文</h3></div><p>initial load 944/1000; quality density 137.7</p><footer>reports/context_budget.json <a href='context_budget.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>运行矩阵</h3></div><p>5 / 5 targets pass</p><footer>reports/conformance_matrix.json <a href='conformance_matrix.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>信任报告</h3></div><p>0 secrets; 69 scripts; 3 network-capable scripts; 0 help smoke failures</p><footer>reports/security_trust_report.json <a href='security_trust_report.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>权限批准</h3></div><p>3/3 permissions approved; gaps 0; required file_write, network, subprocess</p><footer>reports/security_trust_report.json + security/permission_policy.json <a href='../security/permission_policy.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>权限探针</h3></div><p>4/4 targets probed; native 0; metadata fallback 4; residual risks 4</p><footer>reports/runtime_permission_probes.json <a href='runtime_permission_probes.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>组合治理</h3></div><p>12 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 24 scoped non-actionable issues</p><footer>reports/skill_atlas.json <a href='skill_atlas.html'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>运营回路</h3></div><p>1 metadata events; adoption 100.0; missed 0; bad-output 0; risk low</p><footer>reports/adoption_drift_report.json <a href='adoption_drift_report.md'>证据</a></footer></article><article class='gate warn'><div><span>关注</span><h3>人工批准</h3></div><p>0 active waivers; 1 warning gates still need reviewer decision</p><footer>reports/review_waivers.json <a href='review_waivers.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>注册审计</h3></div><p>yao-meta-skill 1.1.0; 6/6 compatibility entries pass; install pass with 4 adapters</p><footer>reports/registry_audit.json + reports/install_simulation.json <a href='registry_audit.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>发布路线</h3></div><p>0 promote; 3 keep current; 0 blocked; upgrade minor declared / minor recommended</p><footer>reports/promotion_decisions.json + reports/upgrade_check.json + docs/migration-v2.md <a href='promotion_decisions.md'>证据</a></footer></article></div>
|
||||
<div class="gates"><article class='gate pass'><div><span>通过</span><h3>意图画布</h3></div><p>intent confidence 100/100; Intent is clear enough to package the first routeable version.</p><footer>reports/intent-confidence.json <a href='intent-confidence.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>触发实验</h3></div><p>13 trigger cases; 0 misroutes; 0 ambiguous</p><footer>reports/route_scorecard.json <a href='route_scorecard.md'>证据</a></footer></article><article class='gate warn'><div><span>关注</span><h3>输出实验</h3></div><p>5/5 cases; with-skill 100.0; baseline 0.0; file-backed 1; near-neighbor 1; blind A/B 5; exec 10; command 10; model 0; recorded 0; reviewed 0/5; review pending 5</p><footer>reports/output_quality_scorecard.json <a href='output_quality_scorecard.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>上下文</h3></div><p>initial load 944/1000; quality density 137.7</p><footer>reports/context_budget.json <a href='context_budget.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>运行矩阵</h3></div><p>5 / 5 targets pass</p><footer>reports/conformance_matrix.json <a href='conformance_matrix.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>信任报告</h3></div><p>0 secrets; 69 scripts; 3 network-capable scripts; 0 help smoke failures</p><footer>reports/security_trust_report.json <a href='security_trust_report.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>权限批准</h3></div><p>3/3 permissions approved; gaps 0; required file_write, network, subprocess</p><footer>reports/security_trust_report.json + security/permission_policy.json <a href='../security/permission_policy.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>权限探针</h3></div><p>4/4 targets probed; native 0; metadata fallback 4; residual risks 4</p><footer>reports/runtime_permission_probes.json <a href='runtime_permission_probes.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>组合治理</h3></div><p>12 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 0 actionable drift; 24 scoped non-actionable issues</p><footer>reports/skill_atlas.json <a href='skill_atlas.html'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>运营回路</h3></div><p>1 metadata events; adoption 0; missed 0; bad-output 0; risk low</p><footer>reports/adoption_drift_report.json <a href='adoption_drift_report.md'>证据</a></footer></article><article class='gate warn'><div><span>关注</span><h3>人工批准</h3></div><p>0 active waivers; 1 warning gates still need reviewer decision</p><footer>reports/review_waivers.json <a href='review_waivers.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>注册审计</h3></div><p>yao-meta-skill 1.1.0; 6/6 compatibility entries pass; install pass with 4 adapters</p><footer>reports/registry_audit.json + reports/install_simulation.json <a href='registry_audit.md'>证据</a></footer></article><article class='gate pass'><div><span>通过</span><h3>发布路线</h3></div><p>0 promote; 3 keep current; 0 blocked; upgrade minor declared / minor recommended</p><footer>reports/promotion_decisions.json + reports/upgrade_check.json + docs/migration-v2.md <a href='promotion_decisions.md'>证据</a></footer></article></div>
|
||||
</section>
|
||||
|
||||
<div class="twocol">
|
||||
@@ -279,7 +279,7 @@
|
||||
|
||||
<section id="trigger" class="twocol">
|
||||
<div class="panel"><h2>触发实验</h2><p>13 trigger cases; 0 misroutes; 0 ambiguous</p></div>
|
||||
<div class="panel"><h2>组合治理</h2><dl class='kv-grid'><div><dt>Skill 数</dt><dd>12</dd></div><div><dt>纳入审查</dt><dd>1</dd></div><div><dt>待处理冲突</dt><dd>0</dd></div><div><dt>待处理 owner</dt><dd>0</dd></div><div><dt>待处理过期</dt><dd>0</dd></div><div><dt>非行动项</dt><dd>24</dd></div></dl></div>
|
||||
<div class="panel"><h2>组合治理</h2><dl class='kv-grid'><div><dt>Skill 数</dt><dd>12</dd></div><div><dt>纳入审查</dt><dd>1</dd></div><div><dt>待处理冲突</dt><dd>0</dd></div><div><dt>待处理 owner</dt><dd>0</dd></div><div><dt>待处理过期</dt><dd>0</dd></div><div><dt>Telemetry Report Count</dt><dd>1</dd></div><div><dt>Actionable Drift Signal Count</dt><dd>0</dd></div><div><dt>Drift Signal Count</dt><dd>0</dd></div><div><dt>非行动项</dt><dd>24</dd></div></dl></div>
|
||||
</section>
|
||||
|
||||
<section id="output" class="twocol">
|
||||
@@ -313,7 +313,7 @@
|
||||
</section>
|
||||
|
||||
<section id="trust" class="twocol">
|
||||
<div class="panel"><h2>信任报告</h2><dl class='kv-grid'><div><dt>Secret</dt><dd>0</dd></div><div><dt>脚本数</dt><dd>69</dd></div><div><dt>网络脚本</dt><dd>3</dd></div><div><dt>Help 失败</dt><dd>0</dd></div><div><dt>包体哈希</dt><dd><code>2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd</code></dd></div></dl></div>
|
||||
<div class="panel"><h2>信任报告</h2><dl class='kv-grid'><div><dt>Secret</dt><dd>0</dd></div><div><dt>脚本数</dt><dd>69</dd></div><div><dt>网络脚本</dt><dd>3</dd></div><div><dt>Help 失败</dt><dd>0</dd></div><div><dt>包体哈希</dt><dd><code>ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb</code></dd></div></dl></div>
|
||||
<div class="panel"><h2>安全边界</h2><p>高风险 secret、远程 inline execution、缺失依赖策略或无法解释的脚本接口应阻断 governed release。</p></div>
|
||||
</section>
|
||||
|
||||
@@ -328,13 +328,13 @@
|
||||
</section>
|
||||
|
||||
<section id="atlas" class="twocol">
|
||||
<div class="panel"><h2>组合治理</h2><p>12 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 24 scoped non-actionable issues</p></div>
|
||||
<div class="panel"><h2>组合治理</h2><p>12 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 0 actionable drift; 24 scoped non-actionable issues</p></div>
|
||||
<div class="panel"><h2>下一动作</h2><p>优先处理真实 portfolio 中的 duplicate names、stale skills、owner gaps，再用运营回路判断真实影响。</p></div>
|
||||
</section>
|
||||
|
||||
<section id="telemetry" class="twocol">
|
||||
<div class="panel"><h2>运营回路</h2><p>1 metadata events; adoption 100.0; missed 0; bad-output 0; risk low</p></div>
|
||||
<div class="panel"><h2>漂移信号</h2><dl class='kv-grid'><div><dt>事件数</dt><dd>1</dd></div><div><dt>采用率</dt><dd>100</dd></div><div><dt>漏触发</dt><dd>0</dd></div><div><dt>Bad Output Count</dt><dd>0</dd></div><div><dt>风险带</dt><dd>low</dd></div></dl></div>
|
||||
<div class="panel"><h2>运营回路</h2><p>1 metadata events; adoption 0; missed 0; bad-output 0; risk low</p></div>
|
||||
<div class="panel"><h2>漂移信号</h2><dl class='kv-grid'><div><dt>事件数</dt><dd>1</dd></div><div><dt>采用率</dt><dd>0</dd></div><div><dt>漏触发</dt><dd>0</dd></div><div><dt>Bad Output Count</dt><dd>0</dd></div><div><dt>风险带</dt><dd>low</dd></div></dl></div>
|
||||
</section>
|
||||
|
||||
<section id="waivers" class="twocol">
|
||||
@@ -344,12 +344,12 @@
|
||||
|
||||
<section id="registry" class="twocol">
|
||||
<div class="panel"><h2>注册审计</h2><p>yao-meta-skill 1.1.0; 6/6 compatibility entries pass; install pass with 4 adapters</p></div>
|
||||
<div class="panel"><h2>包体元数据</h2><dl class='kv-grid'><div><dt>名称</dt><dd>yao-meta-skill</dd></div><div><dt>版本</dt><dd>1.1.0</dd></div><div><dt>Maturity</dt><dd>governed</dd></div><div><dt>Owner</dt><dd>Yao Team</dd></div><div><dt>License</dt><dd>MIT</dd></div><div><dt>信任级别</dt><dd>local</dd></div><div><dt>目标平台</dt><dd>openai, claude, generic, agent-skills-compatible, vscode</dd></div><div><dt>兼容通过</dt><dd>6/6</dd></div><div><dt>归档哈希</dt><dd><code>176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060</code></dd></div></dl></div>
|
||||
<div class="panel"><h2>包体元数据</h2><dl class='kv-grid'><div><dt>名称</dt><dd>yao-meta-skill</dd></div><div><dt>版本</dt><dd>1.1.0</dd></div><div><dt>Maturity</dt><dd>governed</dd></div><div><dt>Owner</dt><dd>Yao Team</dd></div><div><dt>License</dt><dd>MIT</dd></div><div><dt>信任级别</dt><dd>local</dd></div><div><dt>目标平台</dt><dd>openai, claude, generic, agent-skills-compatible, vscode</dd></div><div><dt>兼容通过</dt><dd>6/6</dd></div><div><dt>归档哈希</dt><dd><code>23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20</code></dd></div></dl></div>
|
||||
</section>
|
||||
|
||||
<section id="release" class="twocol">
|
||||
<div class="panel"><h2>发布路线</h2><p>0 promote; 3 keep current; 0 blocked; upgrade minor declared / minor recommended</p></div>
|
||||
<div class="panel"><h2>包体验证</h2><dl class='kv-grid'><div><dt>目标数</dt><dd>4</dd></div><div><dt>Adapter</dt><dd>4</dd></div><div><dt>归档存在</dt><dd>是</dd></div><div><dt>Zip 条目</dt><dd>493</dd></div><div><dt>失败数</dt><dd>0</dd></div><div><dt>警告数</dt><dd>0</dd></div><div><dt>归档哈希</dt><dd><code>176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060</code></dd></div></dl></div>
|
||||
<div class="panel"><h2>包体验证</h2><dl class='kv-grid'><div><dt>目标数</dt><dd>4</dd></div><div><dt>Adapter</dt><dd>4</dd></div><div><dt>归档存在</dt><dd>是</dd></div><div><dt>Zip 条目</dt><dd>494</dd></div><div><dt>失败数</dt><dd>0</dd></div><div><dt>警告数</dt><dd>0</dd></div><div><dt>归档哈希</dt><dd><code>23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20</code></dd></div></dl></div>
|
||||
</section>
|
||||
</main>
|
||||
</body>
|
||||
|
||||
+188
-68
@@ -83,7 +83,7 @@
|
||||
"key": "skill-atlas",
|
||||
"label": "组合治理",
|
||||
"status": "pass",
|
||||
"detail": "12 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 24 scoped non-actionable issues",
|
||||
"detail": "12 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 0 actionable drift; 24 scoped non-actionable issues",
|
||||
"evidence": "reports/skill_atlas.json",
|
||||
"link": "skill_atlas.html"
|
||||
},
|
||||
@@ -91,7 +91,7 @@
|
||||
"key": "operations-loop",
|
||||
"label": "运营回路",
|
||||
"status": "pass",
|
||||
"detail": "1 metadata events; adoption 100.0; missed 0; bad-output 0; risk low",
|
||||
"detail": "1 metadata events; adoption 0; missed 0; bad-output 0; risk low",
|
||||
"evidence": "reports/adoption_drift_report.json",
|
||||
"link": "adoption_drift_report.md"
|
||||
},
|
||||
@@ -355,7 +355,7 @@
|
||||
"label": "上下文成本",
|
||||
"score": 42,
|
||||
"reasons": [
|
||||
"入口约 350 个词/字，references 约 14450 个词/字。",
|
||||
"入口约 350 个词/字，references 约 14610 个词/字。",
|
||||
"分数越高代表上下文成本越低。",
|
||||
"上下文成本偏高，建议压缩入口或拆分 references。"
|
||||
]
|
||||
@@ -459,7 +459,7 @@
|
||||
"已生成 Output Review Adjudication，可记录盲评决策、一致率和待评审项。"
|
||||
],
|
||||
"gaps": [
|
||||
"上下文成本需要补强：入口约 350 个词/字，references 约 14450 个词/字。"
|
||||
"上下文成本需要补强：入口约 350 个词/字，references 约 14610 个词/字。"
|
||||
],
|
||||
"recommendations": [
|
||||
"先改触发边界，再扩展工作流。",
|
||||
@@ -1226,7 +1226,7 @@
|
||||
"interactive_script_count": 0,
|
||||
"package_hash_scope": "source-contract-without-generated-reports",
|
||||
"package_hash_file_count": 153,
|
||||
"package_sha256": "2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
},
|
||||
"skill_atlas": {
|
||||
"skill_count": 12,
|
||||
@@ -1239,6 +1239,9 @@
|
||||
"actionable_stale_count": 0,
|
||||
"shared_resource_count": 0,
|
||||
"no_route_opportunity_count": 0,
|
||||
"telemetry_report_count": 1,
|
||||
"drift_signal_count": 0,
|
||||
"actionable_drift_signal_count": 0,
|
||||
"non_actionable_issue_count": 24
|
||||
},
|
||||
"registry_distribution": {
|
||||
@@ -1261,8 +1264,8 @@
|
||||
"trust_level": "local",
|
||||
"license": "MIT",
|
||||
"checksums": {
|
||||
"package_sha256": "2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd",
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb",
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20"
|
||||
},
|
||||
"compatibility": {
|
||||
"openai": "pass",
|
||||
@@ -1293,7 +1296,7 @@
|
||||
},
|
||||
"distribution": {
|
||||
"archive_verified": true,
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060",
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20",
|
||||
"package_verification": "reports/package_verification.json",
|
||||
"install_simulated": true,
|
||||
"install_simulation": "reports/install_simulation.json"
|
||||
@@ -1309,8 +1312,8 @@
|
||||
"target_count": 4,
|
||||
"adapter_count": 4,
|
||||
"archive_present": true,
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060",
|
||||
"archive_entry_count": 493,
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20",
|
||||
"archive_entry_count": 494,
|
||||
"failure_count": 0,
|
||||
"warning_count": 0
|
||||
},
|
||||
@@ -1321,7 +1324,7 @@
|
||||
"ok": true,
|
||||
"summary": {
|
||||
"archive_present": true,
|
||||
"archive_entry_count": 493,
|
||||
"archive_entry_count": 494,
|
||||
"archive_extracted": true,
|
||||
"entrypoint_loaded": true,
|
||||
"manifest_loaded": true,
|
||||
@@ -1384,12 +1387,12 @@
|
||||
{
|
||||
"field": "archive_sha256",
|
||||
"from": "",
|
||||
"to": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060"
|
||||
"to": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20"
|
||||
},
|
||||
{
|
||||
"field": "package_sha256",
|
||||
"from": "0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"to": "2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd"
|
||||
"to": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1406,14 +1409,14 @@
|
||||
"ok": true,
|
||||
"summary": {
|
||||
"event_count": 1,
|
||||
"adoption_sample_count": 1,
|
||||
"activation_count": 1,
|
||||
"accepted_count": 1,
|
||||
"adoption_sample_count": 0,
|
||||
"activation_count": 0,
|
||||
"accepted_count": 0,
|
||||
"edited_count": 0,
|
||||
"rejected_count": 0,
|
||||
"missed_count": 0,
|
||||
"failed_count": 0,
|
||||
"adoption_rate": 100.0,
|
||||
"adoption_rate": 0,
|
||||
"missed_trigger_count": 0,
|
||||
"wrong_trigger_count": 0,
|
||||
"bad_output_count": 0,
|
||||
@@ -1422,7 +1425,7 @@
|
||||
"review_overdue_count": 0,
|
||||
"risk_band": "low",
|
||||
"event_types": {
|
||||
"skill_activation": 1
|
||||
"review_event": 1
|
||||
},
|
||||
"failure_types": {}
|
||||
},
|
||||
@@ -3119,7 +3122,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 29.87,
|
||||
"duration_ms": 33.11,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -3147,7 +3150,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 30.27,
|
||||
"duration_ms": 31.55,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -3170,7 +3173,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 40.04,
|
||||
"duration_ms": 30.73,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -3198,7 +3201,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 41.0,
|
||||
"duration_ms": 29.84,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -3221,7 +3224,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 37.68,
|
||||
"duration_ms": 29.53,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -3249,7 +3252,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 37.33,
|
||||
"duration_ms": 28.84,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -3272,7 +3275,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 30.73,
|
||||
"duration_ms": 29.54,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -3299,7 +3302,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 30.69,
|
||||
"duration_ms": 29.69,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -3322,7 +3325,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 30.1,
|
||||
"duration_ms": 29.54,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -3351,7 +3354,7 @@
|
||||
"execution_mode": "command",
|
||||
"model_executed": false,
|
||||
"command_executed": true,
|
||||
"duration_ms": 30.27,
|
||||
"duration_ms": 28.91,
|
||||
"provider": "local-output-eval-runner",
|
||||
"model": "",
|
||||
"usage": {
|
||||
@@ -8212,7 +8215,7 @@
|
||||
"interactive_script_count": 0,
|
||||
"package_hash_scope": "source-contract-without-generated-reports",
|
||||
"package_hash_file_count": 153,
|
||||
"package_sha256": "2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
},
|
||||
"failures": [],
|
||||
"warnings": [],
|
||||
@@ -10041,10 +10044,10 @@
|
||||
"context_budget_tier": "production",
|
||||
"context_budget_limit": 1000,
|
||||
"skill_body_tokens": 751,
|
||||
"other_text_tokens": 893597,
|
||||
"other_text_tokens": 910655,
|
||||
"estimated_initial_load_tokens": 944,
|
||||
"estimated_total_text_tokens": 894348,
|
||||
"relevant_file_count": 386,
|
||||
"estimated_total_text_tokens": 911406,
|
||||
"relevant_file_count": 388,
|
||||
"unused_resource_dirs": [],
|
||||
"quality_signal_points": 130,
|
||||
"quality_density": 137.7
|
||||
@@ -10856,6 +10859,9 @@
|
||||
"actionable_stale_count": 0,
|
||||
"shared_resource_count": 0,
|
||||
"no_route_opportunity_count": 0,
|
||||
"telemetry_report_count": 1,
|
||||
"drift_signal_count": 0,
|
||||
"actionable_drift_signal_count": 0,
|
||||
"non_actionable_issue_count": 24
|
||||
},
|
||||
"scope_policy": {
|
||||
@@ -11031,7 +11037,16 @@
|
||||
],
|
||||
"atlas_scope": "release",
|
||||
"actionable": true,
|
||||
"scope_reason": "default release-actionable skill"
|
||||
"scope_reason": "default release-actionable skill",
|
||||
"telemetry": {
|
||||
"report_present": true,
|
||||
"report": "reports/adoption_drift_report.json",
|
||||
"risk_band": "low",
|
||||
"event_count": 1,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "release-orchestrator",
|
||||
@@ -11089,7 +11104,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/complex-release-orchestrator/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -11128,7 +11152,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/evolution-frontend-review/final/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -11151,7 +11184,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/evolution-frontend-review/v0/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -11189,7 +11231,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/evolution-frontend-review/v1/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "incident-command-governor",
|
||||
@@ -11230,7 +11281,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/governed-incident-command/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "note-cleanup",
|
||||
@@ -11267,7 +11327,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/simple-note-cleanup/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -11307,7 +11376,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/team-frontend-review/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "geo-ranking-article-generator",
|
||||
@@ -11367,7 +11445,16 @@
|
||||
],
|
||||
"atlas_scope": "embedded-generated-skill",
|
||||
"actionable": false,
|
||||
"scope_reason": "This nested generated skill is retained as local evidence and should not gate the root Meta Skill release."
|
||||
"scope_reason": "This nested generated skill is retained as local evidence and should not gate the root Meta Skill release.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "geo-ranking-article-generator/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "invalid-governance-skill",
|
||||
@@ -11389,7 +11476,16 @@
|
||||
],
|
||||
"atlas_scope": "test-fixture",
|
||||
"actionable": false,
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers."
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "tests/fixtures/governance_invalid_manifest/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "broken-yaml-skill",
|
||||
@@ -11413,7 +11509,16 @@
|
||||
],
|
||||
"atlas_scope": "test-fixture",
|
||||
"actionable": false,
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers."
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "tests/fixtures/package_invalid_yaml/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "broken-skill",
|
||||
@@ -11436,7 +11541,16 @@
|
||||
],
|
||||
"atlas_scope": "test-fixture",
|
||||
"actionable": false,
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers."
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "tests/fixtures/package_missing_interface_field/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
}
|
||||
],
|
||||
"summary": {
|
||||
@@ -11450,6 +11564,9 @@
|
||||
"actionable_stale_count": 0,
|
||||
"shared_resource_count": 0,
|
||||
"no_route_opportunity_count": 0,
|
||||
"telemetry_report_count": 1,
|
||||
"drift_signal_count": 0,
|
||||
"actionable_drift_signal_count": 0,
|
||||
"non_actionable_issue_count": 24
|
||||
}
|
||||
},
|
||||
@@ -11740,6 +11857,8 @@
|
||||
}
|
||||
],
|
||||
"actionable_owner_review_gaps": [],
|
||||
"drift_signals": [],
|
||||
"actionable_drift_signals": [],
|
||||
"no_route_opportunities": [],
|
||||
"artifacts": {
|
||||
"catalog": "skill_atlas/catalog.json",
|
||||
@@ -11747,6 +11866,7 @@
|
||||
"dependency_graph": "skill_atlas/dependency_graph.json",
|
||||
"stale_skills": "skill_atlas/stale_skills.json",
|
||||
"owner_review_gaps": "skill_atlas/owner_review_gaps.json",
|
||||
"drift_signals": "skill_atlas/drift_signals.json",
|
||||
"no_route_opportunities": "skill_atlas/no_route_opportunities.json",
|
||||
"report_json": "reports/skill_atlas.json",
|
||||
"report_html": "reports/skill_atlas.html"
|
||||
@@ -11755,7 +11875,7 @@
|
||||
"adoption_drift": {
|
||||
"ok": true,
|
||||
"schema_version": "2.0",
|
||||
"generated_at": "2026-06-13T12:10:53Z",
|
||||
"generated_at": "2026-06-13T12:24:01Z",
|
||||
"skill_dir": ".",
|
||||
"privacy_contract": {
|
||||
"storage": "local-first",
|
||||
@@ -11779,14 +11899,14 @@
|
||||
},
|
||||
"summary": {
|
||||
"event_count": 1,
|
||||
"adoption_sample_count": 1,
|
||||
"activation_count": 1,
|
||||
"accepted_count": 1,
|
||||
"adoption_sample_count": 0,
|
||||
"activation_count": 0,
|
||||
"accepted_count": 0,
|
||||
"edited_count": 0,
|
||||
"rejected_count": 0,
|
||||
"missed_count": 0,
|
||||
"failed_count": 0,
|
||||
"adoption_rate": 100.0,
|
||||
"adoption_rate": 0,
|
||||
"missed_trigger_count": 0,
|
||||
"wrong_trigger_count": 0,
|
||||
"bad_output_count": 0,
|
||||
@@ -11795,7 +11915,7 @@
|
||||
"review_overdue_count": 0,
|
||||
"risk_band": "low",
|
||||
"event_types": {
|
||||
"skill_activation": 1
|
||||
"review_event": 1
|
||||
},
|
||||
"failure_types": {}
|
||||
},
|
||||
@@ -11803,29 +11923,29 @@
|
||||
{
|
||||
"skill": "yao-meta-skill",
|
||||
"events": 1,
|
||||
"adoption_events": 1,
|
||||
"accepted": 1,
|
||||
"adoption_events": 0,
|
||||
"accepted": 0,
|
||||
"edited": 0,
|
||||
"rejected": 0,
|
||||
"missed": 0,
|
||||
"adoption_rate": 100.0
|
||||
"adoption_rate": 0
|
||||
}
|
||||
],
|
||||
"next_iteration_candidates": [],
|
||||
"recent_events": [
|
||||
{
|
||||
"event": "skill_activation",
|
||||
"event": "review_event",
|
||||
"skill": "yao-meta-skill",
|
||||
"version": "1.1.0",
|
||||
"activation_type": "explicit",
|
||||
"outcome": "accepted",
|
||||
"activation_type": "manual",
|
||||
"outcome": "reviewed",
|
||||
"failure_type": "none",
|
||||
"timestamp": "2026-06-13T10:00:00Z"
|
||||
"timestamp": "2026-06-13T12:00:00Z"
|
||||
}
|
||||
],
|
||||
"failures": [],
|
||||
"artifacts": {
|
||||
"events_jsonl": "tests/tmp_review_studio/telemetry_events.jsonl",
|
||||
"events_jsonl": "reports/telemetry_events.jsonl",
|
||||
"json": "reports/adoption_drift_report.json",
|
||||
"markdown": "reports/adoption_drift_report.md"
|
||||
}
|
||||
@@ -11874,7 +11994,7 @@
|
||||
"schema_version": "1.0",
|
||||
"ok": true,
|
||||
"skill_dir": ".",
|
||||
"source": "tests/tmp_review_studio/empty_review_annotations_input.json",
|
||||
"source": "reports/review_annotations_input.json",
|
||||
"summary": {
|
||||
"annotation_count": 0,
|
||||
"open_count": 0,
|
||||
@@ -11917,8 +12037,8 @@
|
||||
"trust_level": "local",
|
||||
"license": "MIT",
|
||||
"checksums": {
|
||||
"package_sha256": "2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd",
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb",
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20"
|
||||
},
|
||||
"compatibility": {
|
||||
"openai": "pass",
|
||||
@@ -11949,7 +12069,7 @@
|
||||
},
|
||||
"distribution": {
|
||||
"archive_verified": true,
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060",
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20",
|
||||
"package_verification": "reports/package_verification.json",
|
||||
"install_simulated": true,
|
||||
"install_simulation": "reports/install_simulation.json"
|
||||
@@ -11974,7 +12094,7 @@
|
||||
"vscode"
|
||||
],
|
||||
"package_metadata": "registry/packages/yao-meta-skill.json",
|
||||
"package_sha256": "2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -11997,8 +12117,8 @@
|
||||
"target_count": 4,
|
||||
"adapter_count": 4,
|
||||
"archive_present": true,
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060",
|
||||
"archive_entry_count": 493,
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20",
|
||||
"archive_entry_count": 494,
|
||||
"failure_count": 0,
|
||||
"warning_count": 0
|
||||
},
|
||||
@@ -12663,7 +12783,7 @@
|
||||
"installed_skill_dir": "dist/install-simulation/simulate-yao-meta-skill/yao-meta-skill",
|
||||
"summary": {
|
||||
"archive_present": true,
|
||||
"archive_entry_count": 493,
|
||||
"archive_entry_count": 494,
|
||||
"archive_extracted": true,
|
||||
"entrypoint_loaded": true,
|
||||
"manifest_loaded": true,
|
||||
@@ -12856,12 +12976,12 @@
|
||||
{
|
||||
"field": "archive_sha256",
|
||||
"from": "",
|
||||
"to": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060"
|
||||
"to": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20"
|
||||
},
|
||||
{
|
||||
"field": "package_sha256",
|
||||
"from": "0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"to": "2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd"
|
||||
"to": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -23,7 +23,7 @@
|
||||
"interactive_script_count": 0,
|
||||
"package_hash_scope": "source-contract-without-generated-reports",
|
||||
"package_hash_file_count": 153,
|
||||
"package_sha256": "7a8dab0d542d2994adec2045b54559612db36f14db8b7b66e4138fb0435ae3d3"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
},
|
||||
"failures": [],
|
||||
"warnings": [],
|
||||
|
||||
@@ -16,7 +16,7 @@
|
||||
- Interactive scripts: `0`
|
||||
- Package hash scope: `source-contract-without-generated-reports`
|
||||
- Package hash files: `153`
|
||||
- Package SHA256: `7a8dab0d542d2994adec2045b54559612db36f14db8b7b66e4138fb0435ae3d3`
|
||||
- Package SHA256: `ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb`
|
||||
|
||||
## Failures
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ Yao Meta Skill is no longer only a Meta Skill factory. The current working tree
|
||||
- Output Eval Lab v0 for with-skill vs baseline assertion grading plus local command-runner execution evidence, blind A/B review pack, separate answer key evidence, and reviewer adjudication reporting.
|
||||
- Runtime Conformance v0 for target-consumption checks.
|
||||
- Trust/Security v0 for secret, dependency, script, bounded network host policy, execution-level CLI help smoke checks, trust metadata, and stable source-contract integrity checks.
|
||||
- Skill Atlas v0 for portfolio catalog, route overlap, stale ownership, dependency signals, and no-route opportunities.
|
||||
- Skill Atlas v0 for portfolio catalog, route overlap, stale ownership, dependency signals, aggregate drift signals, and no-route opportunities.
|
||||
- Bilingual Skill Overview v2 that includes these evidence surfaces.
|
||||
- Review Studio 2.0 v0 for one-page blocker, warning, evidence-path, review-action, and release-gate review.
|
||||
- Review Studio Source Refs v0 so every non-pass review action can expose structured relative source/report links with best-effort line numbers.
|
||||
@@ -21,7 +21,7 @@ Yao Meta Skill is no longer only a Meta Skill factory. The current working tree
|
||||
- Package Verification v0 for generated manifest, target adapters, zip archive safety, archive checksum, and registry parity.
|
||||
- Install Simulation v0 for local extraction, entrypoint loading, interface loading, report presence, and adapter readability.
|
||||
- Upgrade Check v0 for registry package diffs, semver bump recommendations, breaking-change notes, and release evidence.
|
||||
- Adoption Drift v0 for local-first metadata-only telemetry, privacy-field blocking, usage-signal aggregation, and iteration candidate generation.
|
||||
- Adoption Drift v0 for local-first metadata-only telemetry, privacy-field blocking, usage-signal aggregation, iteration candidate generation, and Skill Atlas drift-signal input.
|
||||
- Review Waivers v0 for human warning acceptance with reviewer, reason, scope, expiry, and blocker-safe policy.
|
||||
- Governed Permission Gates v0 for reviewer-approved network, file-write, and subprocess capabilities with scope, reason, expiry, evidence, and target-enforcement mapping.
|
||||
- Runtime Permission Probes v0 for packaged target adapter checks, explicit native-enforcement flags, metadata fallback evidence, and residual permission risks.
|
||||
@@ -31,7 +31,7 @@ Yao Meta Skill is no longer only a Meta Skill factory. The current working tree
|
||||
- Review Studio Output Lab actions now link directly to `reports/output_review_decisions.json`, so pending blind A/B reviewer decisions have a concrete template instead of only a general adjudication warning.
|
||||
- Provider Output Eval Runner v0 so `python3 scripts/yao.py output-exec --provider-runner openai` can collect real provider-backed model evidence through a reviewed OpenAI Responses API compatible runner instead of ad hoc shell glue.
|
||||
|
||||
This is still not the final world-class state. Target-native behavior contracts are now explicit, VS Code / Copilot package metadata is auditable, local output-eval command execution is wired, and a provider-backed output runner exists. Review Studio keeps pending human adjudication visible as a warning instead of treating it as a clean pass. Deeper provider-native execution transforms, real client telemetry capture, installer integration, real provider holdout runs, real human adjudication decisions, and native runtime permission enforcement remain open.
|
||||
This is still not the final world-class state. Target-native behavior contracts are now explicit, VS Code / Copilot package metadata is auditable, local output-eval command execution is wired, a provider-backed output runner exists, and Skill Atlas now consumes aggregate drift reports. Review Studio keeps pending human adjudication visible as a warning instead of treating it as a clean pass. Deeper provider-native execution transforms, real client telemetry capture, installer integration, real provider holdout runs, real human adjudication decisions, and native runtime permission enforcement remain open.
|
||||
|
||||
## Coverage Matrix
|
||||
|
||||
@@ -46,7 +46,7 @@ This is still not the final world-class state. Target-native behavior contracts
|
||||
| Review Studio 2.0 | `scripts/render_review_studio.py`, `reports/review-studio.html`, `reports/review-studio.json` with per-warning `review_actions` | v0 landed |
|
||||
| Review Studio Source Refs | `scripts/render_review_studio.py`, `tests/verify_review_studio.py`, `references/review-studio-method.md` | v0 landed |
|
||||
| Review Annotations | `scripts/render_review_annotations.py`, `reports/review_annotations.md`, `tests/verify_review_annotations.py`, Review Studio annotation panel and blocker decision hook | v0 landed |
|
||||
| Skill Atlas | `scripts/build_skill_atlas.py`, `skill_atlas/catalog.json`, `skill_atlas/route_overlap_matrix.csv`, `reports/skill_atlas.html` | v0 landed |
|
||||
| Skill Atlas | `scripts/build_skill_atlas.py`, `skill_atlas/catalog.json`, `skill_atlas/route_overlap_matrix.csv`, `skill_atlas/drift_signals.json`, `reports/skill_atlas.html` | v0 landed |
|
||||
| Registry & Distribution | `registry/*.schema.json`, `scripts/registry_audit.py`, `reports/registry_audit.md`, `registry/packages/yao-meta-skill.json` | v0 landed |
|
||||
| Package Verification | `scripts/verify_package.py`, `reports/package_verification.md`, `tests/verify_package_verification.py` | v0 landed |
|
||||
| Install Simulation | `scripts/simulate_install.py`, `reports/install_simulation.md`, `tests/verify_install_simulation.py` | v0 landed |
|
||||
@@ -79,11 +79,11 @@ The Review Studio page aggregates intent, trigger, output, context, runtime, tru
|
||||
|
||||
Next move: add richer source-line anchors inside generated reports and record real reviewer annotations during the next human review pass.
|
||||
|
||||
### 4. Multi-skill operation now has v0 coverage, but needs real telemetry
|
||||
### 4. Multi-skill operation now links Atlas with drift, but needs real client capture
|
||||
|
||||
The new Skill Atlas can scan a workspace and report catalog, route overlap, dependency graph, stale skill, missing owner/review metadata, and no-route opportunities. It now also supports `skill_atlas/policy.json` so release gates distinguish actionable library skills from examples and fixtures while keeping full visibility. Adoption Drift v0 can record metadata-only local events, block raw prompt/output fields, summarize missed-trigger, bad-output, script-error, and review-overdue signals, and feed next iteration candidates into Review Studio. It still needs real client-side capture instead of manual CLI event recording.
|
||||
The new Skill Atlas can scan a workspace and report catalog, route overlap, dependency graph, stale skill, missing owner/review metadata, aggregate drift signals, and no-route opportunities. It now also supports `skill_atlas/policy.json` so release gates distinguish actionable library skills from examples and fixtures while keeping full visibility. Adoption Drift v0 can record metadata-only local events, block raw prompt/output fields, summarize missed-trigger, bad-output, script-error, and review-overdue signals, feed next iteration candidates into Review Studio, and publish aggregate drift input for Atlas. It still needs real client-side capture instead of manual CLI event recording.
|
||||
|
||||
Next move: connect client activations and failure history so Atlas can rank stale or conflicting skills by real usage impact.
|
||||
Next move: connect client activations and failure history so Atlas can rank stale, drifting, or conflicting skills by real usage impact.
|
||||
|
||||
### 5. Trust report is structural, not full security review
|
||||
|
||||
@@ -101,12 +101,12 @@ Next move: add real client or installer permission enforcement integration.
|
||||
| Trust | `0` secret findings, `1` pinned dependency file, `9` declared internal modules, `3 / 3` network-capable scripts covered by bounded host policy, `60 / 60` CLI help smoke checks passing across `69` scripts, source-contract hash scope explicit |
|
||||
| Permission Governance | `3 / 3` required high-permission capabilities approved, `0` missing, `0` invalid, `0` expired |
|
||||
| Runtime Permission Probes | `4 / 4` target adapters probed, `0` native-enforcement adapters, `4` explicit metadata fallbacks, `4` residual risks retained for reviewer visibility |
|
||||
| Skill Atlas | `12` scanned skills, `1` actionable root skill, `0` actionable route collisions, `0` actionable owner gaps, `0` actionable stale skills, `24` scoped non-actionable issue signals retained for visibility |
|
||||
| Skill Atlas | `12` scanned skills, `1` actionable root skill, `1` telemetry report, `0` actionable route collisions, `0` actionable owner gaps, `0` actionable stale skills, `0` actionable drift signals, `24` scoped non-actionable issue signals retained for visibility |
|
||||
| Registry Audit | package metadata generated with version, owner, license, source checksum, archive checksum, Skill IR provenance, and compatibility matrix |
|
||||
| Package Verification | `4 / 4` target adapters present, archive verified, `493` zip entries, `0` failures, `0` warnings |
|
||||
| Install Simulation | archive with `493` entries extracted into a local verification root, entrypoint/manifest/interface loaded, reports present, `4` adapters readable, `0` failures, `0` warnings |
|
||||
| Package Verification | `4 / 4` target adapters present, archive verified, `494` zip entries, `0` failures, `0` warnings |
|
||||
| Install Simulation | archive with `494` entries extracted into a local verification root, entrypoint/manifest/interface loaded, reports present, `4` adapters readable, `0` failures, `0` warnings |
|
||||
| Upgrade Check | current package declares `minor` over the 1.0.0 baseline, recommended bump is `minor`, and release notes include added targets plus checksum changes |
|
||||
| Adoption Drift | `1` metadata-only activation event, `1` adoption sample, adoption `100.0`, risk band `low`; raw `reports/telemetry_events.jsonl` is gitignored and blocked from zip packages |
|
||||
| Adoption Drift | `1` metadata-only review event, `0` adoption samples, adoption `0`, risk band `low`; raw `reports/telemetry_events.jsonl` is gitignored and blocked from zip packages |
|
||||
| Review Waivers | ledger generated; current release has `1` warning gate that still needs reviewer decision or a time-bounded waiver; blockers remain non-waivable in v0 |
|
||||
| Review Annotations | ledger generated; current release has `0` reviewer annotations and `0` open annotation blockers |
|
||||
| Review Studio | decision `review`, world-class score `92`, `13` gates, `0` blockers, `2` warnings, `2` review actions, `0` open annotation blockers |
|
||||
@@ -120,4 +120,4 @@ Next move: add real client or installer permission enforcement integration.
|
||||
2. Add native client or installer enforcement for approved high-permission capabilities.
|
||||
3. Run the new provider-backed output runner against holdout cases with real credentials, then record the current blind A/B decisions before claiming fully ready status.
|
||||
4. Add real reviewer annotation records during the next human review pass.
|
||||
5. Connect Skill Atlas with real client telemetry and drift history.
|
||||
5. Connect real client telemetry capture and failure history so Atlas drift signals are based on live usage, not manual samples.
|
||||
|
||||
@@ -609,7 +609,7 @@
|
||||
<ul class="list"><li><span data-lang="zh-CN">把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。</span><span data-lang="en">把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。</span></li><li><span data-lang="zh-CN">Skill 作者、复用团队和后续 reviewer。</span><span data-lang="en">Skill 作者、复用团队和后续 reviewer。</span></li><li><span data-lang="zh-CN">创建完成后建议先打开 reports/skill-overview.html，再继续扩展包体。</span><span data-lang="en">创建完成后建议先打开 reports/skill-overview.html，再继续扩展包体。</span></li></ul>
|
||||
</aside>
|
||||
</div>
|
||||
<div class="score-strip" aria-label="报告关键指标"><article class='score-chip'><span>完整度</span><strong>100</strong><i style='--score:100%'></i><small><span data-lang="zh-CN">SKILL.md 已存在，是 Skill 的入口。</span><span data-lang="en">SKILL.md 已存在，是 Skill 的入口。</span></small></article><article class='score-chip'><span>触发清晰</span><strong>100</strong><i style='--score:100%'></i><small><span data-lang="zh-CN">frontmatter description 已存在，具备基础路由面。</span><span data-lang="en">frontmatter description 已存在，具备基础路由面。</span></small></article><article class='score-chip'><span>证据充分</span><strong>100</strong><i style='--score:100%'></i><small><span data-lang="zh-CN">已生成 20 / 20 类报告证据。</span><span data-lang="en">已生成 20 / 20 类报告证据。</span></small></article><article class='score-chip'><span>上下文成本</span><strong>42</strong><i style='--score:42%'></i><small><span data-lang="zh-CN">入口约 350 个词/字，references 约 14450 个词/字。</span><span data-lang="en">入口约 350 个词/字，references 约 14450 个词/字。</span></small></article></div>
|
||||
<div class="score-strip" aria-label="报告关键指标"><article class='score-chip'><span>完整度</span><strong>100</strong><i style='--score:100%'></i><small><span data-lang="zh-CN">SKILL.md 已存在，是 Skill 的入口。</span><span data-lang="en">SKILL.md 已存在，是 Skill 的入口。</span></small></article><article class='score-chip'><span>触发清晰</span><strong>100</strong><i style='--score:100%'></i><small><span data-lang="zh-CN">frontmatter description 已存在，具备基础路由面。</span><span data-lang="en">frontmatter description 已存在，具备基础路由面。</span></small></article><article class='score-chip'><span>证据充分</span><strong>100</strong><i style='--score:100%'></i><small><span data-lang="zh-CN">已生成 20 / 20 类报告证据。</span><span data-lang="en">已生成 20 / 20 类报告证据。</span></small></article><article class='score-chip'><span>上下文成本</span><strong>42</strong><i style='--score:42%'></i><small><span data-lang="zh-CN">入口约 350 个词/字，references 约 14610 个词/字。</span><span data-lang="en">入口约 350 个词/字，references 约 14610 个词/字。</span></small></article></div>
|
||||
</section>
|
||||
|
||||
<section id="overview">
|
||||
@@ -641,10 +641,10 @@
|
||||
<article class="panel metrics-note">
|
||||
<h3><span data-lang="zh-CN">指标判读</span><span data-lang="en">Reading</span></h3>
|
||||
<p><span data-lang="zh-CN">先看雷达图判断能力短板，再看下方每项分数的证据原因。分数不是装饰数字，必须和本地文件、reports 证据或证据不足提示对应。</span><span data-lang="en">Read the radar first for weak spots, then inspect each score with its evidence. Scores must map to local files, reports, or explicit evidence gaps.</span></p>
|
||||
<ol class='metric-summary-list'><li><span class='metric-status'><span data-lang="zh-CN">稳定</span><span data-lang="en">Stable</span></span><b>完整度</b><em>100</em><small><span data-lang="zh-CN">SKILL.md 已存在，是 Skill 的入口。</span><span data-lang="en">SKILL.md 已存在，是 Skill 的入口。</span></small></li><li><span class='metric-status'><span data-lang="zh-CN">稳定</span><span data-lang="en">Stable</span></span><b>触发清晰</b><em>100</em><small><span data-lang="zh-CN">frontmatter description 已存在，具备基础路由面。</span><span data-lang="en">frontmatter description 已存在，具备基础路由面。</span></small></li><li><span class='metric-status'><span data-lang="zh-CN">稳定</span><span data-lang="en">Stable</span></span><b>证据充分</b><em>100</em><small><span data-lang="zh-CN">已生成 20 / 20 类报告证据。</span><span data-lang="en">已生成 20 / 20 类报告证据。</span></small></li><li><span class='metric-status'><span data-lang="zh-CN">稳定</span><span data-lang="en">Stable</span></span><b>可维护性</b><em>100</em><small><span data-lang="zh-CN">SKILL.md 约 350 个词/字。</span><span data-lang="en">SKILL.md 约 350 个词/字。</span></small></li><li><span class='metric-status'><span data-lang="zh-CN">稳定</span><span data-lang="en">Stable</span></span><b>可迁移性</b><em>100</em><small><span data-lang="zh-CN">agents/interface.yaml 已存在。</span><span data-lang="en">agents/interface.yaml 已存在。</span></small></li><li><span class='metric-status'><span data-lang="zh-CN">关注</span><span data-lang="en">Watch</span></span><b>上下文成本</b><em>42</em><small><span data-lang="zh-CN">入口约 350 个词/字，references 约 14450 个词/字。</span><span data-lang="en">入口约 350 个词/字，references 约 14450 个词/字。</span></small></li></ol>
|
||||
<ol class='metric-summary-list'><li><span class='metric-status'><span data-lang="zh-CN">稳定</span><span data-lang="en">Stable</span></span><b>完整度</b><em>100</em><small><span data-lang="zh-CN">SKILL.md 已存在，是 Skill 的入口。</span><span data-lang="en">SKILL.md 已存在，是 Skill 的入口。</span></small></li><li><span class='metric-status'><span data-lang="zh-CN">稳定</span><span data-lang="en">Stable</span></span><b>触发清晰</b><em>100</em><small><span data-lang="zh-CN">frontmatter description 已存在，具备基础路由面。</span><span data-lang="en">frontmatter description 已存在，具备基础路由面。</span></small></li><li><span class='metric-status'><span data-lang="zh-CN">稳定</span><span data-lang="en">Stable</span></span><b>证据充分</b><em>100</em><small><span data-lang="zh-CN">已生成 20 / 20 类报告证据。</span><span data-lang="en">已生成 20 / 20 类报告证据。</span></small></li><li><span class='metric-status'><span data-lang="zh-CN">稳定</span><span data-lang="en">Stable</span></span><b>可维护性</b><em>100</em><small><span data-lang="zh-CN">SKILL.md 约 350 个词/字。</span><span data-lang="en">SKILL.md 约 350 个词/字。</span></small></li><li><span class='metric-status'><span data-lang="zh-CN">稳定</span><span data-lang="en">Stable</span></span><b>可迁移性</b><em>100</em><small><span data-lang="zh-CN">agents/interface.yaml 已存在。</span><span data-lang="en">agents/interface.yaml 已存在。</span></small></li><li><span class='metric-status'><span data-lang="zh-CN">关注</span><span data-lang="en">Watch</span></span><b>上下文成本</b><em>42</em><small><span data-lang="zh-CN">入口约 350 个词/字，references 约 14610 个词/字。</span><span data-lang="en">入口约 350 个词/字，references 约 14610 个词/字。</span></small></li></ol>
|
||||
</article>
|
||||
</div>
|
||||
<div class="metric-grid"><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>完整度</span><strong>100</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">SKILL.md 已存在，是 Skill 的入口。</span><span data-lang="en">SKILL.md 已存在，是 Skill 的入口。</span></li><li><span data-lang="zh-CN">README.md 已存在，便于人工阅读。</span><span data-lang="en">README.md 已存在，便于人工阅读。</span></li><li><span data-lang="zh-CN">agents/interface.yaml 已存在，便于跨平台适配。</span><span data-lang="en">agents/interface.yaml 已存在，便于跨平台适配。</span></li></ul></div></article><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>触发清晰</span><strong>100</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">frontmatter description 已存在，具备基础路由面。</span><span data-lang="en">frontmatter description 已存在，具备基础路由面。</span></li><li><span data-lang="zh-CN">description 有足够长度说明任务边界。</span><span data-lang="en">description 有足够长度说明任务边界。</span></li><li><span data-lang="zh-CN">description 已包含使用场景或排除边界信号。</span><span data-lang="en">description 已包含使用场景或排除边界信号。</span></li></ul></div></article><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>证据充分</span><strong>100</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">已生成 20 / 20 类报告证据。</span><span data-lang="en">已生成 20 / 20 类报告证据。</span></li><li><span data-lang="zh-CN">skill-ir.json 已存在。</span><span data-lang="en">skill-ir.json 已存在。</span></li><li><span data-lang="zh-CN">compiled_targets.json 已存在。</span><span data-lang="en">compiled_targets.json 已存在。</span></li></ul></div></article><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>可维护性</span><strong>100</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">SKILL.md 约 350 个词/字。</span><span data-lang="en">SKILL.md 约 350 个词/字。</span></li><li><span data-lang="zh-CN">入口文件保持克制，可维护性较好。</span><span data-lang="en">入口文件保持克制，可维护性较好。</span></li><li><span data-lang="zh-CN">references/ 已承载扩展指导。</span><span data-lang="en">references/ 已承载扩展指导。</span></li></ul></div></article><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>可迁移性</span><strong>100</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">agents/interface.yaml 已存在。</span><span data-lang="en">agents/interface.yaml 已存在。</span></li><li><span data-lang="zh-CN">manifest.json 已存在。</span><span data-lang="en">manifest.json 已存在。</span></li><li><span data-lang="zh-CN">目标平台或 adapter target 已声明。</span><span data-lang="en">目标平台或 adapter target 已声明。</span></li></ul></div></article><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>上下文成本</span><strong>42</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">入口约 350 个词/字，references 约 14450 个词/字。</span><span data-lang="en">入口约 350 个词/字，references 约 14450 个词/字。</span></li><li><span data-lang="zh-CN">分数越高代表上下文成本越低。</span><span data-lang="en">分数越高代表上下文成本越低。</span></li><li><span data-lang="zh-CN">上下文成本偏高，建议压缩入口或拆分 references。</span><span data-lang="en">上下文成本偏高，建议压缩入口或拆分 references。</span></li></ul></div></article></div>
|
||||
<div class="metric-grid"><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>完整度</span><strong>100</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">SKILL.md 已存在，是 Skill 的入口。</span><span data-lang="en">SKILL.md 已存在，是 Skill 的入口。</span></li><li><span data-lang="zh-CN">README.md 已存在，便于人工阅读。</span><span data-lang="en">README.md 已存在，便于人工阅读。</span></li><li><span data-lang="zh-CN">agents/interface.yaml 已存在，便于跨平台适配。</span><span data-lang="en">agents/interface.yaml 已存在，便于跨平台适配。</span></li></ul></div></article><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>触发清晰</span><strong>100</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">frontmatter description 已存在，具备基础路由面。</span><span data-lang="en">frontmatter description 已存在，具备基础路由面。</span></li><li><span data-lang="zh-CN">description 有足够长度说明任务边界。</span><span data-lang="en">description 有足够长度说明任务边界。</span></li><li><span data-lang="zh-CN">description 已包含使用场景或排除边界信号。</span><span data-lang="en">description 已包含使用场景或排除边界信号。</span></li></ul></div></article><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>证据充分</span><strong>100</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">已生成 20 / 20 类报告证据。</span><span data-lang="en">已生成 20 / 20 类报告证据。</span></li><li><span data-lang="zh-CN">skill-ir.json 已存在。</span><span data-lang="en">skill-ir.json 已存在。</span></li><li><span data-lang="zh-CN">compiled_targets.json 已存在。</span><span data-lang="en">compiled_targets.json 已存在。</span></li></ul></div></article><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>可维护性</span><strong>100</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">SKILL.md 约 350 个词/字。</span><span data-lang="en">SKILL.md 约 350 个词/字。</span></li><li><span data-lang="zh-CN">入口文件保持克制，可维护性较好。</span><span data-lang="en">入口文件保持克制，可维护性较好。</span></li><li><span data-lang="zh-CN">references/ 已承载扩展指导。</span><span data-lang="en">references/ 已承载扩展指导。</span></li></ul></div></article><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>可迁移性</span><strong>100</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">agents/interface.yaml 已存在。</span><span data-lang="en">agents/interface.yaml 已存在。</span></li><li><span data-lang="zh-CN">manifest.json 已存在。</span><span data-lang="en">manifest.json 已存在。</span></li><li><span data-lang="zh-CN">目标平台或 adapter target 已声明。</span><span data-lang="en">目标平台或 adapter target 已声明。</span></li></ul></div></article><article class='metric-card'><div class='metric-card-head'><span class='metric-label'>上下文成本</span><strong>42</strong></div><div class='metric-card-body'><ul class="compact-list"><li><span data-lang="zh-CN">入口约 350 个词/字，references 约 14610 个词/字。</span><span data-lang="en">入口约 350 个词/字，references 约 14610 个词/字。</span></li><li><span data-lang="zh-CN">分数越高代表上下文成本越低。</span><span data-lang="en">分数越高代表上下文成本越低。</span></li><li><span data-lang="zh-CN">上下文成本偏高，建议压缩入口或拆分 references。</span><span data-lang="en">上下文成本偏高，建议压缩入口或拆分 references。</span></li></ul></div></article></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -720,7 +720,7 @@
|
||||
<div>
|
||||
<table>
|
||||
<thead><tr><th><span data-lang="zh-CN">类型</span><span data-lang="en">Type</span></th><th><span data-lang="zh-CN">证据</span><span data-lang="en">Evidence</span></th><th><span data-lang="zh-CN">建议</span><span data-lang="en">Action</span></th></tr></thead>
|
||||
<tbody><tr><td><span data-lang="zh-CN">强项</span><span data-lang="en">Strength</span></td><td><span data-lang="zh-CN">触发面保持精简，并锚定在 frontmatter description。</span><span data-lang="en">The trigger surface stays lean and anchored in the frontmatter description.</span></td><td><span data-lang="zh-CN">保留并复用</span><span data-lang="en">Keep</span></td></tr><tr><td><span data-lang="zh-CN">强项</span><span data-lang="en">Strength</span></td><td><span data-lang="zh-CN">已生成 Skill IR，核心语义可先于平台打包被审查和迁移。</span><span data-lang="en">已生成 Skill IR，核心语义可先于平台打包被审查和迁移。</span></td><td><span data-lang="zh-CN">保留并复用</span><span data-lang="en">Keep</span></td></tr><tr><td><span data-lang="zh-CN">强项</span><span data-lang="en">Strength</span></td><td><span data-lang="zh-CN">已生成目标编译报告，可审查 IR 到 OpenAI、Claude、generic 等目标契约的映射。</span><span data-lang="en">已生成目标编译报告，可审查 IR 到 OpenAI、Claude、generic 等目标契约的映射。</span></td><td><span data-lang="zh-CN">保留并复用</span><span data-lang="en">Keep</span></td></tr><tr><td><span data-lang="zh-CN">缺口</span><span data-lang="en">Gap</span></td><td><span data-lang="zh-CN">上下文成本需要补强：入口约 350 个词/字，references 约 14450 个词/字。</span><span data-lang="en">上下文成本需要补强：入口约 350 个词/字，references 约 14450 个词/字。</span></td><td><span data-lang="zh-CN">纳入下一轮修复</span><span data-lang="en">Fix next</span></td></tr></tbody>
|
||||
<tbody><tr><td><span data-lang="zh-CN">强项</span><span data-lang="en">Strength</span></td><td><span data-lang="zh-CN">触发面保持精简，并锚定在 frontmatter description。</span><span data-lang="en">The trigger surface stays lean and anchored in the frontmatter description.</span></td><td><span data-lang="zh-CN">保留并复用</span><span data-lang="en">Keep</span></td></tr><tr><td><span data-lang="zh-CN">强项</span><span data-lang="en">Strength</span></td><td><span data-lang="zh-CN">已生成 Skill IR，核心语义可先于平台打包被审查和迁移。</span><span data-lang="en">已生成 Skill IR，核心语义可先于平台打包被审查和迁移。</span></td><td><span data-lang="zh-CN">保留并复用</span><span data-lang="en">Keep</span></td></tr><tr><td><span data-lang="zh-CN">强项</span><span data-lang="en">Strength</span></td><td><span data-lang="zh-CN">已生成目标编译报告，可审查 IR 到 OpenAI、Claude、generic 等目标契约的映射。</span><span data-lang="en">已生成目标编译报告，可审查 IR 到 OpenAI、Claude、generic 等目标契约的映射。</span></td><td><span data-lang="zh-CN">保留并复用</span><span data-lang="en">Keep</span></td></tr><tr><td><span data-lang="zh-CN">缺口</span><span data-lang="en">Gap</span></td><td><span data-lang="zh-CN">上下文成本需要补强：入口约 350 个词/字，references 约 14610 个词/字。</span><span data-lang="en">上下文成本需要补强：入口约 350 个词/字，references 约 14610 个词/字。</span></td><td><span data-lang="zh-CN">纳入下一轮修复</span><span data-lang="en">Fix next</span></td></tr></tbody>
|
||||
</table>
|
||||
<div class="two-col quality-panels">
|
||||
<article class="panel">
|
||||
|
||||
+19
-16
@@ -96,7 +96,7 @@
|
||||
"label": "上下文成本",
|
||||
"score": 42,
|
||||
"reasons": [
|
||||
"入口约 350 个词/字，references 约 14450 个词/字。",
|
||||
"入口约 350 个词/字，references 约 14610 个词/字。",
|
||||
"分数越高代表上下文成本越低。",
|
||||
"上下文成本偏高，建议压缩入口或拆分 references。"
|
||||
]
|
||||
@@ -200,7 +200,7 @@
|
||||
"已生成 Output Review Adjudication，可记录盲评决策、一致率和待评审项。"
|
||||
],
|
||||
"gaps": [
|
||||
"上下文成本需要补强：入口约 350 个词/字，references 约 14450 个词/字。"
|
||||
"上下文成本需要补强：入口约 350 个词/字，references 约 14610 个词/字。"
|
||||
],
|
||||
"recommendations": [
|
||||
"先改触发边界，再扩展工作流。",
|
||||
@@ -967,7 +967,7 @@
|
||||
"interactive_script_count": 0,
|
||||
"package_hash_scope": "source-contract-without-generated-reports",
|
||||
"package_hash_file_count": 153,
|
||||
"package_sha256": "2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
},
|
||||
"skill_atlas": {
|
||||
"skill_count": 12,
|
||||
@@ -980,6 +980,9 @@
|
||||
"actionable_stale_count": 0,
|
||||
"shared_resource_count": 0,
|
||||
"no_route_opportunity_count": 0,
|
||||
"telemetry_report_count": 1,
|
||||
"drift_signal_count": 0,
|
||||
"actionable_drift_signal_count": 0,
|
||||
"non_actionable_issue_count": 24
|
||||
},
|
||||
"registry_distribution": {
|
||||
@@ -1002,8 +1005,8 @@
|
||||
"trust_level": "local",
|
||||
"license": "MIT",
|
||||
"checksums": {
|
||||
"package_sha256": "2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd",
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060"
|
||||
"package_sha256": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb",
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20"
|
||||
},
|
||||
"compatibility": {
|
||||
"openai": "pass",
|
||||
@@ -1034,7 +1037,7 @@
|
||||
},
|
||||
"distribution": {
|
||||
"archive_verified": true,
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060",
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20",
|
||||
"package_verification": "reports/package_verification.json",
|
||||
"install_simulated": true,
|
||||
"install_simulation": "reports/install_simulation.json"
|
||||
@@ -1050,8 +1053,8 @@
|
||||
"target_count": 4,
|
||||
"adapter_count": 4,
|
||||
"archive_present": true,
|
||||
"archive_sha256": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060",
|
||||
"archive_entry_count": 493,
|
||||
"archive_sha256": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20",
|
||||
"archive_entry_count": 494,
|
||||
"failure_count": 0,
|
||||
"warning_count": 0
|
||||
},
|
||||
@@ -1062,7 +1065,7 @@
|
||||
"ok": true,
|
||||
"summary": {
|
||||
"archive_present": true,
|
||||
"archive_entry_count": 493,
|
||||
"archive_entry_count": 494,
|
||||
"archive_extracted": true,
|
||||
"entrypoint_loaded": true,
|
||||
"manifest_loaded": true,
|
||||
@@ -1125,12 +1128,12 @@
|
||||
{
|
||||
"field": "archive_sha256",
|
||||
"from": "",
|
||||
"to": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060"
|
||||
"to": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20"
|
||||
},
|
||||
{
|
||||
"field": "package_sha256",
|
||||
"from": "0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"to": "2109ae8e212111b1619d8d728e54c73425fecc174113b45629c656ecceb3fadd"
|
||||
"to": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
}
|
||||
]
|
||||
},
|
||||
@@ -1147,14 +1150,14 @@
|
||||
"ok": true,
|
||||
"summary": {
|
||||
"event_count": 1,
|
||||
"adoption_sample_count": 1,
|
||||
"activation_count": 1,
|
||||
"accepted_count": 1,
|
||||
"adoption_sample_count": 0,
|
||||
"activation_count": 0,
|
||||
"accepted_count": 0,
|
||||
"edited_count": 0,
|
||||
"rejected_count": 0,
|
||||
"missed_count": 0,
|
||||
"failed_count": 0,
|
||||
"adoption_rate": 100.0,
|
||||
"adoption_rate": 0,
|
||||
"missed_trigger_count": 0,
|
||||
"wrong_trigger_count": 0,
|
||||
"bad_output_count": 0,
|
||||
@@ -1163,7 +1166,7 @@
|
||||
"review_overdue_count": 0,
|
||||
"risk_band": "low",
|
||||
"event_types": {
|
||||
"skill_activation": 1
|
||||
"review_event": 1
|
||||
},
|
||||
"failure_types": {}
|
||||
},
|
||||
|
||||
@@ -22,13 +22,14 @@
|
||||
<body>
|
||||
<main>
|
||||
<h1>Skill Atlas</h1>
|
||||
<p>Portfolio-level review for route overlap, stale ownership, shared resources, and no-route opportunities.</p>
|
||||
<p>Portfolio-level review for route overlap, stale ownership, shared resources, telemetry drift, and no-route opportunities.</p>
|
||||
<section class="grid">
|
||||
<div class="card"><span>Skills</span><strong>12</strong></div>
|
||||
<div class="card"><span>Actionable</span><strong>1</strong></div>
|
||||
<div class="card"><span>Route Collisions</span><strong>0</strong></div>
|
||||
<div class="card"><span>Owner Gaps</span><strong>0</strong></div>
|
||||
<div class="card"><span>Stale Skills</span><strong>0</strong></div>
|
||||
<div class="card"><span>Drift Signals</span><strong>0</strong></div>
|
||||
<div class="card"><span>No-Route Opportunities</span><strong>0</strong></div>
|
||||
</section>
|
||||
<section>
|
||||
@@ -37,7 +38,7 @@
|
||||
</section>
|
||||
<section>
|
||||
<h2>Full Portfolio Counts</h2>
|
||||
<p>All scanned skills remain visible: 5 total route collisions, 9 total owner gaps, and 10 total stale signals.</p>
|
||||
<p>All scanned skills remain visible: 5 total route collisions, 9 total owner gaps, 10 total stale signals, and 0 telemetry drift signals.</p>
|
||||
</section>
|
||||
<section>
|
||||
<h2>Catalog</h2>
|
||||
|
||||
+129
-12
@@ -12,6 +12,9 @@
|
||||
"actionable_stale_count": 0,
|
||||
"shared_resource_count": 0,
|
||||
"no_route_opportunity_count": 0,
|
||||
"telemetry_report_count": 1,
|
||||
"drift_signal_count": 0,
|
||||
"actionable_drift_signal_count": 0,
|
||||
"non_actionable_issue_count": 24
|
||||
},
|
||||
"scope_policy": {
|
||||
@@ -187,7 +190,16 @@
|
||||
],
|
||||
"atlas_scope": "release",
|
||||
"actionable": true,
|
||||
"scope_reason": "default release-actionable skill"
|
||||
"scope_reason": "default release-actionable skill",
|
||||
"telemetry": {
|
||||
"report_present": true,
|
||||
"report": "reports/adoption_drift_report.json",
|
||||
"risk_band": "low",
|
||||
"event_count": 1,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "release-orchestrator",
|
||||
@@ -245,7 +257,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/complex-release-orchestrator/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -284,7 +305,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/evolution-frontend-review/final/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -307,7 +337,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/evolution-frontend-review/v0/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -345,7 +384,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/evolution-frontend-review/v1/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "incident-command-governor",
|
||||
@@ -386,7 +434,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/governed-incident-command/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "note-cleanup",
|
||||
@@ -423,7 +480,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/simple-note-cleanup/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -463,7 +529,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/team-frontend-review/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "geo-ranking-article-generator",
|
||||
@@ -523,7 +598,16 @@
|
||||
],
|
||||
"atlas_scope": "embedded-generated-skill",
|
||||
"actionable": false,
|
||||
"scope_reason": "This nested generated skill is retained as local evidence and should not gate the root Meta Skill release."
|
||||
"scope_reason": "This nested generated skill is retained as local evidence and should not gate the root Meta Skill release.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "geo-ranking-article-generator/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "invalid-governance-skill",
|
||||
@@ -545,7 +629,16 @@
|
||||
],
|
||||
"atlas_scope": "test-fixture",
|
||||
"actionable": false,
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers."
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "tests/fixtures/governance_invalid_manifest/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "broken-yaml-skill",
|
||||
@@ -569,7 +662,16 @@
|
||||
],
|
||||
"atlas_scope": "test-fixture",
|
||||
"actionable": false,
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers."
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "tests/fixtures/package_invalid_yaml/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "broken-skill",
|
||||
@@ -592,7 +694,16 @@
|
||||
],
|
||||
"atlas_scope": "test-fixture",
|
||||
"actionable": false,
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers."
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "tests/fixtures/package_missing_interface_field/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
}
|
||||
],
|
||||
"summary": {
|
||||
@@ -606,6 +717,9 @@
|
||||
"actionable_stale_count": 0,
|
||||
"shared_resource_count": 0,
|
||||
"no_route_opportunity_count": 0,
|
||||
"telemetry_report_count": 1,
|
||||
"drift_signal_count": 0,
|
||||
"actionable_drift_signal_count": 0,
|
||||
"non_actionable_issue_count": 24
|
||||
}
|
||||
},
|
||||
@@ -896,6 +1010,8 @@
|
||||
}
|
||||
],
|
||||
"actionable_owner_review_gaps": [],
|
||||
"drift_signals": [],
|
||||
"actionable_drift_signals": [],
|
||||
"no_route_opportunities": [],
|
||||
"artifacts": {
|
||||
"catalog": "skill_atlas/catalog.json",
|
||||
@@ -903,6 +1019,7 @@
|
||||
"dependency_graph": "skill_atlas/dependency_graph.json",
|
||||
"stale_skills": "skill_atlas/stale_skills.json",
|
||||
"owner_review_gaps": "skill_atlas/owner_review_gaps.json",
|
||||
"drift_signals": "skill_atlas/drift_signals.json",
|
||||
"no_route_opportunities": "skill_atlas/no_route_opportunities.json",
|
||||
"report_json": "reports/skill_atlas.json",
|
||||
"report_html": "reports/skill_atlas.html"
|
||||
|
||||
@@ -70,12 +70,12 @@
|
||||
{
|
||||
"field": "archive_sha256",
|
||||
"from": "",
|
||||
"to": "176d93befb446985aaff7cd8325b05c4858da13177cb13db82d210aa2d264060"
|
||||
"to": "23112632363494d9fcb52b0253bb23ffc9111e2dec2df0e9b06e3a49f304ff20"
|
||||
},
|
||||
{
|
||||
"field": "package_sha256",
|
||||
"from": "0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"to": "7a8dab0d542d2994adec2045b54559612db36f14db8b7b66e4138fb0435ae3d3"
|
||||
"to": "ad55cf77f6b8faf7381b78372a93189c26e0e1833257bf7b508afd7ffdf0fbbb"
|
||||
}
|
||||
]
|
||||
},
|
||||
|
||||
@@ -57,6 +57,7 @@ DEFAULT_SCOPE = {
|
||||
"actionable": True,
|
||||
"scope_reason": "default release-actionable skill",
|
||||
}
|
||||
TELEMETRY_REQUIRED_MATURITIES = {"production", "library", "governed"}
|
||||
|
||||
|
||||
def parse_frontmatter(path: Path) -> tuple[dict[str, Any], str]:
|
||||
@@ -217,6 +218,105 @@ def collect_skill(workspace_root: Path, skill_dir: Path, policy: dict[str, Any])
|
||||
}
|
||||
|
||||
|
||||
def load_telemetry_profile(workspace_root: Path, skill_dir: Path, skill: dict[str, Any]) -> tuple[dict[str, Any], list[dict[str, Any]]]:
|
||||
report_path = skill_dir / "reports" / "adoption_drift_report.json"
|
||||
rel_report = safe_rel(workspace_root, report_path)
|
||||
maturity = str(skill.get("maturity", "")).casefold()
|
||||
report_present = report_path.exists()
|
||||
telemetry = {
|
||||
"report_present": report_present,
|
||||
"report": rel_report,
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0,
|
||||
}
|
||||
signals: list[dict[str, Any]] = []
|
||||
if not report_present:
|
||||
if skill.get("actionable") and maturity in TELEMETRY_REQUIRED_MATURITIES:
|
||||
signals.append(
|
||||
{
|
||||
"name": skill["name"],
|
||||
"path": skill["path"],
|
||||
"source": rel_report,
|
||||
"risk_band": "no-data",
|
||||
"signal_types": ["no telemetry"],
|
||||
"recommendation": "Render adoption drift evidence or record a small metadata-only sample before release review.",
|
||||
"actionable": bool(skill.get("actionable")),
|
||||
"scope": str(skill.get("atlas_scope", "")),
|
||||
"summary": {"event_count": 0, "adoption_sample_count": 0},
|
||||
}
|
||||
)
|
||||
return telemetry, signals
|
||||
|
||||
payload = load_json(report_path)
|
||||
summary = payload.get("summary", {}) if isinstance(payload.get("summary"), dict) else {}
|
||||
candidates = payload.get("next_iteration_candidates", [])
|
||||
candidates = candidates if isinstance(candidates, list) else []
|
||||
risk_band = str(summary.get("risk_band") or "unknown")
|
||||
telemetry.update(
|
||||
{
|
||||
"risk_band": risk_band,
|
||||
"event_count": int(summary.get("event_count") or 0),
|
||||
"adoption_sample_count": int(summary.get("adoption_sample_count") or 0),
|
||||
"adoption_rate": summary.get("adoption_rate", 0),
|
||||
"candidate_count": len(candidates),
|
||||
}
|
||||
)
|
||||
|
||||
signal_types: list[str] = []
|
||||
if telemetry["event_count"] == 0 and maturity in TELEMETRY_REQUIRED_MATURITIES:
|
||||
signal_types.append("no telemetry")
|
||||
if int(summary.get("missed_trigger_count") or 0):
|
||||
signal_types.append("missed trigger")
|
||||
if int(summary.get("wrong_trigger_count") or 0):
|
||||
signal_types.append("wrong trigger")
|
||||
if int(summary.get("bad_output_count") or 0):
|
||||
signal_types.append("bad output")
|
||||
if int(summary.get("missing_resource_count") or 0):
|
||||
signal_types.append("missing resource")
|
||||
if int(summary.get("script_error_count") or 0):
|
||||
signal_types.append("script error")
|
||||
if int(summary.get("review_overdue_count") or 0):
|
||||
signal_types.append("review overdue")
|
||||
if risk_band in {"medium", "high"} and not signal_types:
|
||||
signal_types.append("telemetry drift")
|
||||
|
||||
if signal_types:
|
||||
recommendation = "Convert telemetry drift into eval, trust, or owner-review actions."
|
||||
for candidate in candidates:
|
||||
if not isinstance(candidate, dict):
|
||||
continue
|
||||
if str(candidate.get("signal", "")) in signal_types:
|
||||
recommendation = str(candidate.get("recommendation") or recommendation)
|
||||
break
|
||||
signals.append(
|
||||
{
|
||||
"name": skill["name"],
|
||||
"path": skill["path"],
|
||||
"source": rel_report,
|
||||
"risk_band": risk_band,
|
||||
"signal_types": signal_types,
|
||||
"recommendation": recommendation,
|
||||
"actionable": bool(skill.get("actionable")),
|
||||
"scope": str(skill.get("atlas_scope", "")),
|
||||
"summary": {
|
||||
"event_count": telemetry["event_count"],
|
||||
"adoption_sample_count": telemetry["adoption_sample_count"],
|
||||
"adoption_rate": telemetry["adoption_rate"],
|
||||
"missed_trigger_count": int(summary.get("missed_trigger_count") or 0),
|
||||
"wrong_trigger_count": int(summary.get("wrong_trigger_count") or 0),
|
||||
"bad_output_count": int(summary.get("bad_output_count") or 0),
|
||||
"missing_resource_count": int(summary.get("missing_resource_count") or 0),
|
||||
"script_error_count": int(summary.get("script_error_count") or 0),
|
||||
"review_overdue_count": int(summary.get("review_overdue_count") or 0),
|
||||
},
|
||||
}
|
||||
)
|
||||
return telemetry, signals
|
||||
|
||||
|
||||
def route_overlap(skills: list[dict[str, Any]], threshold: float) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
|
||||
rows = []
|
||||
collisions = []
|
||||
@@ -392,9 +492,14 @@ def render_html(payload: dict[str, Any]) -> str:
|
||||
f"<td>{html.escape(skill.get('atlas_scope') or 'release')}</td>"
|
||||
"</tr>"
|
||||
)
|
||||
blockers = payload["actionable_route_collisions"][:20] + payload["actionable_owner_review_gaps"][:20] + payload["actionable_stale_skills"][:20]
|
||||
blockers = (
|
||||
payload["actionable_route_collisions"][:20]
|
||||
+ payload["actionable_owner_review_gaps"][:20]
|
||||
+ payload["actionable_stale_skills"][:20]
|
||||
+ payload["actionable_drift_signals"][:20]
|
||||
)
|
||||
blocker_items = "".join(
|
||||
f"<li><strong>{html.escape(item.get('name', item.get('skill_a', 'issue')))}</strong> {html.escape(item.get('reason', item.get('status', ', '.join(item.get('missing', [])))))}</li>"
|
||||
f"<li><strong>{html.escape(item.get('name', item.get('skill_a', 'issue')))}</strong> {html.escape(item.get('reason', item.get('status', ', '.join(item.get('missing', item.get('signal_types', []))))))}</li>"
|
||||
for item in blockers
|
||||
)
|
||||
return f"""<!DOCTYPE html>
|
||||
@@ -421,13 +526,14 @@ def render_html(payload: dict[str, Any]) -> str:
|
||||
<body>
|
||||
<main>
|
||||
<h1>Skill Atlas</h1>
|
||||
<p>Portfolio-level review for route overlap, stale ownership, shared resources, and no-route opportunities.</p>
|
||||
<p>Portfolio-level review for route overlap, stale ownership, shared resources, telemetry drift, and no-route opportunities.</p>
|
||||
<section class="grid">
|
||||
<div class="card"><span>Skills</span><strong>{summary['skill_count']}</strong></div>
|
||||
<div class="card"><span>Actionable</span><strong>{summary['actionable_skill_count']}</strong></div>
|
||||
<div class="card"><span>Route Collisions</span><strong>{summary['actionable_route_collision_count']}</strong></div>
|
||||
<div class="card"><span>Owner Gaps</span><strong>{summary['actionable_owner_gap_count']}</strong></div>
|
||||
<div class="card"><span>Stale Skills</span><strong>{summary['actionable_stale_count']}</strong></div>
|
||||
<div class="card"><span>Drift Signals</span><strong>{summary['actionable_drift_signal_count']}</strong></div>
|
||||
<div class="card"><span>No-Route Opportunities</span><strong>{summary['no_route_opportunity_count']}</strong></div>
|
||||
</section>
|
||||
<section>
|
||||
@@ -436,7 +542,7 @@ def render_html(payload: dict[str, Any]) -> str:
|
||||
</section>
|
||||
<section>
|
||||
<h2>Full Portfolio Counts</h2>
|
||||
<p>All scanned skills remain visible: {summary['route_collision_count']} total route collisions, {summary['owner_gap_count']} total owner gaps, and {summary['stale_count']} total stale signals.</p>
|
||||
<p>All scanned skills remain visible: {summary['route_collision_count']} total route collisions, {summary['owner_gap_count']} total owner gaps, {summary['stale_count']} total stale signals, and {summary['drift_signal_count']} telemetry drift signals.</p>
|
||||
</section>
|
||||
<section>
|
||||
<h2>Catalog</h2>
|
||||
@@ -455,7 +561,16 @@ def build_atlas(workspace_root: Path, output_dir: Path, report_html: Path, repor
|
||||
workspace_root = workspace_root.resolve()
|
||||
scope_policy = load_scope_policy(workspace_root)
|
||||
skill_dirs = find_skill_dirs(workspace_root)
|
||||
skills = [collect_skill(workspace_root, skill_dir, scope_policy) for skill_dir in skill_dirs]
|
||||
skills = []
|
||||
drift_signals: list[dict[str, Any]] = []
|
||||
telemetry_report_count = 0
|
||||
for skill_dir in skill_dirs:
|
||||
skill = collect_skill(workspace_root, skill_dir, scope_policy)
|
||||
telemetry, signals = load_telemetry_profile(workspace_root, skill_dir, skill)
|
||||
skill["telemetry"] = telemetry
|
||||
telemetry_report_count += 1 if telemetry["report_present"] else 0
|
||||
drift_signals.extend(signals)
|
||||
skills.append(skill)
|
||||
overlap_rows, collisions = route_overlap(skills, threshold)
|
||||
graph = dependency_graph(skills)
|
||||
stale = stale_skills(skills, today)
|
||||
@@ -465,6 +580,7 @@ def build_atlas(workspace_root: Path, output_dir: Path, report_html: Path, repor
|
||||
actionable_collisions = [item for item in collisions if item.get("actionable")]
|
||||
actionable_stale = [item for item in stale if item.get("actionable")]
|
||||
actionable_owner_gaps = [item for item in owner_gaps if item.get("actionable")]
|
||||
actionable_drift_signals = [item for item in drift_signals if item.get("actionable")]
|
||||
summary = {
|
||||
"skill_count": len(skills),
|
||||
"actionable_skill_count": len(actionable_skills),
|
||||
@@ -476,9 +592,13 @@ def build_atlas(workspace_root: Path, output_dir: Path, report_html: Path, repor
|
||||
"actionable_stale_count": len(actionable_stale),
|
||||
"shared_resource_count": len(graph["shared_resources"]),
|
||||
"no_route_opportunity_count": len(opportunities),
|
||||
"telemetry_report_count": telemetry_report_count,
|
||||
"drift_signal_count": len(drift_signals),
|
||||
"actionable_drift_signal_count": len(actionable_drift_signals),
|
||||
"non_actionable_issue_count": (len(collisions) - len(actionable_collisions))
|
||||
+ (len(owner_gaps) - len(actionable_owner_gaps))
|
||||
+ (len(stale) - len(actionable_stale)),
|
||||
+ (len(stale) - len(actionable_stale))
|
||||
+ (len(drift_signals) - len(actionable_drift_signals)),
|
||||
}
|
||||
catalog = {
|
||||
"workspace_root": display_path(workspace_root),
|
||||
@@ -499,6 +619,8 @@ def build_atlas(workspace_root: Path, output_dir: Path, report_html: Path, repor
|
||||
"actionable_stale_skills": actionable_stale,
|
||||
"owner_review_gaps": owner_gaps,
|
||||
"actionable_owner_review_gaps": actionable_owner_gaps,
|
||||
"drift_signals": drift_signals,
|
||||
"actionable_drift_signals": actionable_drift_signals,
|
||||
"no_route_opportunities": opportunities,
|
||||
"artifacts": {
|
||||
"catalog": display_path(output_dir / "catalog.json"),
|
||||
@@ -506,6 +628,7 @@ def build_atlas(workspace_root: Path, output_dir: Path, report_html: Path, repor
|
||||
"dependency_graph": display_path(output_dir / "dependency_graph.json"),
|
||||
"stale_skills": display_path(output_dir / "stale_skills.json"),
|
||||
"owner_review_gaps": display_path(output_dir / "owner_review_gaps.json"),
|
||||
"drift_signals": display_path(output_dir / "drift_signals.json"),
|
||||
"no_route_opportunities": display_path(output_dir / "no_route_opportunities.json"),
|
||||
"report_json": display_path(report_json),
|
||||
"report_html": display_path(report_html),
|
||||
@@ -517,6 +640,7 @@ def build_atlas(workspace_root: Path, output_dir: Path, report_html: Path, repor
|
||||
(output_dir / "dependency_graph.json").write_text(json.dumps(graph, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
(output_dir / "stale_skills.json").write_text(json.dumps(stale, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
(output_dir / "owner_review_gaps.json").write_text(json.dumps(owner_gaps, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
(output_dir / "drift_signals.json").write_text(json.dumps(drift_signals, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
(output_dir / "no_route_opportunities.json").write_text(json.dumps(opportunities, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
report_json.parent.mkdir(parents=True, exist_ok=True)
|
||||
report_html.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@@ -634,6 +634,9 @@ def render_html(report: dict[str, Any]) -> str:
|
||||
"actionable_route_collision_count",
|
||||
"actionable_owner_gap_count",
|
||||
"actionable_stale_count",
|
||||
"telemetry_report_count",
|
||||
"actionable_drift_signal_count",
|
||||
"drift_signal_count",
|
||||
"non_actionable_issue_count",
|
||||
],
|
||||
"skill atlas summary missing",
|
||||
|
||||
@@ -327,7 +327,8 @@ def build_gates(skill_dir: Path, output_html: Path, data: dict[str, dict[str, An
|
||||
)
|
||||
actionable_owner_gaps = int(atlas_summary.get("actionable_owner_gap_count", atlas_summary.get("owner_gap_count", 0)) or 0)
|
||||
actionable_stale = int(atlas_summary.get("actionable_stale_count", atlas_summary.get("stale_count", 0)) or 0)
|
||||
atlas_issues = actionable_route_collisions + actionable_owner_gaps + actionable_stale
|
||||
actionable_drift = int(atlas_summary.get("actionable_drift_signal_count", atlas_summary.get("drift_signal_count", 0)) or 0)
|
||||
atlas_issues = actionable_route_collisions + actionable_owner_gaps + actionable_stale + actionable_drift
|
||||
if not atlas:
|
||||
atlas_status = "warn"
|
||||
atlas_detail = "skill atlas is missing; portfolio-level conflicts are unknown"
|
||||
@@ -339,6 +340,7 @@ def build_gates(skill_dir: Path, output_html: Path, data: dict[str, dict[str, An
|
||||
f"{actionable_route_collisions} actionable route collisions; "
|
||||
f"{actionable_owner_gaps} actionable owner gaps; "
|
||||
f"{actionable_stale} actionable stale; "
|
||||
f"{actionable_drift} actionable drift; "
|
||||
f"{atlas_summary.get('non_actionable_issue_count', 0)} scoped non-actionable issues"
|
||||
)
|
||||
gates.append(
|
||||
|
||||
+123
-12
@@ -146,7 +146,16 @@
|
||||
],
|
||||
"atlas_scope": "release",
|
||||
"actionable": true,
|
||||
"scope_reason": "default release-actionable skill"
|
||||
"scope_reason": "default release-actionable skill",
|
||||
"telemetry": {
|
||||
"report_present": true,
|
||||
"report": "reports/adoption_drift_report.json",
|
||||
"risk_band": "low",
|
||||
"event_count": 1,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "release-orchestrator",
|
||||
@@ -204,7 +213,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/complex-release-orchestrator/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -243,7 +261,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/evolution-frontend-review/final/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -266,7 +293,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/evolution-frontend-review/v0/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -304,7 +340,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/evolution-frontend-review/v1/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "incident-command-governor",
|
||||
@@ -345,7 +390,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/governed-incident-command/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "note-cleanup",
|
||||
@@ -382,7 +436,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/simple-note-cleanup/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "frontend-review",
|
||||
@@ -422,7 +485,16 @@
|
||||
],
|
||||
"atlas_scope": "example",
|
||||
"actionable": false,
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package."
|
||||
"scope_reason": "Generated examples and evolution snapshots are portfolio evidence, not release-actionable library skills for the root package.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "examples/team-frontend-review/generated-skill/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "geo-ranking-article-generator",
|
||||
@@ -482,7 +554,16 @@
|
||||
],
|
||||
"atlas_scope": "embedded-generated-skill",
|
||||
"actionable": false,
|
||||
"scope_reason": "This nested generated skill is retained as local evidence and should not gate the root Meta Skill release."
|
||||
"scope_reason": "This nested generated skill is retained as local evidence and should not gate the root Meta Skill release.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "geo-ranking-article-generator/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "invalid-governance-skill",
|
||||
@@ -504,7 +585,16 @@
|
||||
],
|
||||
"atlas_scope": "test-fixture",
|
||||
"actionable": false,
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers."
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "tests/fixtures/governance_invalid_manifest/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "broken-yaml-skill",
|
||||
@@ -528,7 +618,16 @@
|
||||
],
|
||||
"atlas_scope": "test-fixture",
|
||||
"actionable": false,
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers."
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "tests/fixtures/package_invalid_yaml/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "broken-skill",
|
||||
@@ -551,7 +650,16 @@
|
||||
],
|
||||
"atlas_scope": "test-fixture",
|
||||
"actionable": false,
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers."
|
||||
"scope_reason": "Fixtures intentionally include invalid or broken skills to verify validators and packagers.",
|
||||
"telemetry": {
|
||||
"report_present": false,
|
||||
"report": "tests/fixtures/package_missing_interface_field/reports/adoption_drift_report.json",
|
||||
"risk_band": "missing",
|
||||
"event_count": 0,
|
||||
"adoption_sample_count": 0,
|
||||
"adoption_rate": 0,
|
||||
"candidate_count": 0
|
||||
}
|
||||
}
|
||||
],
|
||||
"summary": {
|
||||
@@ -565,6 +673,9 @@
|
||||
"actionable_stale_count": 0,
|
||||
"shared_resource_count": 0,
|
||||
"no_route_opportunity_count": 0,
|
||||
"telemetry_report_count": 1,
|
||||
"drift_signal_count": 0,
|
||||
"actionable_drift_signal_count": 0,
|
||||
"non_actionable_issue_count": 24
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
[]
|
||||
@@ -276,6 +276,7 @@ def main() -> None:
|
||||
atlas_gate = next(item for item in payload["gates"] if item["key"] == "skill-atlas")
|
||||
assert atlas_gate["status"] == "pass", atlas_gate
|
||||
assert "actionable route collisions" in atlas_gate["detail"], atlas_gate
|
||||
assert "actionable drift" in atlas_gate["detail"], atlas_gate
|
||||
operations_gate = next(item for item in payload["gates"] if item["key"] == "operations-loop")
|
||||
assert operations_gate["status"] == "pass", operations_gate
|
||||
assert "metadata events" in operations_gate["detail"], operations_gate
|
||||
@@ -307,6 +308,7 @@ def main() -> None:
|
||||
action_keys = {item["gate_key"] for item in full_payload["review_actions"]}
|
||||
assert action_keys == {"output-lab", "review-waivers"}, full_payload["review_actions"]
|
||||
assert full_payload["data"]["atlas"]["summary"]["actionable_route_collision_count"] == 0, full_payload["data"]["atlas"]
|
||||
assert full_payload["data"]["atlas"]["summary"]["actionable_drift_signal_count"] == 0, full_payload["data"]["atlas"]
|
||||
assert full_payload["data"]["atlas"]["summary"]["non_actionable_issue_count"] >= 1, full_payload["data"]["atlas"]
|
||||
synthetic_actions = review_studio.build_review_actions(
|
||||
[
|
||||
|
||||
@@ -43,13 +43,54 @@ description: {description}
|
||||
return skill_dir
|
||||
|
||||
|
||||
def write_adoption_report(skill_dir: Path, risk_band: str = "medium") -> None:
|
||||
reports = skill_dir / "reports"
|
||||
reports.mkdir(parents=True, exist_ok=True)
|
||||
(reports / "adoption_drift_report.json").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"ok": True,
|
||||
"summary": {
|
||||
"event_count": 4,
|
||||
"adoption_sample_count": 3,
|
||||
"activation_count": 2,
|
||||
"accepted_count": 1,
|
||||
"edited_count": 0,
|
||||
"rejected_count": 1,
|
||||
"missed_count": 1,
|
||||
"failed_count": 1,
|
||||
"adoption_rate": 33.3,
|
||||
"missed_trigger_count": 1,
|
||||
"wrong_trigger_count": 0,
|
||||
"bad_output_count": 1,
|
||||
"script_error_count": 1,
|
||||
"missing_resource_count": 0,
|
||||
"review_overdue_count": 0,
|
||||
"risk_band": risk_band,
|
||||
},
|
||||
"next_iteration_candidates": [
|
||||
{
|
||||
"signal": "bad output",
|
||||
"recommendation": "Convert the failed output shape into an Output Eval assertion.",
|
||||
}
|
||||
],
|
||||
"failures": [],
|
||||
},
|
||||
ensure_ascii=False,
|
||||
indent=2,
|
||||
)
|
||||
+ "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
tmp_root = ROOT / "tests" / "tmp_skill_atlas"
|
||||
if tmp_root.exists():
|
||||
shutil.rmtree(tmp_root)
|
||||
tmp_root.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
write_skill(
|
||||
alpha = write_skill(
|
||||
tmp_root,
|
||||
"alpha-release-skill",
|
||||
"Use when turning release notes and changelog snippets into a reusable release brief for teams.",
|
||||
@@ -63,7 +104,7 @@ def main() -> None:
|
||||
},
|
||||
shared_reference=True,
|
||||
)
|
||||
write_skill(
|
||||
beta = write_skill(
|
||||
tmp_root,
|
||||
"beta-release-skill",
|
||||
"Use when turning changelog notes and release snippets into reusable release briefs for team review.",
|
||||
@@ -75,6 +116,7 @@ def main() -> None:
|
||||
},
|
||||
shared_reference=True,
|
||||
)
|
||||
write_adoption_report(beta, risk_band="high")
|
||||
write_skill(
|
||||
tmp_root,
|
||||
"gamma-doc-skill",
|
||||
@@ -149,19 +191,29 @@ def main() -> None:
|
||||
assert summary["stale_count"] >= 1, summary
|
||||
assert summary["shared_resource_count"] >= 1, summary
|
||||
assert summary["no_route_opportunity_count"] >= 1, summary
|
||||
assert summary["drift_signal_count"] == 2, summary
|
||||
assert summary["actionable_drift_signal_count"] == 1, summary
|
||||
assert summary["telemetry_report_count"] == 1, summary
|
||||
assert (output_dir / "catalog.json").exists(), output_dir
|
||||
assert (output_dir / "route_overlap_matrix.csv").exists(), output_dir
|
||||
assert (output_dir / "dependency_graph.json").exists(), output_dir
|
||||
assert (output_dir / "stale_skills.json").exists(), output_dir
|
||||
assert (output_dir / "owner_review_gaps.json").exists(), output_dir
|
||||
assert (output_dir / "drift_signals.json").exists(), output_dir
|
||||
assert (output_dir / "no_route_opportunities.json").exists(), output_dir
|
||||
html = report_html.read_text(encoding="utf-8")
|
||||
assert "Skill Atlas" in html, html[:1000]
|
||||
assert "Route Collisions" in html, html[:3000]
|
||||
assert "Actionable Issues" in html, html[:3000]
|
||||
assert "Drift Signals" in html, html[:3000]
|
||||
assert "No-Route Opportunities" in html, html[:3000]
|
||||
catalog = json.loads((output_dir / "catalog.json").read_text(encoding="utf-8"))
|
||||
assert catalog["summary"]["skill_count"] == 3, catalog
|
||||
alpha_catalog = next(item for item in catalog["skills"] if item["name"] == alpha.name)
|
||||
assert alpha_catalog["telemetry"]["risk_band"] == "missing", alpha_catalog
|
||||
drift = json.loads((output_dir / "drift_signals.json").read_text(encoding="utf-8"))
|
||||
assert any(item["name"] == alpha.name and item["signal_types"] == ["no telemetry"] for item in drift), drift
|
||||
assert any(item["name"] == beta.name and "bad output" in item["signal_types"] and not item["actionable"] for item in drift), drift
|
||||
assert payload["scope_policy"]["present"], payload["scope_policy"]
|
||||
|
||||
print(json.dumps({"ok": True}, ensure_ascii=False, indent=2))
|
||||
|
||||
Reference in New Issue
Block a user