From 24bea607a2b16e766bc8f2bc8483f22fae1edc61 Mon Sep 17 00:00:00 2001 From: yaojingang Date: Thu, 2 Jul 2026 11:29:19 +0800 Subject: [PATCH] Improve yao meta skill trigger coverage --- SKILL.md | 2 +- evals/improved_description.txt | 2 +- evals/semantic_config.json | 89 +- evals/trigger_cases.json | 52 + registry/index.json | 4 +- registry/packages/yao-meta-skill.json | 6 +- reports/adaptation_proposals.json | 2 +- reports/adaptation_proposals.md | 2 +- reports/adoption_drift_report.json | 2 +- reports/artifact-design-profile.json | 2 +- reports/artifact-design-profile.md | 2 +- reports/baseline-compare.json | 4 +- reports/benchmark_reproducibility.json | 124 +- reports/benchmark_reproducibility.md | 63 +- reports/candidate_registry.json | 64 +- reports/candidate_registry.md | 2 +- reports/compiled_targets.json | 42 +- reports/context_budget.json | 16 +- reports/context_budget.md | 2 +- reports/context_budget_summary.json | 10 +- reports/description_optimization.json | 1510 ++++++++------- reports/description_optimization.md | 14 +- reports/description_optimization_suite.json | 56 +- reports/description_optimization_suite.md | 2 +- reports/eval_suite.json | 625 +++--- reports/evidence_consistency.json | 122 +- reports/evidence_consistency.md | 2 +- reports/install_simulation.json | 20 +- reports/install_simulation.md | 4 +- .../yao-meta-skill/bundle.json | 8 +- .../yao-meta-skill/bundle.md | 6 +- .../yao-meta-skill/review.md | 4 +- reports/iteration_ledger.md | 4 +- reports/promotion_decisions.json | 64 +- reports/prompt-quality-profile.json | 2 +- reports/prompt-quality-profile.md | 2 +- reports/python_compatibility.json | 8 +- reports/python_compatibility.md | 2 +- reports/reference-synthesis.json | 34 +- reports/reference-synthesis.md | 18 +- reports/registry_audit.json | 10 +- reports/registry_audit.md | 2 +- reports/review-studio.html | 28 +- reports/review-studio.json | 702 ++++--- reports/review-viewer.html | 14 +- reports/review-viewer.json | 1682 ++++++++++------- reports/route_scorecard.json | 99 +- reports/route_scorecard.md | 4 +- reports/runtime_permission_probes.json | 10 +- reports/security_trust_report.json | 2 +- reports/security_trust_report.md | 2 +- reports/skill-interpretation.html | 20 +- reports/skill-interpretation.json | 66 +- reports/skill-os-2-review.md | 2 +- reports/skill-overview.html | 20 +- reports/skill-overview.json | 66 +- reports/skill_atlas.json | 30 +- reports/skill_os2_audit.json | 2 +- reports/skill_os2_audit.md | 2 +- reports/skill_os2_coverage.json | 10 +- reports/skill_os2_coverage.md | 6 +- reports/skillops/daily/2026-07-02.json | 513 +++++ reports/skillops/daily/2026-07-02.md | 174 ++ reports/skillops/weekly/2026-W27.json | 323 ++++ reports/skillops/weekly/2026-W27.md | 110 ++ reports/system-model.json | 4 +- reports/system-model.md | 2 +- reports/upgrade_check.json | 10 +- reports/world_class_claim_guard.json | 106 +- reports/world_class_claim_guard.md | 10 +- reports/world_class_evidence_intake.json | 2 +- reports/world_class_evidence_intake.md | 2 +- reports/world_class_evidence_ledger.json | 2 +- reports/world_class_evidence_ledger.md | 2 +- reports/world_class_evidence_plan.json | 2 +- reports/world_class_evidence_plan.md | 2 +- reports/world_class_evidence_preflight.html | 4 +- reports/world_class_evidence_preflight.json | 10 +- reports/world_class_evidence_preflight.md | 8 +- reports/world_class_operator_runbook.json | 2 +- reports/world_class_operator_runbook.md | 2 +- reports/world_class_submission_review.json | 2 +- reports/world_class_submission_review.md | 2 +- skill-ir/examples/yao-meta-skill.json | 2 +- skill_atlas/catalog.json | 30 +- skill_atlas/route_overlap_matrix.csv | 12 +- 86 files changed, 4506 insertions(+), 2610 deletions(-) create mode 100644 reports/skillops/daily/2026-07-02.json create mode 100644 reports/skillops/daily/2026-07-02.md create mode 100644 reports/skillops/weekly/2026-W27.json create mode 100644 reports/skillops/weekly/2026-W27.md diff --git a/SKILL.md b/SKILL.md index 89809d3b..ff429c54 100644 --- a/SKILL.md +++ b/SKILL.md @@ -1,6 +1,6 @@ --- name: yao-meta-skill -description: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution. +description: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs. metadata: author: Yao Team --- diff --git a/evals/improved_description.txt b/evals/improved_description.txt index a8fdc966..45240557 100644 --- a/evals/improved_description.txt +++ b/evals/improved_description.txt @@ -1 +1 @@ -Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use when asked to create a skill, turn a repeated process into a reusable skill, improve an existing skill, add evals, or package a skill for team reuse. Also trigger on common Chinese asks such as 做一个 skill, 把流程沉淀成可复用能力, 封装成团队可复用的 skill, 优化已有 skill, 补 trigger 评测, or 收紧触发边界. +Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs. diff --git a/evals/semantic_config.json b/evals/semantic_config.json index 80803467..5ed78259 100644 --- a/evals/semantic_config.json +++ b/evals/semantic_config.json @@ -18,12 +18,19 @@ "turn a repeated process into a reusable skill", "improve an existing skill", "add evals", + "add tests", "package a skill for team reuse", + "migrate an existing skill", + "prepare packaging, installation, or release", + "govern a reusable skill", "做一个 skill", "把流程沉淀成可复用 skill", "优化现有 skill", "补 trigger 评测", - "封装成团队可复用 skill" + "封装成团队可复用 skill", + "迁移 skill", + "发布 skill", + "打包发布检查" ], "exclusions": [ "summary-only requests", @@ -75,16 +82,29 @@ "formalize this process into a reusable capability", "turn this playbook into an agent capability package", "convert this into a maintained skill", + "agent capability", + "reusable agent capability", + "skill library entry", + "skill authoring", "做一个 skill", "创建一个 skill", "新建一个 skill", "帮我做个 skill", + "改一个 skill", + "重构一个 skill", + "迁移一个 skill", + "发布一个 skill", + "这个 skill", + "已有 skill", + "现有 skill", "做成一个 skill", "封装成一个 skill", "沉淀成一个 skill", "整理成一个 skill", "抽成一个 skill", "可复用 skill", + "可执行 skill", + "团队 skill", "agent 能力包", "能力包", "skill 包" @@ -103,19 +123,27 @@ "existing skill draft", "rough notes", "standard operating procedure", + "operating procedure", "sop", + "playbook", "transcript", + "chat transcript", + "meeting notes", "workflow fragments", "release notes", "onboarding flow", "support escalation routine", "quarterly release routine", + "scripts", + "repeated team practice", "流程", "工作流", "操作流程", "做事流程", + "作业流程", "方法", "经验", + "复盘", "sop 文档", "会议纪要", "对话记录", @@ -142,14 +170,32 @@ "ops library", "maintained", "operationalize", + "release-ready", + "publish", + "release", + "installation", + "installable", + "portability", + "registry", + "governance", + "review gate", "可复用", "复用", "封装", + "打包", "沉淀", "标准化", "标准化下来", "团队复用", "给团队用", + "团队可用", + "安装复用", + "可安装", + "发布", + "上架", + "交付", + "治理", + "注册表", "方法包", "团队能力包", "沉淀下来", @@ -171,16 +217,32 @@ "false negatives", "stress test", "route boundary", + "routing", + "route evals", + "trigger routing", + "activation", + "description optimization", + "tests", + "install checks", + "release checks", "hardening", "评测", "补评测", "补 eval", + "补测试", "触发测试", + "触发词", + "触发范围", + "召回", "收紧触发边界", "路由边界", + "路由评测", + "激活", "误触发", "漏触发", "质量门", + "安装验证", + "发布检查", "打包检查", "边界测试" ] @@ -198,11 +260,32 @@ "before sharing it with the team", "tighten the boundary", "revise this skill", + "refactor this skill", + "migrate this skill", + "port this skill", + "convert this skill", + "release this skill", + "publish this skill", + "make this skill release-ready", + "codex-ready package", + "claude skill", + "codex format", + "interface metadata", + "manifest", + "install checks", "draft skill", "harden this skill for library reuse", "优化这个 skill", "改进这个 skill", "重构这个 skill", + "扩大这个 skill", + "调整这个 skill", + "优化触发范围", + "扩大触发范围", + "迁移这个 skill", + "发布这个 skill", + "把这个 skill 迁移", + "让这个 skill 可发布", "补一下评测", "补 trigger 测试", "加强路由判断", @@ -351,12 +434,16 @@ "do not convert it into a reusable capability", "keep it as documentation only", "discussion only", + "no agent execution", "先不要做成 skill", + "先不做 skill", "不要做成 skill", "不用封装", "不要打包", "不要沉淀成能力", "先别做成可复用能力", + "不需要 agent 执行", + "不做 agent 能力", "只做文档", "只做说明" ] diff --git a/evals/trigger_cases.json b/evals/trigger_cases.json index 9be8184a..568d7ac6 100644 --- a/evals/trigger_cases.json +++ b/evals/trigger_cases.json @@ -54,6 +54,18 @@ "text": "Add trigger evals to this skill before sharing it with the team.", "family": "iterate_existing_skill" }, + { + "text": "Migrate this Claude skill into a Codex-ready package with interface metadata and install checks.", + "family": "migrate_existing_skill" + }, + { + "text": "Make this existing skill release-ready: add manifest/interface metadata, packaging checks, and trigger evals.", + "family": "release_packaging" + }, + { + "text": "Optimize the activation wording for this skill so broader skill-authoring requests are routed correctly.", + "family": "trigger_expansion" + }, { "text": "We have a messy release runbook, export process, and a prompt history; turn all of that into one reusable skill package with evals and packaging checks.", "family": "complex_multi_asset" @@ -133,6 +145,18 @@ { "text": "把这个客服升级流程标准化成一个可执行的 skill，并加上边界测试。", "family": "cn_eval_and_package" + }, + { + "text": "把这个 Claude skill 迁移成 Codex 可用的 skill 包，补 interface、manifest 和安装验证。", + "family": "cn_migrate_existing_skill" + }, + { + "text": "把这套提示词和脚本整理成团队能安装复用的 skill，并做打包发布检查。", + "family": "cn_release_packaging" + }, + { + "text": "扩大这个 skill 的触发范围，同时补路由评测，避免漏触发。", + "family": "cn_trigger_expansion" } ], "should_not_trigger": [ @@ -239,6 +263,22 @@ { "text": "先一起头脑风暴几个方向，不要封装成 skill。", "family": "cn_brainstorm_only" + }, + { + "text": "Polish this release note for publishing, but do not do any skill work.", + "family": "document_only" + }, + { + "text": "Migrate this wiki article into a new docs folder; do not create an agent skill.", + "family": "document_only" + }, + { + "text": "把这篇 SOP 改成知识库文档，不需要 agent 执行。", + "family": "cn_document_only" + }, + { + "text": "检查安装步骤是否清楚，只做 README，不做 skill 包。", + "family": "cn_document_only" } ], "near_neighbor": [ @@ -333,6 +373,18 @@ { "text": "这次只给我一个一次性 prompt，不用复用成 skill。", "family": "cn_one_off_vs_reusable" + }, + { + "text": "Create a reusable onboarding template for humans, not an agent skill.", + "family": "document_export_vs_agent_skill" + }, + { + "text": "Draft a release checklist for a future skill, but do not package or evaluate the skill yet.", + "family": "future_outline_vs_build" + }, + { + "text": "整理成团队发布流程文档即可，不要做成 skill，也不要做触发评测。", + "family": "cn_document_export_vs_agent_skill" } ] } diff --git a/registry/index.json b/registry/index.json index 0fcfae4f..6e2ea75e 100644 --- a/registry/index.json +++ b/registry/index.json @@ -1,6 +1,6 @@ { "schema_version": "2.0", - "generated_at": "2026-06-21", + "generated_at": "2026-06-13", "package_count": 1, "packages": [ { @@ -16,7 +16,7 @@ "vscode" ], "package_metadata": "registry/packages/yao-meta-skill.json", - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" } ] } diff --git a/registry/packages/yao-meta-skill.json b/registry/packages/yao-meta-skill.json index dac6bc05..c6c2310a 100644 --- a/registry/packages/yao-meta-skill.json +++ b/registry/packages/yao-meta-skill.json @@ -2,7 +2,7 @@ "schema_version": "2.0", "name": "yao-meta-skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "targets": [ "openai", "claude", @@ -16,7 +16,7 @@ "trust_level": "local", "license": "MIT", "checksums": { - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7" }, "compatibility": { @@ -53,5 +53,5 @@ "install_simulated": true, "install_simulation": "reports/install_simulation.json" }, - "generated_at": "2026-06-21" + "generated_at": "2026-06-13" } diff --git a/reports/adaptation_proposals.json b/reports/adaptation_proposals.json index 359d4042..bfc1ca86 100644 --- a/reports/adaptation_proposals.json +++ b/reports/adaptation_proposals.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02T03:41:56Z", "skill_dir": ".", "source_patterns": "reports/user_patterns.json", "pattern_count": 5, diff --git a/reports/adaptation_proposals.md b/reports/adaptation_proposals.md index 8092cdc6..61b2ec94 100644 --- a/reports/adaptation_proposals.md +++ b/reports/adaptation_proposals.md @@ -1,6 +1,6 @@ # Adaptation Proposals -- Generated at: `2026-06-21` +- Generated at: `2026-07-02T03:41:56Z` - Pattern report: `reports/user_patterns.json` - Proposal only: `true` - Writes repository files: `false` diff --git a/reports/adoption_drift_report.json b/reports/adoption_drift_report.json index a49f1da6..a34b0ca2 100644 --- a/reports/adoption_drift_report.json +++ b/reports/adoption_drift_report.json @@ -1,7 +1,7 @@ { "ok": true, "schema_version": "2.0", - "generated_at": "2026-06-20T12:35:58Z", + "generated_at": "2026-07-02T03:42:30Z", "skill_dir": ".", "privacy_contract": { "storage": "local-first", diff --git a/reports/artifact-design-profile.json b/reports/artifact-design-profile.json index c6dc2de0..8037e9fa 100644 --- a/reports/artifact-design-profile.json +++ b/reports/artifact-design-profile.json @@ -1,5 +1,5 @@ { - "skill_name": "yao-meta-skill-operator-ux-worktree", + "skill_name": "yao-meta-skill-trigger-coverage-merge", "design_system": "metric editorial", "primary_artifact": { "key": "dashboard", diff --git a/reports/artifact-design-profile.md b/reports/artifact-design-profile.md index ec07eac7..11c20cb1 100644 --- a/reports/artifact-design-profile.md +++ b/reports/artifact-design-profile.md @@ -1,6 +1,6 @@ # Artifact Design Profile -Skill: `yao-meta-skill-operator-ux-worktree` +Skill: `yao-meta-skill-trigger-coverage-merge` Design system: `metric editorial` ## Primary Artifact Direction diff --git a/reports/baseline-compare.json b/reports/baseline-compare.json index 0896a070..3af1e417 100644 --- a/reports/baseline-compare.json +++ b/reports/baseline-compare.json @@ -20,7 +20,7 @@ "adversarial_errors": 0 }, "current": { - "tokens": 53, + "tokens": 54, "dev_errors": 0, "holdout_errors": 0, "blind_errors": 0, @@ -28,7 +28,7 @@ "adversarial_errors": 0 }, "winner": { - "tokens": 53, + "tokens": 54, "dev_errors": 0, "holdout_errors": 0, "blind_errors": 0, diff --git a/reports/benchmark_reproducibility.json b/reports/benchmark_reproducibility.json index 010c57a9..29ac3648 100644 --- a/reports/benchmark_reproducibility.json +++ b/reports/benchmark_reproducibility.json @@ -1,27 +1,48 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02", "skill_dir": ".", - "commit": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", + "commit": "1710a5a3e0177313954758908735c147796e8981", "git_status": { "available": true, "dirty": true, - "changed_file_count": 3, + "changed_file_count": 84, "generated_dirty": true, - "generated_changed_file_count": 3, - "source_dirty": false, - "source_changed_file_count": 0, + "generated_changed_file_count": 82, + "source_dirty": true, + "source_changed_file_count": 2, "sample": [ - " M reports/context_budget.json", - " M reports/context_budget.md", - " M reports/context_budget_summary.json" + " M SKILL.md", + " M evals/improved_description.txt", + " M registry/index.json", + " M registry/packages/yao-meta-skill.json", + " M reports/adaptation_proposals.json", + " M reports/adaptation_proposals.md", + " M reports/adoption_drift_report.json", + " M reports/artifact-design-profile.json", + " M reports/artifact-design-profile.md", + " M reports/baseline-compare.json", + " M reports/benchmark_reproducibility.json", + " M reports/benchmark_reproducibility.md" + ], + "source_sample": [ + " M SKILL.md", + " M evals/improved_description.txt" ], - "source_sample": [], "generated_sample": [ - " M reports/context_budget.json", - " M reports/context_budget.md", - " M reports/context_budget_summary.json" + " M registry/index.json", + " M registry/packages/yao-meta-skill.json", + " M reports/adaptation_proposals.json", + " M reports/adaptation_proposals.md", + " M reports/adoption_drift_report.json", + " M reports/artifact-design-profile.json", + " M reports/artifact-design-profile.md", + " M reports/baseline-compare.json", + " M reports/benchmark_reproducibility.json", + " M reports/benchmark_reproducibility.md", + " M reports/candidate_registry.json", + " M reports/candidate_registry.md" ], "generated_dirty_prefixes": [ "dist/", @@ -35,12 +56,12 @@ }, "summary": { "reproducibility_ready": true, - "release_lock_ready": true, + "release_lock_ready": false, "methodology_complete": true, "required_artifact_count": 25, "missing_artifact_count": 0, - "evidence_bundle_sha256": "e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64cbe8db9ad", - "source_contract_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "evidence_bundle_sha256": "639707e807e5ec8460f6aa6d1e5ba34874bb3228160555d8a01dbc37620d4873", + "source_contract_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7", "output_case_count": 5, "failure_disclosure_count": 3, @@ -58,22 +79,24 @@ "world_class_source_check_count": 19, "world_class_source_pass_count": 12, "world_class_source_blocked_count": 7, - "beta_test_ready": true, - "beta_test_blocker_count": 0, + "beta_test_ready": false, + "beta_test_blocker_count": 1, "beta_test_deferred_evidence_count": 4, "public_claim_ready": false, - "public_claim_blocker_count": 3, + "public_claim_blocker_count": 4, "working_tree_dirty": true, - "changed_file_count": 3, - "source_tree_dirty": false, - "source_changed_file_count": 0, + "changed_file_count": 84, + "source_tree_dirty": true, + "source_changed_file_count": 2, "generated_tree_dirty": true, - "generated_changed_file_count": 3 + "generated_changed_file_count": 82 }, "beta_test_release": { - "ready": true, + "ready": false, "scope": "beta/public test release without superiority, fully-reviewed, or world-class claims", - "blockers": [], + "blockers": [ + "release lock is not clean or commit is unavailable" + ], "allowed_deferred_evidence": [ { "key": "provider-holdout", @@ -103,6 +126,7 @@ "ready": false, "scope": "public benchmark or world-class readiness claim", "blockers": [ + "release lock is not clean or commit is unavailable", "human blind-review adjudication is incomplete", "world-class evidence is not accepted yet (3 open gaps, 4 ledger pending)", "world-class source checks are not all accepted (12/19 pass, 7 blocked)" @@ -110,12 +134,12 @@ "policy": "Local reproducibility can pass before public claims; public claims require provider evidence, human adjudication, clean release lock, accepted world-class evidence, and complete source checks." }, "release_lock": { - "ready": true, - "commit": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", + "ready": false, + "commit": "1710a5a3e0177313954758908735c147796e8981", "status_scope": "generation-time status before this report is written", - "source_changed_file_count": 0, - "generated_changed_file_count": 3, - "reason": "only generated evidence artifacts were dirty at generation time" + "source_changed_file_count": 2, + "generated_changed_file_count": 82, + "reason": "source files were dirty at generation time" }, "evidence_bundle": { "algorithm": "sha256(path,label,exists,artifact_sha256)", @@ -123,7 +147,7 @@ "existing_count": 25, "missing_count": 0, "missing_paths": [], - "sha256": "e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64cbe8db9ad" + "sha256": "639707e807e5ec8460f6aa6d1e5ba34874bb3228160555d8a01dbc37620d4873" }, "methodology": { "path": "reports/benchmark_methodology.md", @@ -217,8 +241,8 @@ "label": "trigger_scorecard", "path": "reports/route_scorecard.json", "exists": true, - "bytes": 16950, - "sha256": "a649547a7d3d5fe02262ca1b188ed4022d8f557770be425f7b7cf762aa76533a" + "bytes": 17042, + "sha256": "53fc22d220dc11453c1e66a17ec2d935e069f9bc936952fe4a53afdb4c71b2cc" }, { "label": "runtime_conformance", @@ -232,21 +256,21 @@ "path": "reports/security_trust_report.json", "exists": true, "bytes": 139090, - "sha256": "2ef43f009be68b1dd1a994b841d69b89723cf0b6d7725dc6ad86670d14235135" + "sha256": "ffe2c88bbdbac4907bf8db8f01b9a6b2da78e83acdffb3d975e99424274cd9f7" }, { "label": "python_compatibility", "path": "reports/python_compatibility.json", "exists": true, - "bytes": 30351, - "sha256": "7f3fc8e785bf7007755c0b7f277967fd478ea17a5800ab14b9c23de395576cda" + "bytes": 30497, + "sha256": "4d82942052a2ca87db7acea451096332d4b8bf6420a29fa9f5596e83464d21ea" }, { "label": "registry_audit", "path": "reports/registry_audit.json", "exists": true, - "bytes": 3137, - "sha256": "03cb951dc8ba71843602bffa1148d7a40095a7e109a024193422aaa8a94ba4c7" + "bytes": 3155, + "sha256": "9be6de364d69a2ad40b68b460a35c91476e822eb008cfe4d5e8614c1e989d3d4" }, { "label": "package_verification", @@ -259,64 +283,64 @@ "label": "install_simulation", "path": "reports/install_simulation.json", "exists": true, - "bytes": 8793, - "sha256": "6cf79201727352e5f92f4e1fa1697a069ab325d526065f5316ee939e84475ef5" + "bytes": 8947, + "sha256": "65adf51d8c4b1d5ea49edde09004f7e4bfc06127e640baf2de23af695fbe0728" }, { "label": "skill_os2_audit", "path": "reports/skill_os2_audit.json", "exists": true, "bytes": 13824, - "sha256": "1079221b5ac72e19e3e7a0f4970f133d828336f34bac82abf3fb867d18834cfa" + "sha256": "150c0027deaa3c2dd44db5380e9a9b16f123e8a8e3ff6142c711b2b0c432f81c" }, { "label": "world_class_evidence_plan", "path": "reports/world_class_evidence_plan.json", "exists": true, "bytes": 23314, - "sha256": "374ee899a0ba5b84a95337de993439d78e5708be4fdf40dee4292766d0d2bf99" + "sha256": "483ed2b3bd020fa2c5a888300c0d0c1a265eebaf648076581f082f2d527d3669" }, { "label": "world_class_evidence_ledger", "path": "reports/world_class_evidence_ledger.json", "exists": true, "bytes": 26274, - "sha256": "565ae80a82286e912df134c88a269566ecf635f39e55618df4ed4eac69d7c7ea" + "sha256": "b19e7113ec463dbb1cfb87c45d3c51624151aeea1747c8d9c286b6967a385c6a" }, { "label": "world_class_evidence_intake", "path": "reports/world_class_evidence_intake.json", "exists": true, "bytes": 20922, - "sha256": "e36a7e753f4176e7ddd7a74fcc5b064769191baf73b5faa58752f80457ec58ff" + "sha256": "885bff9c1c8546c606f583a982a2283229438dbdcd44f3c7e55aff71f858f8f4" }, { "label": "world_class_evidence_preflight", "path": "reports/world_class_evidence_preflight.json", "exists": true, "bytes": 90852, - "sha256": "9e4f0bc489ca958b5a9e2854f17b847de8ee1f081de40e8c8b86cec138afe633" + "sha256": "a0b00eed5450b33ec29fea6652d41db216931452eeddd0d99f2d90aa20c44d95" }, { "label": "world_class_submission_review", "path": "reports/world_class_submission_review.json", "exists": true, "bytes": 17292, - "sha256": "3c944b4bd27f61483e8e02aa3f3b6c8cdb4aba8dc986241e8cbe60ba64489123" + "sha256": "e8df5c72ade7ad0704b6ee5dd6aedefea98f79a9a370cd4d80655401862e87f4" }, { "label": "world_class_operator_runbook", "path": "reports/world_class_operator_runbook.json", "exists": true, "bytes": 83712, - "sha256": "d8678010fcfd5fb8f28d95925478496749f5498ce13a171c80f79372b874351d" + "sha256": "bb67813244d64f8e5e416620bb9d7856d95d85a1ec7f9581b2a97b9fed47e02e" }, { "label": "world_class_operator_runbook_markdown", "path": "reports/world_class_operator_runbook.md", "exists": true, "bytes": 25504, - "sha256": "fe3a73bf2eb9a018ec03881c95d23d30abcbf8bc36cbb264f68daa08238d8e5f" + "sha256": "e8d4dee7541323f0e6fa6be4903e7c482b1d00ef1db5594f0c44212088b53d85" }, { "label": "world_class_operator_runbook_html", @@ -329,8 +353,8 @@ "label": "world_class_claim_guard", "path": "reports/world_class_claim_guard.json", "exists": true, - "bytes": 18759, - "sha256": "556700f05a65bda9b55d829c109d1aec54ae5b4da80ffbefe1196319df86f117" + "bytes": 20102, + "sha256": "5eb808496802e8f972e5aa738d99f3b4a37ff34c8ab495f25ff2db3e7237239f" } ], "missing_artifacts": [], diff --git a/reports/benchmark_reproducibility.md b/reports/benchmark_reproducibility.md index b4efcd75..38ce2c60 100644 --- a/reports/benchmark_reproducibility.md +++ b/reports/benchmark_reproducibility.md @@ -1,20 +1,20 @@ # Benchmark Reproducibility -Generated at: `2026-06-21` -Commit: `139d0fd15ce995757d6d75fb45fb417d0b1b4e7d` +Generated at: `2026-07-02` +Commit: `1710a5a3e0177313954758908735c147796e8981` Working tree dirty at generation: `true` -Source tree dirty at generation: `false` +Source tree dirty at generation: `true` Generated evidence dirty at generation: `true` -Evidence bundle SHA256: `e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64cbe8db9ad` +Evidence bundle SHA256: `639707e807e5ec8460f6aa6d1e5ba34874bb3228160555d8a01dbc37620d4873` ## Summary - reproducibility ready: `true` -- release lock ready: `true` +- release lock ready: `false` - methodology complete: `true` - required artifacts: `25` - missing artifacts: `0` -- source contract sha256: `eddeea80a657` +- source contract sha256: `e0863533e0a0` - archive sha256: `c56a8519244c` - output cases: `5` - disclosed failure cases: `3` @@ -23,27 +23,27 @@ Evidence bundle SHA256: `e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64 - human review complete: `false` - world-class ready: `false` - world-class source checks: `12` pass / `19` total; `7` blocked -- beta test ready: `true` -- beta test blockers: `0` +- beta test ready: `false` +- beta test blockers: `1` - beta deferred evidence: `4` - public claim ready: `false` -- public claim blockers: `3` -- changed files at generation: `3` -- source changed files at generation: `0` -- generated changed files at generation: `3` +- public claim blockers: `4` +- changed files at generation: `84` +- source changed files at generation: `2` +- generated changed files at generation: `82` This report proves local benchmark reproducibility only. It keeps external provider and human-review gaps visible instead of counting them as complete. The git commit and dirty samples are generation-time context; the evidence bundle SHA is the durable anchor for the artifacts listed below. ## Beta Test Boundary -- ready: `true` +- ready: `false` - scope: beta/public test release without superiority, fully-reviewed, or world-class claims - policy: Human blind-review, native permission enforcement, real client telemetry, and ledger acceptance may be deferred for beta/public testing, but public claims must remain blocked until those evidence entries are accepted. - required wording: Use beta, public test, or technical preview wording; do not claim world-class readiness, fully reviewed quality, or proven superiority over baseline. | Blocker | | --- | -| none | +| release lock is not clean or commit is unavailable | | Deferred evidence | Reason | | --- | --- | @@ -60,21 +60,22 @@ This report proves local benchmark reproducibility only. It keeps external provi | Blocker | | --- | +| release lock is not clean or commit is unavailable | | human blind-review adjudication is incomplete | | world-class evidence is not accepted yet (3 open gaps, 4 ledger pending) | | world-class source checks are not all accepted (12/19 pass, 7 blocked) | ## Release Lock -- ready: `true` -- reason: only generated evidence artifacts were dirty at generation time +- ready: `false` +- reason: source files were dirty at generation time - status scope: generation-time status before this report is written ## Evidence Bundle - algorithm: `sha256(path,label,exists,artifact_sha256)` - artifacts: `25` / `25` -- sha256: `e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64cbe8db9ad` +- sha256: `639707e807e5ec8460f6aa6d1e5ba34874bb3228160555d8a01dbc37620d4873` ## Methodology Sections @@ -99,23 +100,23 @@ This report proves local benchmark reproducibility only. It keeps external provi | output_execution | `reports/output_execution_runs.json` | present | `4df66b63d2e7` | | blind_review | `reports/output_blind_review_pack.json` | present | `bbe2db8ec277` | | review_adjudication | `reports/output_review_adjudication.json` | present | `91fd88dd9b0f` | -| trigger_scorecard | `reports/route_scorecard.json` | present | `a649547a7d3d` | +| trigger_scorecard | `reports/route_scorecard.json` | present | `53fc22d220dc` | | runtime_conformance | `reports/conformance_matrix.json` | present | `97f9ba949c23` | -| trust_report | `reports/security_trust_report.json` | present | `2ef43f009be6` | -| python_compatibility | `reports/python_compatibility.json` | present | `7f3fc8e785bf` | -| registry_audit | `reports/registry_audit.json` | present | `03cb951dc8ba` | +| trust_report | `reports/security_trust_report.json` | present | `ffe2c88bbdba` | +| python_compatibility | `reports/python_compatibility.json` | present | `4d82942052a2` | +| registry_audit | `reports/registry_audit.json` | present | `9be6de364d69` | | package_verification | `reports/package_verification.json` | present | `e2ad726ce048` | -| install_simulation | `reports/install_simulation.json` | present | `6cf792017273` | -| skill_os2_audit | `reports/skill_os2_audit.json` | present | `1079221b5ac7` | -| world_class_evidence_plan | `reports/world_class_evidence_plan.json` | present | `374ee899a0ba` | -| world_class_evidence_ledger | `reports/world_class_evidence_ledger.json` | present | `565ae80a8228` | -| world_class_evidence_intake | `reports/world_class_evidence_intake.json` | present | `e36a7e753f41` | -| world_class_evidence_preflight | `reports/world_class_evidence_preflight.json` | present | `9e4f0bc489ca` | -| world_class_submission_review | `reports/world_class_submission_review.json` | present | `3c944b4bd27f` | -| world_class_operator_runbook | `reports/world_class_operator_runbook.json` | present | `d8678010fcfd` | -| world_class_operator_runbook_markdown | `reports/world_class_operator_runbook.md` | present | `fe3a73bf2eb9` | +| install_simulation | `reports/install_simulation.json` | present | `65adf51d8c4b` | +| skill_os2_audit | `reports/skill_os2_audit.json` | present | `150c0027deaa` | +| world_class_evidence_plan | `reports/world_class_evidence_plan.json` | present | `483ed2b3bd02` | +| world_class_evidence_ledger | `reports/world_class_evidence_ledger.json` | present | `b19e7113ec46` | +| world_class_evidence_intake | `reports/world_class_evidence_intake.json` | present | `885bff9c1c85` | +| world_class_evidence_preflight | `reports/world_class_evidence_preflight.json` | present | `a0b00eed5450` | +| world_class_submission_review | `reports/world_class_submission_review.json` | present | `e8df5c72ade7` | +| world_class_operator_runbook | `reports/world_class_operator_runbook.json` | present | `bb67813244d6` | +| world_class_operator_runbook_markdown | `reports/world_class_operator_runbook.md` | present | `e8d4dee75413` | | world_class_operator_runbook_html | `reports/world_class_operator_runbook.html` | present | `022594013b24` | -| world_class_claim_guard | `reports/world_class_claim_guard.json` | present | `556700f05a65` | +| world_class_claim_guard | `reports/world_class_claim_guard.json` | present | `5eb808496802` | ## Reproduction Commands diff --git a/reports/candidate_registry.json b/reports/candidate_registry.json index 192a53c8..38805181 100644 --- a/reports/candidate_registry.json +++ b/reports/candidate_registry.json @@ -3,9 +3,9 @@ { "name": "yao-meta-skill", "winner_label": "Current", - "winner_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "winner_tokens": 53, - "current_tokens": 53, + "winner_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "winner_tokens": 54, + "current_tokens": 54, "winner_holdout_fp": 0, "winner_holdout_fn": 0, "current_holdout_fp": 0, @@ -36,44 +36,44 @@ "calibration": { "holdout": { "threshold": 0.33, - "mean_positive_score": 0.908, - "mean_non_trigger_score": 0.003, - "mean_near_neighbor_score": 0.006, - "min_positive_score": 0.767, - "max_non_trigger_score": 0.023, - "score_gap": 0.744, - "threshold_margin": 0.307, - "positive_threshold_buffer": 0.437, - "negative_threshold_buffer": 0.307, + "mean_positive_score": 0.704, + "mean_non_trigger_score": 0.0, + "mean_near_neighbor_score": 0.0, + "min_positive_score": 0.465, + "max_non_trigger_score": 0.0, + "score_gap": 0.465, + "threshold_margin": 0.135, + "positive_threshold_buffer": 0.135, + "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, "risk_band": "healthy" }, "blind_holdout": { "threshold": 0.33, - "mean_positive_score": 0.669, - "mean_non_trigger_score": 0.116, - "mean_near_neighbor_score": 0.152, - "min_positive_score": 0.665, - "max_non_trigger_score": 0.305, - "score_gap": 0.36, - "threshold_margin": 0.025, - "positive_threshold_buffer": 0.335, - "negative_threshold_buffer": 0.025, - "boundary_case_count": 1, - "boundary_case_rate": 0.167, - "risk_band": "tight" + "mean_positive_score": 0.377, + "mean_non_trigger_score": 0.024, + "mean_near_neighbor_score": 0.006, + "min_positive_score": 0.374, + "max_non_trigger_score": 0.084, + "score_gap": 0.29, + "threshold_margin": 0.044, + "positive_threshold_buffer": 0.044, + "negative_threshold_buffer": 0.246, + "boundary_case_count": 2, + "boundary_case_rate": 0.333, + "risk_band": "watch" }, "adversarial_holdout": { "threshold": 0.33, - "mean_positive_score": 1.0, + "mean_positive_score": 0.81, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, - "min_positive_score": 1.0, + "min_positive_score": 0.808, "max_non_trigger_score": 0.0, - "score_gap": 1.0, + "score_gap": 0.808, "threshold_margin": 0.33, - "positive_threshold_buffer": 0.67, + "positive_threshold_buffer": 0.478, "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, @@ -164,7 +164,7 @@ "route_metrics": { "precision": 1.0, "recall": 1.0, - "average_margin": 0.701, + "average_margin": 0.644, "misroute_count": 0, "ambiguous_case_count": 0 } @@ -187,8 +187,8 @@ "label": "Current", "ranking_state": "selected_by_dev", "promotion_state": "kept_current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "tokens": 53, + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "tokens": 54, "dev_errors": 0, "holdout_errors": 0, "reason_tags": [] @@ -696,7 +696,7 @@ "route_metrics": { "precision": 1.0, "recall": 1.0, - "average_margin": 0.586, + "average_margin": 0.709, "misroute_count": 0, "ambiguous_case_count": 0 } diff --git a/reports/candidate_registry.md b/reports/candidate_registry.md index d6eae6ea..fe35c95a 100644 --- a/reports/candidate_registry.md +++ b/reports/candidate_registry.md @@ -3,7 +3,7 @@ | Target | Role | Label | Ranking State | Promotion State | Tokens | Dev Errors | Holdout Errors | Reason Tags | | --- | --- | --- | --- | --- | ---: | ---: | ---: | --- | | `yao-meta-skill` | baseline | `Baseline` | reference | reference | 8 | 1 | 0 | - | -| `yao-meta-skill` | current | `Current` | selected_by_dev | kept_current | 53 | 0 | 0 | - | +| `yao-meta-skill` | current | `Current` | selected_by_dev | kept_current | 54 | 0 | 0 | - | | `yao-meta-skill` | candidate | `Minimal` | not_selected | blocked | 41 | 2 | 0 | weaker_dev_fit | | `yao-meta-skill` | candidate | `Guardrail` | not_selected | blocked | 56 | 2 | 0 | weaker_dev_fit, longer_without_gain | | `yao-meta-skill` | candidate | `Balanced` | not_selected | blocked | 60 | 2 | 0 | weaker_dev_fit, longer_without_gain | diff --git a/reports/compiled_targets.json b/reports/compiled_targets.json index 7eb7eaac..54a08252 100644 --- a/reports/compiled_targets.json +++ b/reports/compiled_targets.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-06-13", "skill_dir": ".", "summary": { "target_count": 5, @@ -38,10 +38,10 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", @@ -623,7 +623,7 @@ "native_surface": "OpenAI-style interface metadata plus neutral Agent Skills source", "activation": { "policy": "Use frontmatter description for catalog routing and targets/openai/agents/openai.yaml for display name, default prompt, and compatibility metadata.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -881,7 +881,7 @@ "native_surface": "OpenAI-style interface metadata plus neutral Agent Skills source", "activation": { "policy": "Use frontmatter description for catalog routing and targets/openai/agents/openai.yaml for display name, default prompt, and compatibility metadata.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -995,10 +995,10 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", @@ -1580,7 +1580,7 @@ "native_surface": "Claude-compatible neutral source folder with adapter notes", "activation": { "policy": "Use SKILL.md frontmatter description as the primary activation contract and adapter.json for review metadata.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -1838,7 +1838,7 @@ "native_surface": "Claude-compatible neutral source folder with adapter notes", "activation": { "policy": "Use SKILL.md frontmatter description as the primary activation contract and adapter.json for review metadata.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -1952,10 +1952,10 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", @@ -2537,7 +2537,7 @@ "native_surface": "Agent Skills compatible neutral package", "activation": { "policy": "Use SKILL.md name and description; consumers decide automatic or manual activation.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -2788,7 +2788,7 @@ "native_surface": "Agent Skills compatible neutral package", "activation": { "policy": "Use SKILL.md name and description; consumers decide automatic or manual activation.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -2893,10 +2893,10 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", @@ -3478,7 +3478,7 @@ "native_surface": "Agent Skills standard source tree", "activation": { "policy": "Use SKILL.md frontmatter name and description for progressive disclosure.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -3729,7 +3729,7 @@ "native_surface": "Agent Skills standard source tree", "activation": { "policy": "Use SKILL.md frontmatter name and description for progressive disclosure.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -3834,10 +3834,10 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", @@ -4419,7 +4419,7 @@ "native_surface": "VS Code/Copilot Agent Skills project or user scope", "activation": { "policy": "Use folder name plus SKILL.md name/description; keep description under platform limits.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -4674,7 +4674,7 @@ "native_surface": "VS Code/Copilot Agent Skills project or user scope", "activation": { "policy": "Use folder name plus SKILL.md name/description; keep description under platform limits.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, diff --git a/reports/context_budget.json b/reports/context_budget.json index 40a03442..8a71ea1a 100644 --- a/reports/context_budget.json +++ b/reports/context_budget.json @@ -5,11 +5,11 @@ "stats": { "context_budget_tier": "production", "context_budget_limit": 1000, - "skill_body_tokens": 805, - "other_text_tokens": 1169049, - "estimated_initial_load_tokens": 998, - "estimated_total_text_tokens": 1169854, - "deferred_resource_tokens": 530871, + "skill_body_tokens": 806, + "other_text_tokens": 1200438, + "estimated_initial_load_tokens": 999, + "estimated_total_text_tokens": 1201244, + "deferred_resource_tokens": 531767, "deferred_resource_warn_threshold": 120000, "deferred_resource_dirs": [ { @@ -24,7 +24,7 @@ }, { "path": "evals", - "estimated_tokens": 21835, + "estimated_tokens": 22731, "file_count": 27 }, { @@ -66,9 +66,9 @@ ], "summary": "Large deferred resources are indexed and backed by evidence." }, - "relevant_file_count": 571, + "relevant_file_count": 590, "unused_resource_dirs": [], "quality_signal_points": 140, - "quality_density": 140.3 + "quality_density": 140.1 } } diff --git a/reports/context_budget.md b/reports/context_budget.md index 87bfecaa..66d0f511 100644 --- a/reports/context_budget.md +++ b/reports/context_budget.md @@ -2,7 +2,7 @@ | Target | Path | Tier | Limit | Initial | SKILL | Deferred | Resource Governance | Large Deferred Dirs | Quality Density | Unused Dirs | Status | | --- | --- | --- | ---: | ---: | ---: | ---: | --- | --- | ---: | --- | --- | -| root | `.` | `production` | 1000 | 998 | 805 | 530871 | `governed` | scripts:469568 | 140.3 | - | ok | +| root | `.` | `production` | 1000 | 999 | 806 | 531767 | `governed` | scripts:469568 | 140.1 | - | ok | | complex-release-orchestrator | `examples/complex-release-orchestrator/generated-skill` | `production` | 1000 | 790 | 718 | 1657 | `not-required` | - | 164.6 | - | ok | | governed-incident-command | `examples/governed-incident-command/generated-skill` | `production` | 1000 | 760 | 658 | 1030 | `not-required` | - | 171.1 | - | ok | diff --git a/reports/context_budget_summary.json b/reports/context_budget_summary.json index 95c5a2e5..877503be 100644 --- a/reports/context_budget_summary.json +++ b/reports/context_budget_summary.json @@ -1,14 +1,14 @@ { - "generated_at": "2026-06-21", + "generated_at": "2026-03-31", "targets": [ { "label": "root", "path": ".", "budget_tier": "production", "budget_limit": 1000, - "initial_tokens": 998, - "skill_body_tokens": 805, - "deferred_resource_tokens": 530871, + "initial_tokens": 999, + "skill_body_tokens": 806, + "deferred_resource_tokens": 531767, "large_deferred_resource_dirs": [ { "path": "scripts", @@ -42,7 +42,7 @@ ], "summary": "Large deferred resources are indexed and backed by evidence." }, - "quality_density": 140.3, + "quality_density": 140.1, "unused_resource_dirs": [], "ok": true, "warnings": [] diff --git a/reports/description_optimization.json b/reports/description_optimization.json index 08f61621..410954df 100644 --- a/reports/description_optimization.json +++ b/reports/description_optimization.json @@ -1,11 +1,11 @@ { - "current_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "current_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "current_candidate": { "id": "current", "label": "Current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "strategy": "current", - "estimated_tokens": 53, + "estimated_tokens": 54, "dev": { "false_positives": 0, "false_negatives": 0, @@ -21,7 +21,7 @@ -1.0, -1.0, -1.0, - 53 + 54 ], "holdout": { "false_positives": 0, @@ -326,7 +326,8 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ "runbook", @@ -379,7 +380,8 @@ ], "transform_workflow": [ "workflow", - "workflow fragments" + "workflow fragments", + "scripts" ] }, "negative": {} @@ -603,7 +605,8 @@ "transform_workflow": [ "rough notes", "transcript", - "onboarding flow" + "onboarding flow", + "scripts" ] }, "negative": {} @@ -1486,7 +1489,8 @@ ], "transform_workflow": [ "process", - "internal process note" + "internal process note", + "scripts" ] }, "negative": {} @@ -1553,7 +1557,7 @@ { "prompt": "Turn this recurring onboarding process into a reusable skill library entry with validation notes.", "family": "workflow_to_skill", - "score": 0.546, + "score": 0.841, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -1567,6 +1571,7 @@ "iterate_existing_skill" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" @@ -1574,13 +1579,16 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.528, + "semantic_coverage": 0.849, "support_score": 0, "lexical_support": 0.0, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { + "build_skill": [ + "skill library entry" + ], "eval_optimize": [ "validation" ], @@ -1643,10 +1651,12 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ - "quarterly release routine" + "quarterly release routine", + "scripts" ] }, "negative": {} @@ -1687,17 +1697,20 @@ "concept_evidence": { "positive": { "build_skill": [ - "turn this playbook into an agent capability package" + "turn this playbook into an agent capability package", + "agent capability" ], "reuse_package": [ "package", - "team library" + "team library", + "release" ], "team_operationalize": [ "team" ], "transform_workflow": [ "prompt history", + "playbook", "transcript", "release notes" ] @@ -1800,7 +1813,8 @@ "concept_evidence": { "positive": { "reuse_package": [ - "package" + "package", + "release" ], "transform_workflow": [ "checklist" @@ -1835,6 +1849,7 @@ "iterate_existing_skill" ], "matched_desired_concepts": [ + "reuse_package", "transform_workflow" ], "extra_positive_concepts": [], @@ -1845,13 +1860,16 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.189, + "semantic_coverage": 0.377, "support_score": 0, "lexical_support": 0.111, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { + "reuse_package": [ + "release" + ], "transform_workflow": [ "workflow" ] @@ -2148,9 +2166,9 @@ "winner": { "id": "current", "label": "Current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "strategy": "current", - "estimated_tokens": 53, + "estimated_tokens": 54, "dev": { "false_positives": 0, "false_negatives": 0, @@ -2166,7 +2184,7 @@ -1.0, -1.0, -1.0, - 53 + 54 ], "holdout": { "false_positives": 0, @@ -2321,17 +2339,19 @@ { "prompt": "Convert this operations checklist into a reusable skill.", "family": "workflow_to_skill", - "score": 0.673, + "score": 0.379, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -2340,9 +2360,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.25, + "lexical_support": 0.125, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -2357,34 +2377,37 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Add trigger evals to this skill before sharing it with the team.", "family": "iterate_existing_skill", - "score": 0.407, + "score": 0.333, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", - "team_operationalize" + "iterate_existing_skill" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.333, - "support_score": 0.16, + "semantic_coverage": 0.286, + "support_score": 0.04, "lexical_support": 0.25, "negative_penalty": 0, "coverage_boost": 0.04, @@ -2412,31 +2435,32 @@ { "prompt": "We have a messy release runbook, export process, and a prompt history; turn all of that into one reusable skill package with evals and packaging checks.", "family": "complex_multi_asset", - "score": 1.0, + "score": 0.816, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.25, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.208, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -2452,7 +2476,8 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ "runbook", @@ -2467,32 +2492,33 @@ { "prompt": "Turn these workflow fragments, transcripts, and prompts into one production-ready skill with scripts and references.", "family": "complex_multi_asset", - "score": 0.7, + "score": 0.454, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, - "support_score": 0.06, - "lexical_support": 0.4, + "semantic_coverage": 0.411, + "support_score": 0, + "lexical_support": 0.2, "negative_penalty": 0, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "multi_asset": [ @@ -2504,7 +2530,8 @@ ], "transform_workflow": [ "workflow", - "workflow fragments" + "workflow fragments", + "scripts" ] }, "negative": {} @@ -2514,31 +2541,33 @@ { "prompt": "Tighten the trigger boundary on this existing skill and prepare it for team distribution.", "family": "iterate_existing_skill", - "score": 0.742, + "score": 0.514, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", - "reuse_package", - "team_operationalize" + "iterate_existing_skill", + "reuse_package" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, - "support_score": 0.16, - "lexical_support": 0.357, + "semantic_coverage": 0.464, + "support_score": 0.04, + "lexical_support": 0.214, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -2564,31 +2593,34 @@ { "prompt": "Turn the SOP, transcript, and cleanup notes into a maintained skill for the team library.", "family": "paraphrase_trigger", - "score": 0.763, + "score": 0.39, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0, - "lexical_support": 0.357, + "semantic_coverage": 0.357, + "support_score": 0.04, + "lexical_support": 0.143, "negative_penalty": 0, - "coverage_boost": 0.06, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "reuse_package": [ @@ -2605,37 +2637,40 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Codify this workflow into a shared capability package and validate the route boundary before distribution.", "family": "paraphrase_trigger", - "score": 1.0, + "score": 0.815, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.34, - "lexical_support": 0.267, + "semantic_coverage": 0.804, + "support_score": 0.04, + "lexical_support": 0.067, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -2667,33 +2702,35 @@ { "prompt": "After the long handoff notes below, build a skill package for team reuse. The materials include a transcript, rough notes, and an onboarding flow, and the result should include references, scripts, and validation.", "family": "long_context_trigger", - "score": 1.0, + "score": 0.871, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill", - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.4, - "lexical_support": 0.231, + "semantic_coverage": 0.857, + "support_score": 0.04, + "lexical_support": 0.154, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -2721,7 +2758,8 @@ "transform_workflow": [ "rough notes", "transcript", - "onboarding flow" + "onboarding flow", + "scripts" ] }, "negative": {} @@ -2740,10 +2778,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -2755,7 +2795,7 @@ "exclusive_negative_concepts": [ "brainstorm_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0.24, @@ -2784,10 +2824,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -2799,7 +2841,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.083, + "lexical_support": 0.0, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -2823,17 +2865,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [ - "reuse_package" - ], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset", + "reuse_package" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "brainstorm_only", "no_build_directive" @@ -2842,11 +2885,11 @@ "brainstorm_only", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.06, + "semantic_coverage": 0.232, + "support_score": 0, "lexical_support": 0.133, "negative_penalty": 0.66, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "multi_asset": [ @@ -2878,10 +2921,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -2893,7 +2938,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.167, + "lexical_support": 0.0, "negative_penalty": 0.35, "coverage_boost": 0.0, "concept_evidence": { @@ -2917,10 +2962,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -2932,9 +2979,9 @@ "exclusive_negative_concepts": [ "summarize_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.222, + "lexical_support": 0.0, "negative_penalty": 0.3, "coverage_boost": 0.0, "concept_evidence": { @@ -2961,10 +3008,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package" @@ -2978,9 +3027,9 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.087, + "lexical_support": 0.0, "negative_penalty": 0.68, "coverage_boost": 0.0, "concept_evidence": { @@ -3005,36 +3054,37 @@ { "prompt": "I pasted a very long process note. For now I only need a recap and a short summary, not a skill package.", "family": "long_context_summary_only", - "score": 0.306, + "score": 0.236, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "summarize_only" ], "exclusive_negative_concepts": [ "summarize_only" ], - "semantic_coverage": 0.667, - "support_score": 0.34, + "semantic_coverage": 0.661, + "support_score": 0, "lexical_support": 0.222, "negative_penalty": 0.3, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -3054,8 +3104,7 @@ ] } } - }, - "boundary_case": true + } } ], "near_neighbor": [ @@ -3069,10 +3118,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -3087,9 +3138,9 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.1, "negative_penalty": 0.68, "coverage_boost": 0.04, "concept_evidence": { @@ -3122,10 +3173,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -3137,9 +3190,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.143, + "lexical_support": 0.071, "negative_penalty": 0.42, "coverage_boost": 0.0, "concept_evidence": { @@ -3166,17 +3219,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -3184,11 +3238,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, + "semantic_coverage": 0.482, + "support_score": 0, "lexical_support": 0.3, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -3219,17 +3273,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "iterate_existing_skill", "reuse_package" ], - "extra_positive_concepts": [ - "iterate_existing_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline", "no_build_directive" @@ -3238,11 +3293,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.16, - "lexical_support": 0.267, + "semantic_coverage": 0.321, + "support_score": 0, + "lexical_support": 0.2, "negative_penalty": 0.64, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "iterate_existing_skill": [ @@ -3273,17 +3328,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -3291,11 +3347,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.286, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.143, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -3327,17 +3383,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "future_outline", @@ -3347,11 +3404,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.19, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.048, "negative_penalty": 0.82, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -3390,17 +3447,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline", "no_build_directive" @@ -3409,11 +3467,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.263, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.158, "negative_penalty": 0.64, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -3554,30 +3612,32 @@ { "prompt": "Build a reusable skill from this long internal process note, and make sure it has references, scripts, and a trigger description.", "family": "complex_multi_asset", - "score": 0.767, + "score": 0.465, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0.06, + "semantic_coverage": 0.411, + "support_score": 0.04, "lexical_support": 0.211, "negative_penalty": 0, "coverage_boost": 0.06, @@ -3595,7 +3655,8 @@ ], "transform_workflow": [ "process", - "internal process note" + "internal process note", + "scripts" ] }, "negative": {} @@ -3605,32 +3666,34 @@ { "prompt": "Take this existing skill draft, tighten the trigger boundary, add near-neighbor evals, and package it for distribution.", "family": "iterate_existing_skill", - "score": 1.0, + "score": 0.684, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", + "iterate_existing_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.16, - "lexical_support": 0.333, + "semantic_coverage": 0.643, + "support_score": 0.04, + "lexical_support": 0.278, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -3661,19 +3724,22 @@ { "prompt": "Turn this recurring onboarding process into a reusable skill library entry with validation notes.", "family": "workflow_to_skill", - "score": 0.936, + "score": 0.805, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" @@ -3681,13 +3747,16 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, + "semantic_coverage": 0.804, "support_score": 0, - "lexical_support": 0.214, + "lexical_support": 0.071, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { + "build_skill": [ + "skill library entry" + ], "eval_optimize": [ "validation" ], @@ -3706,31 +3775,32 @@ { "prompt": "Codify this quarterly release routine into a reusable skill package with scripts, references, and packaging checks.", "family": "paraphrase_trigger", - "score": 1.0, + "score": 0.874, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill", - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.4, + "semantic_coverage": 0.857, + "support_score": 0, "lexical_support": 0.312, "negative_penalty": 0, "coverage_boost": 0.06, @@ -3749,10 +3819,12 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ - "quarterly release routine" + "quarterly release routine", + "scripts" ] }, "negative": {} @@ -3762,47 +3834,52 @@ { "prompt": "Below is a noisy transcript plus release notes and prompt history. Ignore the chatter and turn this playbook into an agent capability package for the team library.", "family": "long_context_trigger", - "score": 0.839, + "score": 0.691, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0.34, - "lexical_support": 0.24, + "semantic_coverage": 0.661, + "support_score": 0.04, + "lexical_support": 0.16, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ - "turn this playbook into an agent capability package" + "turn this playbook into an agent capability package", + "agent capability" ], "reuse_package": [ "package", - "team library" + "team library", + "release" ], "team_operationalize": [ "team" ], "transform_workflow": [ "prompt history", + "playbook", "transcript", "release notes" ] @@ -3823,10 +3900,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -3841,7 +3920,7 @@ "no_build_directive", "summarize_only" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0.72, @@ -3877,10 +3956,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -3895,15 +3976,16 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.091, + "lexical_support": 0.182, "negative_penalty": 0.68, "coverage_boost": 0.04, "concept_evidence": { "positive": { "reuse_package": [ - "package" + "package", + "release" ], "transform_workflow": [ "checklist" @@ -3931,12 +4013,15 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "reuse_package", "transform_workflow" ], "extra_positive_concepts": [], @@ -3947,13 +4032,16 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.222, + "lexical_support": 0.167, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { + "reuse_package": [ + "release" + ], "transform_workflow": [ "workflow" ] @@ -3981,18 +4069,19 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "no_build_directive", "translate_only" @@ -4001,11 +4090,11 @@ "no_build_directive", "translate_only" ], - "semantic_coverage": 0.667, - "support_score": 0.34, + "semantic_coverage": 0.661, + "support_score": 0, "lexical_support": 0.188, "negative_penalty": 0.77, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -4043,15 +4132,17 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], - "matched_desired_concepts": [], - "extra_positive_concepts": [ + "matched_desired_concepts": [ "build_skill" ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "brainstorm_only", "future_outline" @@ -4060,8 +4151,8 @@ "brainstorm_only", "future_outline" ], - "semantic_coverage": 0.0, - "support_score": 0.34, + "semantic_coverage": 0.304, + "support_score": 0, "lexical_support": 0.077, "negative_penalty": 0.46, "coverage_boost": 0.0, @@ -4092,10 +4183,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -4137,17 +4230,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -4155,11 +4249,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, + "semantic_coverage": 0.482, + "support_score": 0, "lexical_support": 0.308, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -4185,25 +4279,26 @@ { "prompt": "From the long workflow below, create a template only and a checklist for discussion only. No skill package.", "family": "long_context_near_neighbor", - "score": 0.023, + "score": 0.0, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -4211,11 +4306,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.667, - "support_score": 0.34, - "lexical_support": 0.438, + "semantic_coverage": 0.661, + "support_score": 0, + "lexical_support": 0.375, "negative_penalty": 0.6, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -4388,17 +4483,19 @@ { "prompt": "Convert this operations checklist into a reusable skill.", "family": "workflow_to_skill", - "score": 0.673, + "score": 0.379, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -4407,9 +4504,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.25, + "lexical_support": 0.125, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -4424,34 +4521,37 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Add trigger evals to this skill before sharing it with the team.", "family": "iterate_existing_skill", - "score": 0.407, + "score": 0.333, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", - "team_operationalize" + "iterate_existing_skill" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.333, - "support_score": 0.16, + "semantic_coverage": 0.286, + "support_score": 0.04, "lexical_support": 0.25, "negative_penalty": 0, "coverage_boost": 0.04, @@ -4479,31 +4579,32 @@ { "prompt": "We have a messy release runbook, export process, and a prompt history; turn all of that into one reusable skill package with evals and packaging checks.", "family": "complex_multi_asset", - "score": 1.0, + "score": 0.816, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.25, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.208, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -4519,7 +4620,8 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ "runbook", @@ -4534,32 +4636,33 @@ { "prompt": "Turn these workflow fragments, transcripts, and prompts into one production-ready skill with scripts and references.", "family": "complex_multi_asset", - "score": 0.7, + "score": 0.454, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, - "support_score": 0.06, - "lexical_support": 0.4, + "semantic_coverage": 0.411, + "support_score": 0, + "lexical_support": 0.2, "negative_penalty": 0, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "multi_asset": [ @@ -4571,7 +4674,8 @@ ], "transform_workflow": [ "workflow", - "workflow fragments" + "workflow fragments", + "scripts" ] }, "negative": {} @@ -4581,31 +4685,33 @@ { "prompt": "Tighten the trigger boundary on this existing skill and prepare it for team distribution.", "family": "iterate_existing_skill", - "score": 0.742, + "score": 0.514, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", - "reuse_package", - "team_operationalize" + "iterate_existing_skill", + "reuse_package" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, - "support_score": 0.16, - "lexical_support": 0.357, + "semantic_coverage": 0.464, + "support_score": 0.04, + "lexical_support": 0.214, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -4631,31 +4737,34 @@ { "prompt": "Turn the SOP, transcript, and cleanup notes into a maintained skill for the team library.", "family": "paraphrase_trigger", - "score": 0.763, + "score": 0.39, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0, - "lexical_support": 0.357, + "semantic_coverage": 0.357, + "support_score": 0.04, + "lexical_support": 0.143, "negative_penalty": 0, - "coverage_boost": 0.06, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "reuse_package": [ @@ -4672,37 +4781,40 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Codify this workflow into a shared capability package and validate the route boundary before distribution.", "family": "paraphrase_trigger", - "score": 1.0, + "score": 0.815, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.34, - "lexical_support": 0.267, + "semantic_coverage": 0.804, + "support_score": 0.04, + "lexical_support": 0.067, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -4734,33 +4846,35 @@ { "prompt": "After the long handoff notes below, build a skill package for team reuse. The materials include a transcript, rough notes, and an onboarding flow, and the result should include references, scripts, and validation.", "family": "long_context_trigger", - "score": 1.0, + "score": 0.871, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill", - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.4, - "lexical_support": 0.231, + "semantic_coverage": 0.857, + "support_score": 0.04, + "lexical_support": 0.154, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -4788,7 +4902,8 @@ "transform_workflow": [ "rough notes", "transcript", - "onboarding flow" + "onboarding flow", + "scripts" ] }, "negative": {} @@ -4807,10 +4922,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -4822,7 +4939,7 @@ "exclusive_negative_concepts": [ "brainstorm_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0.24, @@ -4851,10 +4968,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -4866,7 +4985,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.083, + "lexical_support": 0.0, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -4890,17 +5009,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [ - "reuse_package" - ], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset", + "reuse_package" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "brainstorm_only", "no_build_directive" @@ -4909,11 +5029,11 @@ "brainstorm_only", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.06, + "semantic_coverage": 0.232, + "support_score": 0, "lexical_support": 0.133, "negative_penalty": 0.66, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "multi_asset": [ @@ -4945,10 +5065,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -4960,7 +5082,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.167, + "lexical_support": 0.0, "negative_penalty": 0.35, "coverage_boost": 0.0, "concept_evidence": { @@ -4984,10 +5106,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -4999,9 +5123,9 @@ "exclusive_negative_concepts": [ "summarize_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.222, + "lexical_support": 0.0, "negative_penalty": 0.3, "coverage_boost": 0.0, "concept_evidence": { @@ -5028,10 +5152,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package" @@ -5045,9 +5171,9 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.087, + "lexical_support": 0.0, "negative_penalty": 0.68, "coverage_boost": 0.0, "concept_evidence": { @@ -5072,36 +5198,37 @@ { "prompt": "I pasted a very long process note. For now I only need a recap and a short summary, not a skill package.", "family": "long_context_summary_only", - "score": 0.306, + "score": 0.236, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "summarize_only" ], "exclusive_negative_concepts": [ "summarize_only" ], - "semantic_coverage": 0.667, - "support_score": 0.34, + "semantic_coverage": 0.661, + "support_score": 0, "lexical_support": 0.222, "negative_penalty": 0.3, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -5121,8 +5248,7 @@ ] } } - }, - "boundary_case": true + } } ], "near_neighbor": [ @@ -5136,10 +5262,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -5154,9 +5282,9 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.1, "negative_penalty": 0.68, "coverage_boost": 0.04, "concept_evidence": { @@ -5189,10 +5317,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -5204,9 +5334,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.143, + "lexical_support": 0.071, "negative_penalty": 0.42, "coverage_boost": 0.0, "concept_evidence": { @@ -5233,17 +5363,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -5251,11 +5382,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, + "semantic_coverage": 0.482, + "support_score": 0, "lexical_support": 0.3, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -5286,17 +5417,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "iterate_existing_skill", "reuse_package" ], - "extra_positive_concepts": [ - "iterate_existing_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline", "no_build_directive" @@ -5305,11 +5437,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.16, - "lexical_support": 0.267, + "semantic_coverage": 0.321, + "support_score": 0, + "lexical_support": 0.2, "negative_penalty": 0.64, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "iterate_existing_skill": [ @@ -5340,17 +5472,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -5358,11 +5491,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.286, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.143, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -5394,17 +5527,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "future_outline", @@ -5414,11 +5548,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.19, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.048, "negative_penalty": 0.82, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -5457,17 +5591,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline", "no_build_directive" @@ -5476,11 +5611,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.263, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.158, "negative_penalty": 0.64, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -5621,30 +5756,32 @@ { "prompt": "Build a reusable skill from this long internal process note, and make sure it has references, scripts, and a trigger description.", "family": "complex_multi_asset", - "score": 0.767, + "score": 0.465, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0.06, + "semantic_coverage": 0.411, + "support_score": 0.04, "lexical_support": 0.211, "negative_penalty": 0, "coverage_boost": 0.06, @@ -5662,7 +5799,8 @@ ], "transform_workflow": [ "process", - "internal process note" + "internal process note", + "scripts" ] }, "negative": {} @@ -5672,32 +5810,34 @@ { "prompt": "Take this existing skill draft, tighten the trigger boundary, add near-neighbor evals, and package it for distribution.", "family": "iterate_existing_skill", - "score": 1.0, + "score": 0.684, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", + "iterate_existing_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.16, - "lexical_support": 0.333, + "semantic_coverage": 0.643, + "support_score": 0.04, + "lexical_support": 0.278, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -5728,19 +5868,22 @@ { "prompt": "Turn this recurring onboarding process into a reusable skill library entry with validation notes.", "family": "workflow_to_skill", - "score": 0.936, + "score": 0.805, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" @@ -5748,13 +5891,16 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, + "semantic_coverage": 0.804, "support_score": 0, - "lexical_support": 0.214, + "lexical_support": 0.071, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { + "build_skill": [ + "skill library entry" + ], "eval_optimize": [ "validation" ], @@ -5773,31 +5919,32 @@ { "prompt": "Codify this quarterly release routine into a reusable skill package with scripts, references, and packaging checks.", "family": "paraphrase_trigger", - "score": 1.0, + "score": 0.874, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill", - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.4, + "semantic_coverage": 0.857, + "support_score": 0, "lexical_support": 0.312, "negative_penalty": 0, "coverage_boost": 0.06, @@ -5816,10 +5963,12 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ - "quarterly release routine" + "quarterly release routine", + "scripts" ] }, "negative": {} @@ -5829,47 +5978,52 @@ { "prompt": "Below is a noisy transcript plus release notes and prompt history. Ignore the chatter and turn this playbook into an agent capability package for the team library.", "family": "long_context_trigger", - "score": 0.839, + "score": 0.691, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0.34, - "lexical_support": 0.24, + "semantic_coverage": 0.661, + "support_score": 0.04, + "lexical_support": 0.16, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ - "turn this playbook into an agent capability package" + "turn this playbook into an agent capability package", + "agent capability" ], "reuse_package": [ "package", - "team library" + "team library", + "release" ], "team_operationalize": [ "team" ], "transform_workflow": [ "prompt history", + "playbook", "transcript", "release notes" ] @@ -5890,10 +6044,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -5908,7 +6064,7 @@ "no_build_directive", "summarize_only" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0.72, @@ -5944,10 +6100,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -5962,15 +6120,16 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.091, + "lexical_support": 0.182, "negative_penalty": 0.68, "coverage_boost": 0.04, "concept_evidence": { "positive": { "reuse_package": [ - "package" + "package", + "release" ], "transform_workflow": [ "checklist" @@ -5998,12 +6157,15 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "reuse_package", "transform_workflow" ], "extra_positive_concepts": [], @@ -6014,13 +6176,16 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.222, + "lexical_support": 0.167, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { + "reuse_package": [ + "release" + ], "transform_workflow": [ "workflow" ] @@ -6048,18 +6213,19 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "no_build_directive", "translate_only" @@ -6068,11 +6234,11 @@ "no_build_directive", "translate_only" ], - "semantic_coverage": 0.667, - "support_score": 0.34, + "semantic_coverage": 0.661, + "support_score": 0, "lexical_support": 0.188, "negative_penalty": 0.77, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -6110,15 +6276,17 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], - "matched_desired_concepts": [], - "extra_positive_concepts": [ + "matched_desired_concepts": [ "build_skill" ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "brainstorm_only", "future_outline" @@ -6127,8 +6295,8 @@ "brainstorm_only", "future_outline" ], - "semantic_coverage": 0.0, - "support_score": 0.34, + "semantic_coverage": 0.304, + "support_score": 0, "lexical_support": 0.077, "negative_penalty": 0.46, "coverage_boost": 0.0, @@ -6159,10 +6327,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -6204,17 +6374,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -6222,11 +6393,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, + "semantic_coverage": 0.482, + "support_score": 0, "lexical_support": 0.308, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -6252,25 +6423,26 @@ { "prompt": "From the long workflow below, create a template only and a checklist for discussion only. No skill package.", "family": "long_context_near_neighbor", - "score": 0.023, + "score": 0.0, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -6278,11 +6450,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.667, - "support_score": 0.34, - "lexical_support": 0.438, + "semantic_coverage": 0.661, + "support_score": 0, + "lexical_support": 0.375, "negative_penalty": 0.6, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -6385,17 +6557,19 @@ { "prompt": "Operationalize this support escalation routine as a maintained skill and keep trigger eval coverage.", "family": "blind_operationalize", - "score": 0.665, + "score": 0.38, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -6404,7 +6578,7 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0, @@ -6421,22 +6595,25 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Take these SOP scraps, prompt notes, and transcript excerpts and formalize them into a reusable capability package.", "family": "blind_multi_asset", - "score": 0.673, + "score": 0.374, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -6445,9 +6622,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.25, + "lexical_support": 0.062, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -6463,40 +6640,42 @@ }, "negative": {} } - } + }, + "boundary_case": true } ], "should_not_trigger": [ { "prompt": "Turn this workflow note into a wiki reference only, do not create any skill files, and keep it out of agent execution.", "family": "blind_document_only", - "score": 0.16, + "score": 0.084, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [ - "transform_workflow" - ], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset", + "transform_workflow" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only" ], "exclusive_negative_concepts": [], - "semantic_coverage": 0.333, - "support_score": 0.06, - "lexical_support": 0.227, + "semantic_coverage": 0.232, + "support_score": 0, + "lexical_support": 0.136, "negative_penalty": 0.18, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "multi_asset": [ @@ -6524,10 +6703,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -6539,7 +6720,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.3, + "lexical_support": 0.2, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -6564,24 +6745,26 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline" ], "exclusive_negative_concepts": [ "future_outline" ], - "semantic_coverage": 0.0, - "support_score": 0.06, - "lexical_support": 0.231, + "semantic_coverage": 0.054, + "support_score": 0, + "lexical_support": 0.154, "negative_penalty": 0.22, "coverage_boost": 0.0, "concept_evidence": { @@ -6601,17 +6784,19 @@ { "prompt": "Review this process and suggest improvements for later; no packaging yet.", "family": "blind_brainstorm_only", - "score": 0.305, + "score": 0.013, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -6624,9 +6809,9 @@ "exclusive_negative_concepts": [ "future_outline" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.273, + "lexical_support": 0.182, "negative_penalty": 0.22, "coverage_boost": 0.04, "concept_evidence": { @@ -6644,8 +6829,7 @@ ] } } - }, - "boundary_case": true + } } ] } @@ -6724,17 +6908,19 @@ { "prompt": "Operationalize this support escalation routine as a maintained skill and keep trigger eval coverage.", "family": "blind_operationalize", - "score": 0.665, + "score": 0.38, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -6743,7 +6929,7 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0, @@ -6760,22 +6946,25 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Take these SOP scraps, prompt notes, and transcript excerpts and formalize them into a reusable capability package.", "family": "blind_multi_asset", - "score": 0.673, + "score": 0.374, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -6784,9 +6973,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.25, + "lexical_support": 0.062, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -6802,40 +6991,42 @@ }, "negative": {} } - } + }, + "boundary_case": true } ], "should_not_trigger": [ { "prompt": "Turn this workflow note into a wiki reference only, do not create any skill files, and keep it out of agent execution.", "family": "blind_document_only", - "score": 0.16, + "score": 0.084, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [ - "transform_workflow" - ], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset", + "transform_workflow" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only" ], "exclusive_negative_concepts": [], - "semantic_coverage": 0.333, - "support_score": 0.06, - "lexical_support": 0.227, + "semantic_coverage": 0.232, + "support_score": 0, + "lexical_support": 0.136, "negative_penalty": 0.18, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "multi_asset": [ @@ -6863,10 +7054,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -6878,7 +7071,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.3, + "lexical_support": 0.2, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -6903,24 +7096,26 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline" ], "exclusive_negative_concepts": [ "future_outline" ], - "semantic_coverage": 0.0, - "support_score": 0.06, - "lexical_support": 0.231, + "semantic_coverage": 0.054, + "support_score": 0, + "lexical_support": 0.154, "negative_penalty": 0.22, "coverage_boost": 0.0, "concept_evidence": { @@ -6940,17 +7135,19 @@ { "prompt": "Review this process and suggest improvements for later; no packaging yet.", "family": "blind_brainstorm_only", - "score": 0.305, + "score": 0.013, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -6963,9 +7160,9 @@ "exclusive_negative_concepts": [ "future_outline" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.273, + "lexical_support": 0.182, "negative_penalty": 0.22, "coverage_boost": 0.04, "concept_evidence": { @@ -6983,8 +7180,7 @@ ] } } - }, - "boundary_case": true + } } ] } @@ -7504,7 +7700,8 @@ "judge_detail": { "mode": "judge-rubric", "focused_positive_concepts": [ - "transform_workflow" + "transform_workflow", + "multi_asset" ], "strong_positive_concepts": [ "transform_workflow" @@ -7593,7 +7790,9 @@ "passed": true, "judge_detail": { "mode": "judge-rubric", - "focused_positive_concepts": [], + "focused_positive_concepts": [ + "multi_asset" + ], "strong_positive_concepts": [], "trigger_action_hits": [], "input_hits": [ @@ -7851,7 +8050,8 @@ "judge_detail": { "mode": "judge-rubric", "focused_positive_concepts": [ - "transform_workflow" + "transform_workflow", + "multi_asset" ], "strong_positive_concepts": [ "transform_workflow" @@ -7940,7 +8140,9 @@ "passed": true, "judge_detail": { "mode": "judge-rubric", - "focused_positive_concepts": [], + "focused_positive_concepts": [ + "multi_asset" + ], "strong_positive_concepts": [], "trigger_action_hits": [], "input_hits": [ @@ -8450,31 +8652,32 @@ { "prompt": "Ignore the noisy meeting chatter and operationalize this support escalation workflow into a maintained skill package with trigger hardening.", "family": "adversarial_noisy_operationalize", - "score": 1.0, + "score": 0.812, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.211, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.158, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -8501,31 +8704,32 @@ { "prompt": "Codify this routine into a reusable capability from docs, transcripts, and prompt history, then stress test the route boundary.", "family": "adversarial_multi_asset_boundary", - "score": 1.0, + "score": 0.808, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.263, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.105, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -8560,18 +8764,19 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -8579,11 +8784,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.667, - "support_score": 0.06, - "lexical_support": 0.357, + "semantic_coverage": 0.411, + "support_score": 0, + "lexical_support": 0.286, "negative_penalty": 0.6, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "multi_asset": [ @@ -8617,10 +8822,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -8637,9 +8844,9 @@ "summarize_only", "translate_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.125, + "lexical_support": 0.062, "negative_penalty": 1.25, "coverage_boost": 0.0, "concept_evidence": { @@ -8678,10 +8885,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -8695,7 +8904,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.1, "negative_penalty": 0.64, "coverage_boost": 0.0, "concept_evidence": { @@ -8724,10 +8933,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package" @@ -8743,9 +8954,9 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.273, + "lexical_support": 0.091, "negative_penalty": 0.88, "coverage_boost": 0.0, "concept_evidence": { @@ -8845,31 +9056,32 @@ { "prompt": "Ignore the noisy meeting chatter and operationalize this support escalation workflow into a maintained skill package with trigger hardening.", "family": "adversarial_noisy_operationalize", - "score": 1.0, + "score": 0.812, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.211, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.158, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -8896,31 +9108,32 @@ { "prompt": "Codify this routine into a reusable capability from docs, transcripts, and prompt history, then stress test the route boundary.", "family": "adversarial_multi_asset_boundary", - "score": 1.0, + "score": 0.808, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.263, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.105, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -8955,18 +9168,19 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -8974,11 +9188,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.667, - "support_score": 0.06, - "lexical_support": 0.357, + "semantic_coverage": 0.411, + "support_score": 0, + "lexical_support": 0.286, "negative_penalty": 0.6, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "multi_asset": [ @@ -9012,10 +9226,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -9032,9 +9248,9 @@ "summarize_only", "translate_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.125, + "lexical_support": 0.062, "negative_penalty": 1.25, "coverage_boost": 0.0, "concept_evidence": { @@ -9073,10 +9289,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -9090,7 +9308,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.1, "negative_penalty": 0.64, "coverage_boost": 0.0, "concept_evidence": { @@ -9119,10 +9337,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package" @@ -9138,9 +9358,9 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.273, + "lexical_support": 0.091, "negative_penalty": 0.88, "coverage_boost": 0.0, "concept_evidence": { @@ -9569,9 +9789,9 @@ { "id": "current", "label": "Current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "strategy": "current", - "estimated_tokens": 53, + "estimated_tokens": 54, "dev": { "false_positives": 0, "false_negatives": 0, @@ -9587,7 +9807,7 @@ -1.0, -1.0, -1.0, - 53 + 54 ], "holdout": { "false_positives": 0, @@ -9921,37 +10141,37 @@ }, "winner_calibration": { "threshold": 0.33, - "mean_positive_score": 0.908, - "mean_non_trigger_score": 0.003, - "mean_near_neighbor_score": 0.006, - "min_positive_score": 0.767, - "max_non_trigger_score": 0.023, - "score_gap": 0.744, - "threshold_margin": 0.307, - "positive_threshold_buffer": 0.437, - "negative_threshold_buffer": 0.307, + "mean_positive_score": 0.704, + "mean_non_trigger_score": 0.0, + "mean_near_neighbor_score": 0.0, + "min_positive_score": 0.465, + "max_non_trigger_score": 0.0, + "score_gap": 0.465, + "threshold_margin": 0.135, + "positive_threshold_buffer": 0.135, + "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, "risk_band": "healthy" }, "current_calibration": { "threshold": 0.33, - "mean_positive_score": 0.908, - "mean_non_trigger_score": 0.003, - "mean_near_neighbor_score": 0.006, - "min_positive_score": 0.767, - "max_non_trigger_score": 0.023, - "score_gap": 0.744, - "threshold_margin": 0.307, - "positive_threshold_buffer": 0.437, - "negative_threshold_buffer": 0.307, + "mean_positive_score": 0.704, + "mean_non_trigger_score": 0.0, + "mean_near_neighbor_score": 0.0, + "min_positive_score": 0.465, + "max_non_trigger_score": 0.0, + "score_gap": 0.465, + "threshold_margin": 0.135, + "positive_threshold_buffer": 0.135, + "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, "risk_band": "healthy" }, "baseline_calibration": { "threshold": 0.33, - "mean_positive_score": 0.648, + "mean_positive_score": 0.707, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, "min_positive_score": 0.416, @@ -10022,33 +10242,33 @@ }, "winner_calibration": { "threshold": 0.33, - "mean_positive_score": 0.669, - "mean_non_trigger_score": 0.116, - "mean_near_neighbor_score": 0.152, - "min_positive_score": 0.665, - "max_non_trigger_score": 0.305, - "score_gap": 0.36, - "threshold_margin": 0.025, - "positive_threshold_buffer": 0.335, - "negative_threshold_buffer": 0.025, - "boundary_case_count": 1, - "boundary_case_rate": 0.167, - "risk_band": "tight" + "mean_positive_score": 0.377, + "mean_non_trigger_score": 0.024, + "mean_near_neighbor_score": 0.006, + "min_positive_score": 0.374, + "max_non_trigger_score": 0.084, + "score_gap": 0.29, + "threshold_margin": 0.044, + "positive_threshold_buffer": 0.044, + "negative_threshold_buffer": 0.246, + "boundary_case_count": 2, + "boundary_case_rate": 0.333, + "risk_band": "watch" }, "current_calibration": { "threshold": 0.33, - "mean_positive_score": 0.669, - "mean_non_trigger_score": 0.116, - "mean_near_neighbor_score": 0.152, - "min_positive_score": 0.665, - "max_non_trigger_score": 0.305, - "score_gap": 0.36, - "threshold_margin": 0.025, - "positive_threshold_buffer": 0.335, - "negative_threshold_buffer": 0.025, - "boundary_case_count": 1, - "boundary_case_rate": 0.167, - "risk_band": "tight" + "mean_positive_score": 0.377, + "mean_non_trigger_score": 0.024, + "mean_near_neighbor_score": 0.006, + "min_positive_score": 0.374, + "max_non_trigger_score": 0.084, + "score_gap": 0.29, + "threshold_margin": 0.044, + "positive_threshold_buffer": 0.044, + "negative_threshold_buffer": 0.246, + "boundary_case_count": 2, + "boundary_case_rate": 0.333, + "risk_band": "watch" }, "baseline_calibration": { "threshold": 0.33, @@ -10197,14 +10417,14 @@ }, "winner_calibration": { "threshold": 0.33, - "mean_positive_score": 1.0, + "mean_positive_score": 0.81, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, - "min_positive_score": 1.0, + "min_positive_score": 0.808, "max_non_trigger_score": 0.0, - "score_gap": 1.0, + "score_gap": 0.808, "threshold_margin": 0.33, - "positive_threshold_buffer": 0.67, + "positive_threshold_buffer": 0.478, "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, @@ -10212,14 +10432,14 @@ }, "current_calibration": { "threshold": 0.33, - "mean_positive_score": 1.0, + "mean_positive_score": 0.81, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, - "min_positive_score": 1.0, + "min_positive_score": 0.808, "max_non_trigger_score": 0.0, - "score_gap": 1.0, + "score_gap": 0.808, "threshold_margin": 0.33, - "positive_threshold_buffer": 0.67, + "positive_threshold_buffer": 0.478, "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, @@ -10274,8 +10494,8 @@ }, "summary": { "winner_label": "Current", - "winner_tokens": 53, - "current_tokens": 53, + "winner_tokens": 54, + "current_tokens": 54, "winner_dev_total_errors": 0, "current_dev_total_errors": 0, "winner_holdout_total_errors": 0, @@ -10288,7 +10508,7 @@ "current_adversarial_holdout_total_errors": 0, "winner_judge_blind_agreement_rate": 1.0, "winner_adversarial_risk_band": "healthy", - "winner_adversarial_score_gap": 1.0, + "winner_adversarial_score_gap": 0.808, "candidate_count": 6, "baseline_tokens": 8, "baseline_dev_total_errors": 1, diff --git a/reports/description_optimization.md b/reports/description_optimization.md index 3f86caf9..fe74b1d7 100644 --- a/reports/description_optimization.md +++ b/reports/description_optimization.md @@ -2,19 +2,19 @@ Winner: `Current` -- current tokens: `53` -- winner tokens: `53` +- current tokens: `54` +- winner tokens: `54` - baseline tokens: `8` ## Winner -Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution. +Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs. ## Candidate Ranking | Candidate | Tokens | Dev FP | Dev FN | Dev Near | Holdout FP | Holdout FN | | --- | ---: | ---: | ---: | ---: | ---: | ---: | -| `Current` | 53 | 0 | 0 | 1.0 | 0 | 0 | +| `Current` | 54 | 0 | 0 | 1.0 | 0 | 0 | | `Minimal` | 41 | 1 | 1 | 1.0 | 0 | 0 | | `Guardrail` | 56 | 1 | 1 | 1.0 | 0 | 0 | | `Balanced` | 60 | 1 | 1 | 1.0 | 0 | 0 | @@ -34,9 +34,9 @@ Create, refactor, evaluate, and package agent skills from workflows, prompts, tr | Gate | Winner Gap | Winner Risk | Winner Boundary Rate | Current Gap | Baseline Gap | | --- | ---: | --- | ---: | ---: | ---: | -| Holdout | 0.744 | healthy | 0.0 | 0.744 | 0.416 | -| Blind Holdout | 0.36 | tight | 0.167 | 0.36 | 0.368 | -| Adversarial Holdout | 1.0 | healthy | 0.0 | 1.0 | 0.845 | +| Holdout | 0.465 | healthy | 0.0 | 0.465 | 0.416 | +| Blind Holdout | 0.29 | watch | 0.333 | 0.29 | 0.368 | +| Adversarial Holdout | 0.808 | healthy | 0.0 | 0.808 | 0.845 | ## Judge Blind Summary diff --git a/reports/description_optimization_suite.json b/reports/description_optimization_suite.json index 46800ec1..29d8590e 100644 --- a/reports/description_optimization_suite.json +++ b/reports/description_optimization_suite.json @@ -3,9 +3,9 @@ { "name": "yao-meta-skill", "winner_label": "Current", - "winner_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "winner_tokens": 53, - "current_tokens": 53, + "winner_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "winner_tokens": 54, + "current_tokens": 54, "winner_holdout_fp": 0, "winner_holdout_fn": 0, "current_holdout_fp": 0, @@ -36,44 +36,44 @@ "calibration": { "holdout": { "threshold": 0.33, - "mean_positive_score": 0.908, - "mean_non_trigger_score": 0.003, - "mean_near_neighbor_score": 0.006, - "min_positive_score": 0.767, - "max_non_trigger_score": 0.023, - "score_gap": 0.744, - "threshold_margin": 0.307, - "positive_threshold_buffer": 0.437, - "negative_threshold_buffer": 0.307, + "mean_positive_score": 0.704, + "mean_non_trigger_score": 0.0, + "mean_near_neighbor_score": 0.0, + "min_positive_score": 0.465, + "max_non_trigger_score": 0.0, + "score_gap": 0.465, + "threshold_margin": 0.135, + "positive_threshold_buffer": 0.135, + "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, "risk_band": "healthy" }, "blind_holdout": { "threshold": 0.33, - "mean_positive_score": 0.669, - "mean_non_trigger_score": 0.116, - "mean_near_neighbor_score": 0.152, - "min_positive_score": 0.665, - "max_non_trigger_score": 0.305, - "score_gap": 0.36, - "threshold_margin": 0.025, - "positive_threshold_buffer": 0.335, - "negative_threshold_buffer": 0.025, - "boundary_case_count": 1, - "boundary_case_rate": 0.167, - "risk_band": "tight" + "mean_positive_score": 0.377, + "mean_non_trigger_score": 0.024, + "mean_near_neighbor_score": 0.006, + "min_positive_score": 0.374, + "max_non_trigger_score": 0.084, + "score_gap": 0.29, + "threshold_margin": 0.044, + "positive_threshold_buffer": 0.044, + "negative_threshold_buffer": 0.246, + "boundary_case_count": 2, + "boundary_case_rate": 0.333, + "risk_band": "watch" }, "adversarial_holdout": { "threshold": 0.33, - "mean_positive_score": 1.0, + "mean_positive_score": 0.81, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, - "min_positive_score": 1.0, + "min_positive_score": 0.808, "max_non_trigger_score": 0.0, - "score_gap": 1.0, + "score_gap": 0.808, "threshold_margin": 0.33, - "positive_threshold_buffer": 0.67, + "positive_threshold_buffer": 0.478, "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, diff --git a/reports/description_optimization_suite.md b/reports/description_optimization_suite.md index 6344d2fa..8743457e 100644 --- a/reports/description_optimization_suite.md +++ b/reports/description_optimization_suite.md @@ -2,7 +2,7 @@ | Target | Winner | Winner Tokens | Holdout FP | Holdout FN | Blind FP | Blind FN | Judge Blind Errors | Adv FP | Adv FN | Adv Gap | Adv Risk | Status | | --- | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | --- | --- | -| `yao-meta-skill` | `Current` | 53 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1.0 | healthy | ok | +| `yao-meta-skill` | `Current` | 54 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.808 | healthy | ok | | `team-frontend-review` | `Current` | 50 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.509 | healthy | ok | | `governed-incident-command` | `Current` | 37 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0.538 | watch | ok | diff --git a/reports/eval_suite.json b/reports/eval_suite.json index 9bf12c59..d4c7f90b 100644 --- a/reports/eval_suite.json +++ b/reports/eval_suite.json @@ -309,7 +309,7 @@ { "prompt": "Create a skill from this repeated workflow.", "family": "workflow_to_skill", - "score": 0.549, + "score": 0.518, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -321,7 +321,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -330,9 +330,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.491, + "semantic_coverage": 0.482, "support_score": 0, - "lexical_support": 0.714, + "lexical_support": 0.429, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -351,7 +351,7 @@ { "prompt": "Turn this runbook into a reusable agent skill.", "family": "workflow_to_skill", - "score": 0.739, + "score": 0.688, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -363,7 +363,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -373,9 +373,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.673, + "semantic_coverage": 0.661, "support_score": 0, - "lexical_support": 0.75, + "lexical_support": 0.25, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -397,7 +397,7 @@ { "prompt": "Convert this process note into a reusable skill package for the team.", "family": "workflow_to_skill", - "score": 0.766, + "score": 0.698, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -409,20 +409,21 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.709, - "support_score": 0, - "lexical_support": 0.667, + "semantic_coverage": 0.661, + "support_score": 0.04, + "lexical_support": 0.25, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -448,7 +449,7 @@ { "prompt": "Improve this skill description and add evals.", "family": "iterate_existing_skill", - "score": 0.365, + "score": 0.337, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -460,7 +461,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", @@ -469,9 +470,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.291, + "semantic_coverage": 0.286, "support_score": 0, - "lexical_support": 0.714, + "lexical_support": 0.429, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -492,7 +493,7 @@ { "prompt": "Refactor this prompt into a proper skill package.", "family": "iterate_existing_skill", - "score": 0.542, + "score": 0.504, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -504,7 +505,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -513,9 +514,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.491, + "semantic_coverage": 0.482, "support_score": 0, - "lexical_support": 0.625, + "lexical_support": 0.25, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -534,7 +535,7 @@ { "prompt": "Package this skill for team reuse.", "family": "package_for_team", - "score": 0.455, + "score": 0.386, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -546,21 +547,22 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "iterate_existing_skill", - "reuse_package", + "reuse_package" + ], + "extra_positive_concepts": [ "team_operationalize" ], - "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.364, - "support_score": 0, - "lexical_support": 0.833, + "semantic_coverage": 0.321, + "support_score": 0.04, + "lexical_support": 0.5, "negative_penalty": 0, - "coverage_boost": 0.06, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "iterate_existing_skill": [ @@ -579,12 +581,13 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Create a meta-skill for packaging internal workflows.", "family": "meta_skill_creation", - "score": 0.595, + "score": 0.534, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -596,21 +599,22 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", - "reuse_package", + "reuse_package" + ], + "extra_positive_concepts": [ "team_operationalize" ], - "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.527, - "support_score": 0, - "lexical_support": 0.625, + "semantic_coverage": 0.482, + "support_score": 0.04, + "lexical_support": 0.5, "negative_penalty": 0, - "coverage_boost": 0.06, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -632,7 +636,7 @@ { "prompt": "Standardize this recurring workflow as a shareable skill package with references.", "family": "package_for_team", - "score": 0.756, + "score": 0.742, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -644,22 +648,22 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.709, - "support_score": 0.06, - "lexical_support": 0.364, + "semantic_coverage": 0.714, + "support_score": 0.04, + "lexical_support": 0.182, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -687,7 +691,7 @@ { "prompt": "Codify this recurring support escalation workflow into a maintained skill for the ops library.", "family": "paraphrase_trigger", - "score": 0.397, + "score": 0.38, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -699,7 +703,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -708,9 +712,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.364, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.286, + "lexical_support": 0.143, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -731,7 +735,7 @@ { "prompt": "Operationalize this workflow as a reusable skill with validation notes and team library metadata.", "family": "paraphrase_trigger", - "score": 0.886, + "score": 0.815, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -743,21 +747,22 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", "eval_optimize", "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.855, - "support_score": 0, - "lexical_support": 0.5, + "semantic_coverage": 0.804, + "support_score": 0.04, + "lexical_support": 0.071, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -787,7 +792,7 @@ { "prompt": "Below are release notes, rough notes, and a transcript from the last handoff. Ignore the noise and turn this runbook into a reusable agent skill with references and scripts so the team can reuse it every quarter.", "family": "long_context_trigger", - "score": 0.756, + "score": 0.74, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -799,22 +804,22 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.709, - "support_score": 0.06, - "lexical_support": 0.355, + "semantic_coverage": 0.714, + "support_score": 0.04, + "lexical_support": 0.161, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -828,7 +833,8 @@ ], "reuse_package": [ "reusable", - "reuse" + "reuse", + "release" ], "team_operationalize": [ "team" @@ -837,7 +843,8 @@ "runbook", "rough notes", "transcript", - "release notes" + "release notes", + "scripts" ] }, "negative": {} @@ -847,7 +854,7 @@ { "prompt": "We have a standard operating procedure, checklist, and prompt history scattered across docs. Formalize this process into a reusable capability, add validation, and package it for team reuse.", "family": "long_context_trigger", - "score": 0.88, + "score": 0.819, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -859,21 +866,22 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", "eval_optimize", "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.855, - "support_score": 0, - "lexical_support": 0.423, + "semantic_coverage": 0.804, + "support_score": 0.04, + "lexical_support": 0.115, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -898,7 +906,8 @@ "process", "checklist", "prompt history", - "standard operating procedure" + "standard operating procedure", + "operating procedure" ] }, "negative": {} @@ -922,7 +931,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -934,9 +943,9 @@ "exclusive_negative_concepts": [ "explain_only" ], - "semantic_coverage": 0.182, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.0, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -968,7 +977,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -980,7 +989,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.333, + "lexical_support": 0.167, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -1009,7 +1018,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -1049,7 +1058,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -1062,7 +1071,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.0, "negative_penalty": 0.53, "coverage_boost": 0.0, "concept_evidence": { @@ -1094,7 +1103,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -1104,7 +1113,7 @@ "exclusive_negative_concepts": [], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.25, + "lexical_support": 0.125, "negative_penalty": 0.18, "coverage_boost": 0.0, "concept_evidence": { @@ -1133,7 +1142,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -1143,7 +1152,7 @@ "exclusive_negative_concepts": [], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.286, + "lexical_support": 0.143, "negative_penalty": 0.18, "coverage_boost": 0.0, "concept_evidence": { @@ -1160,7 +1169,7 @@ { "prompt": "Rewrite this paragraph more clearly, but do not package anything.", "family": "document_only", - "score": 0.175, + "score": 0.172, "predicted_trigger": false, "expected_trigger": false, "passed": true, @@ -1172,7 +1181,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package" @@ -1180,7 +1189,7 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.182, + "semantic_coverage": 0.179, "support_score": 0, "lexical_support": 0.1, "negative_penalty": 0, @@ -1198,7 +1207,7 @@ { "prompt": "Explain the difference between a runbook and a workflow.", "family": "explain_only", - "score": 0.187, + "score": 0.164, "predicted_trigger": false, "expected_trigger": false, "passed": true, @@ -1210,7 +1219,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -1218,9 +1227,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.182, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.25, + "lexical_support": 0.0, "negative_penalty": 0, "coverage_boost": 0.0, "concept_evidence": { @@ -1249,7 +1258,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -1264,9 +1273,9 @@ "no_build_directive", "summarize_only" ], - "semantic_coverage": 0.364, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.174, + "lexical_support": 0.087, "negative_penalty": 0.72, "coverage_boost": 0.04, "concept_evidence": { @@ -1306,7 +1315,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -1316,7 +1325,7 @@ "exclusive_negative_concepts": [], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.267, + "lexical_support": 0.067, "negative_penalty": 0.18, "coverage_boost": 0.0, "concept_evidence": { @@ -1346,7 +1355,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -1362,9 +1371,9 @@ "no_build_directive", "translate_only" ], - "semantic_coverage": 0.673, + "semantic_coverage": 0.661, "support_score": 0, - "lexical_support": 0.294, + "lexical_support": 0.176, "negative_penalty": 0.77, "coverage_boost": 0.06, "concept_evidence": { @@ -1409,7 +1418,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -1421,7 +1430,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.375, + "lexical_support": 0.25, "negative_penalty": 0.24, "coverage_boost": 0.0, "concept_evidence": { @@ -1437,7 +1446,7 @@ { "prompt": "Write a custom answer for this request without creating a skill package.", "family": "one_off_vs_reusable", - "score": 0.131, + "score": 0.115, "predicted_trigger": false, "expected_trigger": false, "passed": true, @@ -1449,7 +1458,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -1462,9 +1471,9 @@ "exclusive_negative_concepts": [ "one_off_request" ], - "semantic_coverage": 0.491, + "semantic_coverage": 0.482, "support_score": 0, - "lexical_support": 0.364, + "lexical_support": 0.273, "negative_penalty": 0.24, "coverage_boost": 0.04, "concept_evidence": { @@ -1500,7 +1509,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -1514,9 +1523,9 @@ "brainstorm_only", "no_build_directive" ], - "semantic_coverage": 0.182, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.333, + "lexical_support": 0.111, "negative_penalty": 0.66, "coverage_boost": 0.0, "concept_evidence": { @@ -1552,7 +1561,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -1565,7 +1574,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.455, + "lexical_support": 0.182, "negative_penalty": 0.6, "coverage_boost": 0.0, "concept_evidence": { @@ -1596,7 +1605,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -1609,9 +1618,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.364, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.4, + "lexical_support": 0.2, "negative_penalty": 0.42, "coverage_boost": 0.04, "concept_evidence": { @@ -1646,7 +1655,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -1659,9 +1668,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.364, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.357, + "lexical_support": 0.143, "negative_penalty": 0.42, "coverage_boost": 0.04, "concept_evidence": { @@ -1697,7 +1706,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -1710,9 +1719,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.182, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.211, + "lexical_support": 0.053, "negative_penalty": 0.6, "coverage_boost": 0.0, "concept_evidence": { @@ -1748,7 +1757,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -1763,9 +1772,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.673, + "semantic_coverage": 0.661, "support_score": 0, - "lexical_support": 0.4, + "lexical_support": 0.2, "negative_penalty": 0.6, "coverage_boost": 0.06, "concept_evidence": { @@ -1951,7 +1960,7 @@ { "prompt": "Convert this operations checklist into a reusable skill.", "family": "workflow_to_skill", - "score": 0.415, + "score": 0.379, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -1963,7 +1972,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -1972,9 +1981,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.364, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.5, + "lexical_support": 0.125, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -1989,12 +1998,13 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Add trigger evals to this skill before sharing it with the team.", "family": "iterate_existing_skill", - "score": 0.401, + "score": 0.333, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -2006,21 +2016,22 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", - "iterate_existing_skill", + "iterate_existing_skill" + ], + "extra_positive_concepts": [ "team_operationalize" ], - "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.327, - "support_score": 0, - "lexical_support": 0.5, + "semantic_coverage": 0.286, + "support_score": 0.04, + "lexical_support": 0.25, "negative_penalty": 0, - "coverage_boost": 0.06, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "eval_optimize": [ @@ -2045,7 +2056,7 @@ { "prompt": "We have a messy release runbook, export process, and a prompt history; turn all of that into one reusable skill package with evals and packaging checks.", "family": "complex_multi_asset", - "score": 0.843, + "score": 0.816, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -2057,7 +2068,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -2068,9 +2079,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.818, + "semantic_coverage": 0.804, "support_score": 0, - "lexical_support": 0.375, + "lexical_support": 0.208, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -2086,7 +2097,8 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ "runbook", @@ -2101,7 +2113,7 @@ { "prompt": "Turn these workflow fragments, transcripts, and prompts into one production-ready skill with scripts and references.", "family": "complex_multi_asset", - "score": 0.422, + "score": 0.454, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -2113,22 +2125,21 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.364, - "support_score": 0.06, - "lexical_support": 0.4, + "semantic_coverage": 0.411, + "support_score": 0, + "lexical_support": 0.2, "negative_penalty": 0, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "multi_asset": [ @@ -2140,7 +2151,8 @@ ], "transform_workflow": [ "workflow", - "workflow fragments" + "workflow fragments", + "scripts" ] }, "negative": {} @@ -2150,7 +2162,7 @@ { "prompt": "Tighten the trigger boundary on this existing skill and prepare it for team distribution.", "family": "iterate_existing_skill", - "score": 0.568, + "score": 0.514, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -2162,20 +2174,21 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", "iterate_existing_skill", - "reuse_package", + "reuse_package" + ], + "extra_positive_concepts": [ "team_operationalize" ], - "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.509, - "support_score": 0, - "lexical_support": 0.5, + "semantic_coverage": 0.464, + "support_score": 0.04, + "lexical_support": 0.214, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -2201,7 +2214,7 @@ { "prompt": "Turn the SOP, transcript, and cleanup notes into a maintained skill for the team library.", "family": "paraphrase_trigger", - "score": 0.474, + "score": 0.39, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -2213,21 +2226,22 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.4, - "support_score": 0, - "lexical_support": 0.571, + "semantic_coverage": 0.357, + "support_score": 0.04, + "lexical_support": 0.143, "negative_penalty": 0, - "coverage_boost": 0.06, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "reuse_package": [ @@ -2244,12 +2258,13 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Codify this workflow into a shared capability package and validate the route boundary before distribution.", "family": "paraphrase_trigger", - "score": 0.868, + "score": 0.815, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -2261,21 +2276,22 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", "eval_optimize", "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.855, - "support_score": 0, - "lexical_support": 0.267, + "semantic_coverage": 0.804, + "support_score": 0.04, + "lexical_support": 0.067, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -2307,7 +2323,7 @@ { "prompt": "After the long handoff notes below, build a skill package for team reuse. The materials include a transcript, rough notes, and an onboarding flow, and the result should include references, scripts, and validation.", "family": "long_context_trigger", - "score": 0.889, + "score": 0.871, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -2319,23 +2335,23 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", "eval_optimize", + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.855, - "support_score": 0.06, - "lexical_support": 0.346, + "semantic_coverage": 0.857, + "support_score": 0.04, + "lexical_support": 0.154, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -2363,7 +2379,8 @@ "transform_workflow": [ "rough notes", "transcript", - "onboarding flow" + "onboarding flow", + "scripts" ] }, "negative": {} @@ -2387,7 +2404,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -2399,9 +2416,9 @@ "exclusive_negative_concepts": [ "brainstorm_only" ], - "semantic_coverage": 0.182, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.286, + "lexical_support": 0.143, "negative_penalty": 0.24, "coverage_boost": 0.0, "concept_evidence": { @@ -2433,7 +2450,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -2445,7 +2462,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.083, + "lexical_support": 0.0, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -2474,14 +2491,13 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" - ], - "matched_desired_concepts": [ - "reuse_package" - ], - "extra_positive_concepts": [ "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset", + "reuse_package" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "brainstorm_only", "no_build_directive" @@ -2490,11 +2506,11 @@ "brainstorm_only", "no_build_directive" ], - "semantic_coverage": 0.182, - "support_score": 0.06, - "lexical_support": 0.2, + "semantic_coverage": 0.232, + "support_score": 0, + "lexical_support": 0.133, "negative_penalty": 0.66, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "multi_asset": [ @@ -2531,7 +2547,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -2543,7 +2559,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.333, + "lexical_support": 0.0, "negative_penalty": 0.35, "coverage_boost": 0.0, "concept_evidence": { @@ -2572,7 +2588,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -2584,9 +2600,9 @@ "exclusive_negative_concepts": [ "summarize_only" ], - "semantic_coverage": 0.182, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.111, + "lexical_support": 0.0, "negative_penalty": 0.3, "coverage_boost": 0.0, "concept_evidence": { @@ -2618,7 +2634,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package" @@ -2632,9 +2648,9 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.182, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.217, + "lexical_support": 0.0, "negative_penalty": 0.68, "coverage_boost": 0.0, "concept_evidence": { @@ -2659,7 +2675,7 @@ { "prompt": "I pasted a very long process note. For now I only need a recap and a short summary, not a skill package.", "family": "long_context_summary_only", - "score": 0.256, + "score": 0.236, "predicted_trigger": false, "expected_trigger": false, "passed": true, @@ -2671,7 +2687,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -2685,9 +2701,9 @@ "exclusive_negative_concepts": [ "summarize_only" ], - "semantic_coverage": 0.673, + "semantic_coverage": 0.661, "support_score": 0, - "lexical_support": 0.333, + "lexical_support": 0.222, "negative_penalty": 0.3, "coverage_boost": 0.06, "concept_evidence": { @@ -2709,8 +2725,7 @@ ] } } - }, - "boundary_case": true + } } ], "near_neighbor": [ @@ -2729,7 +2744,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -2744,9 +2759,9 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.364, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.1, "negative_penalty": 0.68, "coverage_boost": 0.04, "concept_evidence": { @@ -2784,7 +2799,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -2796,9 +2811,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.182, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.357, + "lexical_support": 0.071, "negative_penalty": 0.42, "coverage_boost": 0.0, "concept_evidence": { @@ -2830,7 +2845,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -2844,9 +2859,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.491, + "semantic_coverage": 0.482, "support_score": 0, - "lexical_support": 0.6, + "lexical_support": 0.3, "negative_penalty": 0.6, "coverage_boost": 0.04, "concept_evidence": { @@ -2884,7 +2899,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "iterate_existing_skill", @@ -2899,9 +2914,9 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.327, + "semantic_coverage": 0.321, "support_score": 0, - "lexical_support": 0.267, + "lexical_support": 0.2, "negative_penalty": 0.64, "coverage_boost": 0.04, "concept_evidence": { @@ -2939,7 +2954,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -2953,9 +2968,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.491, + "semantic_coverage": 0.482, "support_score": 0, - "lexical_support": 0.5, + "lexical_support": 0.143, "negative_penalty": 0.6, "coverage_boost": 0.04, "concept_evidence": { @@ -2994,7 +3009,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -3010,9 +3025,9 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.491, + "semantic_coverage": 0.482, "support_score": 0, - "lexical_support": 0.238, + "lexical_support": 0.048, "negative_penalty": 0.82, "coverage_boost": 0.04, "concept_evidence": { @@ -3058,7 +3073,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -3073,9 +3088,9 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.491, + "semantic_coverage": 0.482, "support_score": 0, - "lexical_support": 0.263, + "lexical_support": 0.158, "negative_penalty": 0.64, "coverage_boost": 0.04, "concept_evidence": { @@ -3231,7 +3246,7 @@ { "prompt": "Build a reusable skill from this long internal process note, and make sure it has references, scripts, and a trigger description.", "family": "complex_multi_asset", - "score": 0.472, + "score": 0.465, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -3243,21 +3258,21 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.4, - "support_score": 0.06, - "lexical_support": 0.368, + "semantic_coverage": 0.411, + "support_score": 0.04, + "lexical_support": 0.211, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -3274,7 +3289,8 @@ ], "transform_workflow": [ "process", - "internal process note" + "internal process note", + "scripts" ] }, "negative": {} @@ -3284,7 +3300,7 @@ { "prompt": "Take this existing skill draft, tighten the trigger boundary, add near-neighbor evals, and package it for distribution.", "family": "iterate_existing_skill", - "score": 0.731, + "score": 0.684, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -3296,21 +3312,22 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", "iterate_existing_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.691, - "support_score": 0, - "lexical_support": 0.444, + "semantic_coverage": 0.643, + "support_score": 0.04, + "lexical_support": 0.278, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -3341,7 +3358,7 @@ { "prompt": "Turn this recurring onboarding process into a reusable skill library entry with validation notes.", "family": "workflow_to_skill", - "score": 0.568, + "score": 0.805, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -3353,9 +3370,10 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" @@ -3363,13 +3381,16 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.509, + "semantic_coverage": 0.804, "support_score": 0, - "lexical_support": 0.5, + "lexical_support": 0.071, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { + "build_skill": [ + "skill library entry" + ], "eval_optimize": [ "validation" ], @@ -3388,7 +3409,7 @@ { "prompt": "Codify this quarterly release routine into a reusable skill package with scripts, references, and packaging checks.", "family": "paraphrase_trigger", - "score": 0.858, + "score": 0.874, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -3400,22 +3421,21 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", "eval_optimize", + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.818, - "support_score": 0.06, - "lexical_support": 0.375, + "semantic_coverage": 0.857, + "support_score": 0, + "lexical_support": 0.312, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -3433,10 +3453,12 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ - "quarterly release routine" + "quarterly release routine", + "scripts" ] }, "negative": {} @@ -3446,7 +3468,7 @@ { "prompt": "Below is a noisy transcript plus release notes and prompt history. Ignore the chatter and turn this playbook into an agent capability package for the team library.", "family": "long_context_trigger", - "score": 0.744, + "score": 0.691, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -3458,36 +3480,40 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.709, - "support_score": 0, - "lexical_support": 0.4, + "semantic_coverage": 0.661, + "support_score": 0.04, + "lexical_support": 0.16, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ - "turn this playbook into an agent capability package" + "turn this playbook into an agent capability package", + "agent capability" ], "reuse_package": [ "package", - "team library" + "team library", + "release" ], "team_operationalize": [ "team" ], "transform_workflow": [ "prompt history", + "playbook", "transcript", "release notes" ] @@ -3513,7 +3539,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -3528,9 +3554,9 @@ "no_build_directive", "summarize_only" ], - "semantic_coverage": 0.364, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.286, + "lexical_support": 0.143, "negative_penalty": 0.72, "coverage_boost": 0.04, "concept_evidence": { @@ -3569,7 +3595,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -3584,15 +3610,16 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.364, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.091, + "lexical_support": 0.182, "negative_penalty": 0.68, "coverage_boost": 0.04, "concept_evidence": { "positive": { "reuse_package": [ - "package" + "package", + "release" ], "transform_workflow": [ "checklist" @@ -3625,9 +3652,10 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ + "reuse_package", "transform_workflow" ], "extra_positive_concepts": [], @@ -3638,13 +3666,16 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.182, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.278, + "lexical_support": 0.167, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { + "reuse_package": [ + "release" + ], "transform_workflow": [ "workflow" ] @@ -3677,7 +3708,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -3693,9 +3724,9 @@ "no_build_directive", "translate_only" ], - "semantic_coverage": 0.673, + "semantic_coverage": 0.661, "support_score": 0, - "lexical_support": 0.312, + "lexical_support": 0.188, "negative_penalty": 0.77, "coverage_boost": 0.06, "concept_evidence": { @@ -3740,7 +3771,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill" @@ -3754,9 +3785,9 @@ "brainstorm_only", "future_outline" ], - "semantic_coverage": 0.309, + "semantic_coverage": 0.304, "support_score": 0, - "lexical_support": 0.308, + "lexical_support": 0.077, "negative_penalty": 0.46, "coverage_boost": 0.0, "concept_evidence": { @@ -3791,7 +3822,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -3805,7 +3836,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.357, + "lexical_support": 0.214, "negative_penalty": 0.64, "coverage_boost": 0.0, "concept_evidence": { @@ -3838,7 +3869,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -3852,9 +3883,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.491, + "semantic_coverage": 0.482, "support_score": 0, - "lexical_support": 0.615, + "lexical_support": 0.308, "negative_penalty": 0.6, "coverage_boost": 0.04, "concept_evidence": { @@ -3894,7 +3925,7 @@ "reuse_package", "eval_optimize", "iterate_existing_skill", - "team_operationalize" + "multi_asset" ], "matched_desired_concepts": [ "build_skill", @@ -3909,9 +3940,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.673, + "semantic_coverage": 0.661, "support_score": 0, - "lexical_support": 0.438, + "lexical_support": 0.375, "negative_penalty": 0.6, "coverage_boost": 0.06, "concept_evidence": { diff --git a/reports/evidence_consistency.json b/reports/evidence_consistency.json index 11f91876..93185470 100644 --- a/reports/evidence_consistency.json +++ b/reports/evidence_consistency.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "check_count": 39, @@ -324,12 +324,12 @@ "status": "pass", "expected": { "status": "pass", - "detail": "initial load 998/1000; deferred 530871/120000; top deferred scripts 469568; resource governance governed; quality density 140.3", + "detail": "initial load 999/1000; deferred 531767/120000; top deferred scripts 469568; resource governance governed; quality density 140.1", "evidence": "reports/context_budget.json" }, "actual": { "status": "pass", - "detail": "initial load 998/1000; deferred 530871/120000; top deferred scripts 469568; resource governance governed; quality density 140.3", + "detail": "initial load 999/1000; deferred 531767/120000; top deferred scripts 469568; resource governance governed; quality density 140.1", "evidence": "reports/context_budget.json" }, "paths": [ @@ -342,8 +342,8 @@ "key": "benchmark-release-lock-self-consistency", "label": "Benchmark release lock matches source dirty state", "status": "pass", - "expected": true, - "actual": true, + "expected": false, + "actual": false, "paths": [ "reports/benchmark_reproducibility.json" ], @@ -357,7 +357,7 @@ "actual": { "available": true, "clean": false, - "changed_file_count": 21 + "changed_file_count": 76 }, "paths": [ "reports/benchmark_reproducibility.json" @@ -398,8 +398,8 @@ "key": "overview-benchmark-commit", "label": "overview embeds the benchmark commit", "status": "pass", - "expected": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", - "actual": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", + "expected": "1710a5a3e0177313954758908735c147796e8981", + "actual": "1710a5a3e0177313954758908735c147796e8981", "paths": [ "reports/benchmark_reproducibility.json", "reports/skill-overview.json" @@ -411,36 +411,36 @@ "label": "overview embeds benchmark summary fields", "status": "pass", "expected": { - "release_lock_ready": true, + "release_lock_ready": false, "required_artifact_count": 25, "missing_artifact_count": 0, - "source_contract_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "source_contract_sha256": "f0333ea256d221c8993809b4366f93d9874defe6c722fbf95c8f524bdf9d3289", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7", "world_class_ledger_pending_count": 4, "world_class_source_check_count": 19, "world_class_source_pass_count": 12, "world_class_source_blocked_count": 7, - "beta_test_ready": true, - "beta_test_blocker_count": 0, + "beta_test_ready": false, + "beta_test_blocker_count": 1, "beta_test_deferred_evidence_count": 4, "public_claim_ready": false, - "public_claim_blocker_count": 3 + "public_claim_blocker_count": 4 }, "actual": { - "release_lock_ready": true, + "release_lock_ready": false, "required_artifact_count": 25, "missing_artifact_count": 0, - "source_contract_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "source_contract_sha256": "f0333ea256d221c8993809b4366f93d9874defe6c722fbf95c8f524bdf9d3289", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7", "world_class_ledger_pending_count": 4, "world_class_source_check_count": 19, "world_class_source_pass_count": 12, "world_class_source_blocked_count": 7, - "beta_test_ready": true, - "beta_test_blocker_count": 0, + "beta_test_ready": false, + "beta_test_blocker_count": 1, "beta_test_deferred_evidence_count": 4, "public_claim_ready": false, - "public_claim_blocker_count": 3 + "public_claim_blocker_count": 4 }, "paths": [ "reports/benchmark_reproducibility.json", @@ -550,8 +550,8 @@ "key": "interpretation-benchmark-commit", "label": "interpretation embeds the benchmark commit", "status": "pass", - "expected": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", - "actual": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", + "expected": "1710a5a3e0177313954758908735c147796e8981", + "actual": "1710a5a3e0177313954758908735c147796e8981", "paths": [ "reports/benchmark_reproducibility.json", "reports/skill-interpretation.json" @@ -563,36 +563,36 @@ "label": "interpretation embeds benchmark summary fields", "status": "pass", "expected": { - "release_lock_ready": true, + "release_lock_ready": false, "required_artifact_count": 25, "missing_artifact_count": 0, - "source_contract_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "source_contract_sha256": "f0333ea256d221c8993809b4366f93d9874defe6c722fbf95c8f524bdf9d3289", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7", "world_class_ledger_pending_count": 4, "world_class_source_check_count": 19, "world_class_source_pass_count": 12, "world_class_source_blocked_count": 7, - "beta_test_ready": true, - "beta_test_blocker_count": 0, + "beta_test_ready": false, + "beta_test_blocker_count": 1, "beta_test_deferred_evidence_count": 4, "public_claim_ready": false, - "public_claim_blocker_count": 3 + "public_claim_blocker_count": 4 }, "actual": { - "release_lock_ready": true, + "release_lock_ready": false, "required_artifact_count": 25, "missing_artifact_count": 0, - "source_contract_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "source_contract_sha256": "f0333ea256d221c8993809b4366f93d9874defe6c722fbf95c8f524bdf9d3289", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7", "world_class_ledger_pending_count": 4, "world_class_source_check_count": 19, "world_class_source_pass_count": 12, "world_class_source_blocked_count": 7, - "beta_test_ready": true, - "beta_test_blocker_count": 0, + "beta_test_ready": false, + "beta_test_blocker_count": 1, "beta_test_deferred_evidence_count": 4, "public_claim_ready": false, - "public_claim_blocker_count": 3 + "public_claim_blocker_count": 4 }, "paths": [ "reports/benchmark_reproducibility.json", @@ -739,7 +739,7 @@ "label": "可维护性", "score": 100, "reasons": [ - "SKILL.md 约 339 个词/字。", + "SKILL.md 约 342 个词/字。", "入口文件保持克制，可维护性较好。", "references/ 已承载扩展指导。", "scripts/ 已承载确定性逻辑。", @@ -760,7 +760,7 @@ "label": "上下文成本", "score": 42, "reasons": [ - "入口约 339 个词/字，references 约 16934 个词/字。", + "入口约 342 个词/字，references 约 16934 个词/字。", "分数越高代表上下文成本越低。", "上下文成本偏高，建议压缩入口或拆分 references。" ] @@ -803,7 +803,7 @@ "label": "可维护性", "score": 100, "reasons": [ - "SKILL.md 约 339 个词/字。", + "SKILL.md 约 342 个词/字。", "入口文件保持克制，可维护性较好。", "references/ 已承载扩展指导。", "scripts/ 已承载确定性逻辑。", @@ -824,7 +824,7 @@ "label": "上下文成本", "score": 42, "reasons": [ - "入口约 339 个词/字，references 约 16934 个词/字。", + "入口约 342 个词/字，references 约 16934 个词/字。", "分数越高代表上下文成本越低。", "上下文成本偏高，建议压缩入口或拆分 references。" ] @@ -954,7 +954,7 @@ "status": "pass", "expected": { "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "activation": "manual", "execution": "inline", "shell": "bash" @@ -982,7 +982,7 @@ "boundary_cards": [ { "label": "Owned", - "body": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "body": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." }, { "label": "Adjacent", @@ -996,7 +996,7 @@ }, "actual": { "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "activation": "manual", "execution": "inline", "shell": "bash" @@ -1024,7 +1024,7 @@ "boundary_cards": [ { "label": "Owned", - "body": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "body": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." }, { "label": "Adjacent", @@ -1056,7 +1056,7 @@ "已生成 Output Review Adjudication，可记录盲评决策、一致率和待评审项。" ], "gaps": [ - "上下文成本需要补强：入口约 339 个词/字，references 约 16934 个词/字。" + "上下文成本需要补强：入口约 342 个词/字，references 约 16934 个词/字。" ], "recommendations": [ "先改触发边界，再扩展工作流。", @@ -1104,7 +1104,7 @@ "已生成 Output Review Adjudication，可记录盲评决策、一致率和待评审项。" ], "gaps": [ - "上下文成本需要补强：入口约 339 个词/字，references 约 16934 个词/字。" + "上下文成本需要补强：入口约 342 个词/字，references 约 16934 个词/字。" ], "recommendations": [ "先改触发边界，再扩展工作流。", @@ -1179,7 +1179,7 @@ "name": "包体膨胀风险", "impact": 2, "probability": 1, - "signal": "SKILL.md 约 339 个词/字。", + "signal": "SKILL.md 约 342 个词/字。", "response": "先补证据和边界，再增加包体复杂度。" }, { @@ -1338,7 +1338,7 @@ "name": "包体膨胀风险", "impact": 2, "probability": 1, - "signal": "SKILL.md 约 339 个词/字。", + "signal": "SKILL.md 约 342 个词/字。", "response": "先补证据和边界，再增加包体复杂度。" }, { @@ -1680,10 +1680,10 @@ "path": "reports", "label": "Generated evidence and overview artifacts", "kind": "folder", - "file_count": 152 + "file_count": 170 } ], - "file_count": 375, + "file_count": 393, "folder_count": 4, "distribution": [ { @@ -1716,7 +1716,7 @@ }, { "label": "reports", - "value": 152 + "value": 170 } ] }, @@ -1768,10 +1768,10 @@ "path": "reports", "label": "Generated evidence and overview artifacts", "kind": "folder", - "file_count": 152 + "file_count": 170 } ], - "file_count": 375, + "file_count": 393, "folder_count": 4, "distribution": [ { @@ -1804,7 +1804,7 @@ }, { "label": "reports", - "value": 152 + "value": 170 } ] }, @@ -1989,10 +1989,10 @@ "label": "Benchmark separates beta testing from public claims", "status": "pass", "expected": { - "beta_test_ready": true, + "beta_test_ready": false, "public_claim_ready": false, "human_review_complete": false, - "beta_release_ready": true, + "beta_release_ready": false, "beta_release_scope": "beta/public test release without superiority, fully-reviewed, or world-class claims", "deferred_evidence_keys": [ "human-adjudication", @@ -2003,10 +2003,10 @@ "deferred_human_review": true }, "actual": { - "beta_test_ready": true, + "beta_test_ready": false, "public_claim_ready": false, "human_review_complete": false, - "beta_release_ready": true, + "beta_release_ready": false, "beta_release_scope": "beta/public test release without superiority, fully-reviewed, or world-class claims", "deferred_evidence_keys": [ "human-adjudication", @@ -2139,8 +2139,8 @@ "artifact_prefill_counts_as_evidence": false, "submission_ref_total_count": 2, "submission_ref_ready_count": 2, - "supporting_evidence_total_count": 11, - "supporting_evidence_ready_count": 11, + "supporting_evidence_total_count": 8, + "supporting_evidence_ready_count": 7, "submission_ref_copy_to_artifact_refs": true, "supporting_evidence_copy_to_artifact_refs": false }, @@ -2185,8 +2185,8 @@ "artifact_prefill_counts_as_evidence": false, "submission_ref_total_count": 2, "submission_ref_ready_count": 2, - "supporting_evidence_total_count": 11, - "supporting_evidence_ready_count": 11, + "supporting_evidence_total_count": 8, + "supporting_evidence_ready_count": 7, "submission_ref_copy_to_artifact_refs": true, "supporting_evidence_copy_to_artifact_refs": false }, @@ -3200,9 +3200,7 @@ "agents/interface.yaml": true, "evidence/world_class/README.md": true, "security/permission_policy.json": true, - "reports/world_class_evidence_ledger.json": true, - "dist/manifest.json": true, - "dist/targets/openai/adapter.json": true + "reports/world_class_evidence_ledger.json": true }, "prohibited_surfaces": [] }, @@ -3221,9 +3219,7 @@ "agents/interface.yaml": true, "evidence/world_class/README.md": true, "security/permission_policy.json": true, - "reports/world_class_evidence_ledger.json": true, - "dist/manifest.json": true, - "dist/targets/openai/adapter.json": true + "reports/world_class_evidence_ledger.json": true }, "prohibited_surfaces": [] }, @@ -3585,7 +3581,7 @@ "`0` permission failures", "`25` required artifacts", "`23` reproduction commands", - "initial load `998/1000`", + "initial load `999/1000`", "target count is `83`" ], "actual": "all present", diff --git a/reports/evidence_consistency.md b/reports/evidence_consistency.md index 92322f9b..995a8516 100644 --- a/reports/evidence_consistency.md +++ b/reports/evidence_consistency.md @@ -1,6 +1,6 @@ # Evidence Consistency -Generated at: `2026-06-21` +Generated at: `2026-07-02` ## Summary diff --git a/reports/install_simulation.json b/reports/install_simulation.json index 341369f3..5f9c1684 100644 --- a/reports/install_simulation.json +++ b/reports/install_simulation.json @@ -1,14 +1,14 @@ { "ok": true, "schema_version": "2.0", - "generated_at": "2026-06-21", + "generated_at": "2026-06-13", "skill_dir": ".", - "package_dir": "dist", - "install_root": "dist/install-simulation/simulate-yao-meta-skill", - "installed_skill_dir": "dist/install-simulation/simulate-yao-meta-skill/yao-meta-skill", + "package_dir": "tests/tmp_review_studio/dist", + "install_root": "tests/tmp_review_studio/install-root/simulate-yao-meta-skill", + "installed_skill_dir": "tests/tmp_review_studio/install-root/simulate-yao-meta-skill/yao-meta-skill", "summary": { "archive_present": true, - "archive_entry_count": 580, + "archive_entry_count": 598, "nested_skill_entry_count": 0, "archive_extracted": true, "entrypoint_loaded": true, @@ -27,7 +27,7 @@ { "id": "archive-present", "status": "pass", - "detail": "Package archive exists: dist/yao-meta-skill.zip" + "detail": "Package archive exists: tests/tmp_review_studio/dist/yao-meta-skill.zip" }, { "id": "archive-safe-paths", @@ -278,9 +278,9 @@ "failures": [], "warnings": [], "artifacts": { - "archive": "dist/yao-meta-skill.zip", - "package_manifest": "dist/manifest.json", - "json": "reports/install_simulation.json", - "markdown": "reports/install_simulation.md" + "archive": "tests/tmp_review_studio/dist/yao-meta-skill.zip", + "package_manifest": "tests/tmp_review_studio/dist/manifest.json", + "json": "tests/tmp_review_studio/install_simulation.json", + "markdown": "tests/tmp_review_studio/install_simulation.md" } } diff --git a/reports/install_simulation.md b/reports/install_simulation.md index 77a11ed1..fcf6ac83 100644 --- a/reports/install_simulation.md +++ b/reports/install_simulation.md @@ -1,7 +1,7 @@ # Install Simulation - OK: `True` -- Package directory: `dist` +- Package directory: `tests/tmp_review_studio/dist` - Archive extracted: `True` - Nested SKILL.md entries: `0` - Entrypoint loaded: `True` @@ -17,7 +17,7 @@ | Check | Status | Detail | | --- | --- | --- | -| `archive-present` | `pass` | Package archive exists: dist/yao-meta-skill.zip | +| `archive-present` | `pass` | Package archive exists: tests/tmp_review_studio/dist/yao-meta-skill.zip | | `archive-safe-paths` | `pass` | Archive has no absolute or parent-traversal entries | | `single-skill-entrypoint` | `pass` | Installed package exposes only the root SKILL.md entrypoint | | `single-top-level` | `pass` | Archive top-level directory is yao-meta-skill | diff --git a/reports/iteration_bundles/yao-meta-skill/bundle.json b/reports/iteration_bundles/yao-meta-skill/bundle.json index c071b070..04c26cab 100644 --- a/reports/iteration_bundles/yao-meta-skill/bundle.json +++ b/reports/iteration_bundles/yao-meta-skill/bundle.json @@ -35,8 +35,8 @@ "label": "Current", "ranking_state": "selected_by_dev", "promotion_state": "kept_current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "tokens": 53, + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "tokens": 54, "dev_errors": 0, "holdout_errors": 0, "reason_tags": [] @@ -118,8 +118,8 @@ ], "review_template": { "target": "yao-meta-skill", - "current_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "candidate_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "current_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "candidate_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "focus": [ "no_candidate_outperformed_current" ] diff --git a/reports/iteration_bundles/yao-meta-skill/bundle.md b/reports/iteration_bundles/yao-meta-skill/bundle.md index b207ee61..417c21f8 100644 --- a/reports/iteration_bundles/yao-meta-skill/bundle.md +++ b/reports/iteration_bundles/yao-meta-skill/bundle.md @@ -27,7 +27,7 @@ | Role | Label | Ranking State | Promotion State | Tokens | Dev Errors | Holdout Errors | | --- | --- | --- | --- | ---: | ---: | ---: | | baseline | `Baseline` | reference | reference | 8 | 1 | 0 | -| current | `Current` | selected_by_dev | kept_current | 53 | 0 | 0 | +| current | `Current` | selected_by_dev | kept_current | 54 | 0 | 0 | | candidate | `Minimal` | not_selected | blocked | 41 | 2 | 0 | | candidate | `Guardrail` | not_selected | blocked | 56 | 2 | 0 | | candidate | `Balanced` | not_selected | blocked | 60 | 2 | 0 | @@ -37,8 +37,8 @@ ## Human Review Stub - target: yao-meta-skill -- current description: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution. -- candidate description: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution. +- current description: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs. +- candidate description: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs. - review focus: no_candidate_outperformed_current ## Artifact Paths diff --git a/reports/iteration_bundles/yao-meta-skill/review.md b/reports/iteration_bundles/yao-meta-skill/review.md index 69c18c9d..636577af 100644 --- a/reports/iteration_bundles/yao-meta-skill/review.md +++ b/reports/iteration_bundles/yao-meta-skill/review.md @@ -1,8 +1,8 @@ # Human Review - target: yao-meta-skill -- current description: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution. -- candidate description: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution. +- current description: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs. +- candidate description: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs. - suggested focus: no_candidate_outperformed_current Use the shared template in `references/human-review-template.md` to complete the final decision. diff --git a/reports/iteration_ledger.md b/reports/iteration_ledger.md index acf95708..c41782a3 100644 --- a/reports/iteration_ledger.md +++ b/reports/iteration_ledger.md @@ -43,7 +43,7 @@ | Route | Precision | Recall | Avg Margin | | --- | ---: | ---: | ---: | -| `yao-meta-skill` | 1.0 | 1.0 | 0.701 | +| `yao-meta-skill` | 1.0 | 1.0 | 0.644 | | `team-frontend-review` | 1.0 | 1.0 | 0.805 | -| `governed-incident-command` | 1.0 | 1.0 | 0.586 | +| `governed-incident-command` | 1.0 | 1.0 | 0.709 | | `no_route` | 1.0 | 1.0 | - | diff --git a/reports/promotion_decisions.json b/reports/promotion_decisions.json index e84063f2..59e6cc39 100644 --- a/reports/promotion_decisions.json +++ b/reports/promotion_decisions.json @@ -9,9 +9,9 @@ { "name": "yao-meta-skill", "winner_label": "Current", - "winner_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "winner_tokens": 53, - "current_tokens": 53, + "winner_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "winner_tokens": 54, + "current_tokens": 54, "winner_holdout_fp": 0, "winner_holdout_fn": 0, "current_holdout_fp": 0, @@ -42,44 +42,44 @@ "calibration": { "holdout": { "threshold": 0.33, - "mean_positive_score": 0.908, - "mean_non_trigger_score": 0.003, - "mean_near_neighbor_score": 0.006, - "min_positive_score": 0.767, - "max_non_trigger_score": 0.023, - "score_gap": 0.744, - "threshold_margin": 0.307, - "positive_threshold_buffer": 0.437, - "negative_threshold_buffer": 0.307, + "mean_positive_score": 0.704, + "mean_non_trigger_score": 0.0, + "mean_near_neighbor_score": 0.0, + "min_positive_score": 0.465, + "max_non_trigger_score": 0.0, + "score_gap": 0.465, + "threshold_margin": 0.135, + "positive_threshold_buffer": 0.135, + "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, "risk_band": "healthy" }, "blind_holdout": { "threshold": 0.33, - "mean_positive_score": 0.669, - "mean_non_trigger_score": 0.116, - "mean_near_neighbor_score": 0.152, - "min_positive_score": 0.665, - "max_non_trigger_score": 0.305, - "score_gap": 0.36, - "threshold_margin": 0.025, - "positive_threshold_buffer": 0.335, - "negative_threshold_buffer": 0.025, - "boundary_case_count": 1, - "boundary_case_rate": 0.167, - "risk_band": "tight" + "mean_positive_score": 0.377, + "mean_non_trigger_score": 0.024, + "mean_near_neighbor_score": 0.006, + "min_positive_score": 0.374, + "max_non_trigger_score": 0.084, + "score_gap": 0.29, + "threshold_margin": 0.044, + "positive_threshold_buffer": 0.044, + "negative_threshold_buffer": 0.246, + "boundary_case_count": 2, + "boundary_case_rate": 0.333, + "risk_band": "watch" }, "adversarial_holdout": { "threshold": 0.33, - "mean_positive_score": 1.0, + "mean_positive_score": 0.81, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, - "min_positive_score": 1.0, + "min_positive_score": 0.808, "max_non_trigger_score": 0.0, - "score_gap": 1.0, + "score_gap": 0.808, "threshold_margin": 0.33, - "positive_threshold_buffer": 0.67, + "positive_threshold_buffer": 0.478, "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, @@ -170,7 +170,7 @@ "route_metrics": { "precision": 1.0, "recall": 1.0, - "average_margin": 0.701, + "average_margin": 0.644, "misroute_count": 0, "ambiguous_case_count": 0 } @@ -193,8 +193,8 @@ "label": "Current", "ranking_state": "selected_by_dev", "promotion_state": "kept_current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "tokens": 53, + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "tokens": 54, "dev_errors": 0, "holdout_errors": 0, "reason_tags": [] @@ -702,7 +702,7 @@ "route_metrics": { "precision": 1.0, "recall": 1.0, - "average_margin": 0.586, + "average_margin": 0.709, "misroute_count": 0, "ambiguous_case_count": 0 } diff --git a/reports/prompt-quality-profile.json b/reports/prompt-quality-profile.json index b16302c5..4ce2fb8b 100644 --- a/reports/prompt-quality-profile.json +++ b/reports/prompt-quality-profile.json @@ -1,5 +1,5 @@ { - "skill_name": "yao-meta-skill-operator-ux-worktree", + "skill_name": "yao-meta-skill-trigger-coverage-merge", "relevance": "prompt-aware", "primary_task_family": { "key": "execution_operation", diff --git a/reports/prompt-quality-profile.md b/reports/prompt-quality-profile.md index 191f177d..f11955ba 100644 --- a/reports/prompt-quality-profile.md +++ b/reports/prompt-quality-profile.md @@ -1,6 +1,6 @@ # Prompt Quality Profile -Skill: `yao-meta-skill-operator-ux-worktree` +Skill: `yao-meta-skill-trigger-coverage-merge` Relevance: `prompt-aware` Overall quality score: `89.0/100` diff --git a/reports/python_compatibility.json b/reports/python_compatibility.json index b754a3cd..a9b4eb8f 100644 --- a/reports/python_compatibility.json +++ b/reports/python_compatibility.json @@ -5,7 +5,7 @@ "root": ".", "summary": { "target_python": "3.11", - "file_count": 234, + "file_count": 235, "issue_count": 0, "syntax_error_count": 0, "fstring_311_violation_count": 0, @@ -34,6 +34,12 @@ "issue_count": 0, "issues": [] }, + { + "path": "reports/blind-human-review-2026-06-29/generate_pack.py", + "ok": true, + "issue_count": 0, + "issues": [] + }, { "path": "scripts/adaptation_patch_safety.py", "ok": true, diff --git a/reports/python_compatibility.md b/reports/python_compatibility.md index 12279a95..23a3e889 100644 --- a/reports/python_compatibility.md +++ b/reports/python_compatibility.md @@ -6,7 +6,7 @@ Generated at: `2026-06-13` - decision: `pass` - target python: `3.11` -- files scanned: `234` +- files scanned: `235` - issues: `0` - syntax errors: `0` - f-string 3.11 violations: `0` diff --git a/reports/reference-synthesis.json b/reports/reference-synthesis.json index b27e846a..1bef3f55 100644 --- a/reports/reference-synthesis.json +++ b/reports/reference-synthesis.json @@ -1,6 +1,6 @@ { "skill_name": "yao-meta-skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "intent_confidence": { "score": 100, "band": "high", @@ -50,39 +50,41 @@ }, { "source_type": "research", - "name": "Human-in-the-loop verification", + "name": "Hypothesis-test-learn loop", "evidence_mode": "curated-pattern-track", "matched_keywords": [ - "review", - "govern" + "test", + "iteration" ], - "borrow": "Borrow a review checkpoint wherever trust matters more than raw speed.", - "avoid": "Do not force every skill through heavyweight review when the risk is low.", - "why_relevant": "This track matches: review, govern." + "borrow": "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", + "avoid": "Do not create experimental overhead that exceeds the skill's real risk tier.", + "why_relevant": "This track matches: test, iteration." }, { "source_type": "principles", "name": "Boundary-first design", "evidence_mode": "curated-pattern-track", "matched_keywords": [ - "route" + "route", + "trigger", + "exclude" ], "borrow": "Borrow the discipline of defining what the skill should not own before growing the package.", "avoid": "Do not expand execution assets until route boundaries stay clean.", - "why_relevant": "This track matches: route." + "why_relevant": "This track matches: route, trigger, exclude." } ], "synthesis": { "borrow_now": [ "Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.", - "Borrow a review checkpoint wherever trust matters more than raw speed.", + "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", "Borrow the discipline of defining what the skill should not own before growing the package.", "Borrow the way it turns a messy workflow into a repeatable operating path.", "Borrow the clear execution entrypoints and command structure." ], "avoid_now": [ "Do not let packaging or platform concerns swallow the core job boundary.", - "Do not force every skill through heavyweight review when the risk is low.", + "Do not create experimental overhead that exceeds the skill's real risk tier.", "Do not expand execution assets until route boundaries stay clean.", "Do not import process overhead that only exists for that project's scale.", "Do not copy repo-specific commands or environment assumptions verbatim." @@ -118,10 +120,10 @@ "score": 4 }, { - "name": "Human-in-the-loop verification", + "name": "Hypothesis-test-learn loop", "source_type": "research", - "borrow": "Borrow a review checkpoint wherever trust matters more than raw speed.", - "avoid": "Do not force every skill through heavyweight review when the risk is low.", + "borrow": "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", + "avoid": "Do not create experimental overhead that exceeds the skill's real risk tier.", "gates": { "recurrence": true, "generativity": true, @@ -235,11 +237,11 @@ "summary": "Start by borrowing this pattern: Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts. Avoid this for the first pass: Do not let packaging or platform concerns swallow the core job boundary.", "borrow_now": [ "Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.", - "Borrow a review checkpoint wherever trust matters more than raw speed." + "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed." ], "avoid_for_now": [ "Do not let packaging or platform concerns swallow the core job boundary.", - "Do not force every skill through heavyweight review when the risk is low." + "Do not create experimental overhead that exceeds the skill's real risk tier." ], "why": "There is a real design conflict to resolve: The stated preference leans lightweight or speed-first, while the benchmark mix leans toward governance, review, or heavier evaluation structure.", "user_decision_required": true diff --git a/reports/reference-synthesis.md b/reports/reference-synthesis.md index 82c5a69e..a839f847 100644 --- a/reports/reference-synthesis.md +++ b/reports/reference-synthesis.md @@ -1,7 +1,7 @@ # Reference Synthesis Skill: `yao-meta-skill` -- Description: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution. +- Description: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs. - Intent confidence: `100/100` (`high`) ## Live GitHub Benchmarks @@ -33,24 +33,24 @@ Skill: `yao-meta-skill` - Borrow: Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts. - Avoid: Do not let packaging or platform concerns swallow the core job boundary. -### Human-in-the-loop verification +### Hypothesis-test-learn loop - Type: `research` - Evidence mode: `curated-pattern-track` -- Why relevant: This track matches: review, govern. -- Borrow: Borrow a review checkpoint wherever trust matters more than raw speed. -- Avoid: Do not force every skill through heavyweight review when the risk is low. +- Why relevant: This track matches: test, iteration. +- Borrow: Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed. +- Avoid: Do not create experimental overhead that exceeds the skill's real risk tier. ### Boundary-first design - Type: `principles` - Evidence mode: `curated-pattern-track` -- Why relevant: This track matches: route. +- Why relevant: This track matches: route, trigger, exclude. - Borrow: Borrow the discipline of defining what the skill should not own before growing the package. - Avoid: Do not expand execution assets until route boundaries stay clean. ## Borrow Now - Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts. -- Borrow a review checkpoint wherever trust matters more than raw speed. +- Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed. - Borrow the discipline of defining what the skill should not own before growing the package. - Borrow the way it turns a messy workflow into a repeatable operating path. - Borrow the clear execution entrypoints and command structure. @@ -58,7 +58,7 @@ Skill: `yao-meta-skill` ## Avoid Now - Do not let packaging or platform concerns swallow the core job boundary. -- Do not force every skill through heavyweight review when the risk is low. +- Do not create experimental overhead that exceeds the skill's real risk tier. - Do not expand execution assets until route boundaries stay clean. - Do not import process overhead that only exists for that project's scale. - Do not copy repo-specific commands or environment assumptions verbatim. @@ -69,7 +69,7 @@ Skill: `yao-meta-skill` - Acceptance threshold: `4/4` - Accepted patterns: - **Official skill anatomy and context discipline**: 4/4 (recurrence, generativity, distinctiveness, boundary) - - **Human-in-the-loop verification**: 4/4 (recurrence, generativity, distinctiveness, boundary) + - **Hypothesis-test-learn loop**: 4/4 (recurrence, generativity, distinctiveness, boundary) - **Boundary-first design**: 4/4 (recurrence, generativity, distinctiveness, boundary) - **obra/superpowers**: 4/4 (recurrence, generativity, distinctiveness, boundary) - **affaan-m/ECC**: 4/4 (recurrence, generativity, distinctiveness, boundary) diff --git a/reports/registry_audit.json b/reports/registry_audit.json index 82b38827..356a913b 100644 --- a/reports/registry_audit.json +++ b/reports/registry_audit.json @@ -7,7 +7,7 @@ "schema_version": "2.0", "name": "yao-meta-skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "targets": [ "openai", "claude", @@ -21,7 +21,7 @@ "trust_level": "local", "license": "MIT", "checksums": { - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7" }, "compatibility": { @@ -58,11 +58,11 @@ "install_simulated": true, "install_simulation": "reports/install_simulation.json" }, - "generated_at": "2026-06-21" + "generated_at": "2026-06-13" }, "index": { "schema_version": "2.0", - "generated_at": "2026-06-21", + "generated_at": "2026-06-13", "package_count": 1, "packages": [ { @@ -78,7 +78,7 @@ "vscode" ], "package_metadata": "registry/packages/yao-meta-skill.json", - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" } ] }, diff --git a/reports/registry_audit.md b/reports/registry_audit.md index b2b885c5..852f7c57 100644 --- a/reports/registry_audit.md +++ b/reports/registry_audit.md @@ -6,7 +6,7 @@ - Maturity: `governed` - Owner: `Yao Team` - License: `MIT` -- Package SHA256: `eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383` +- Package SHA256: `e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3` - Archive SHA256: `c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7` - Install simulated: `True` diff --git a/reports/review-studio.html b/reports/review-studio.html index 23bc94b0..c4fac5d6 100644 --- a/reports/review-studio.html +++ b/reports/review-studio.html @@ -762,7 +762,7 @@
Review Studio 2.0

Yao Meta Skill

-

Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

+

Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.

审查结论 review @@ -772,12 +772,12 @@

核心指标

-
Skill IR2.0.0

5 targets in platform-neutral contract

Compiler5/5

target contracts compiled from Skill IR

Output Delta100.0

5 cases; 1 file-backed

Exec Runs10

command 10; model 10; recorded 0

Blind A/B5

review pairs hide baseline vs with-skill labels

Review Kit0/5

pending 5; answer key hidden

Review A/B0/5

adjudication decisions; pending 5

Public Claimblocked

3 blockers; local reproducible true

Blueprint21/21

2.0 coverage; extensions partial 0, planned 0; evidence pending 4

Runtime5/5

target conformance pass rate

Perm Probe4/4

0 native; 4 installer-enforced

Trust0

155 scripts scanned; secrets found

Py Compat0

234 files scanned for Python 3.11

Arch Debt0

719 largest lines; 0 watchlist; 6 early; 71 CLI handlers; 18 in entrypoint

Atlas5

11 scanned skills; route collisions

Driftlow

1 metadata events; 0 missed triggers

Daily Ops5

proposal-review; approval 0; release lock true

Weekly Queue5

curator-review; ready 1; top score 88

Waivers0

0 gates covered; human risk decisions

Intake4/4

0 valid submissions; 0 invalid

Claim Guard0

184 public surfaces scanned

Notes0/0

0 open blocker annotations

Registry1.1.0

5 targets; MIT license

Archivepass

580 zip entries; package verification

Installpass

4 adapters; 12 permissions enforced; 0 permission failures

Upgrademinor

declared minor; 0 breaking changes

+
Skill IR2.0.0

5 targets in platform-neutral contract

Compiler5/5

target contracts compiled from Skill IR

Output Delta100.0

5 cases; 1 file-backed

Exec Runs10

command 10; model 10; recorded 0

Blind A/B5

review pairs hide baseline vs with-skill labels

Review Kit0/5

pending 5; answer key hidden

Review A/B0/5

adjudication decisions; pending 5

Public Claimblocked

4 blockers; local reproducible true

Blueprint21/21

2.0 coverage; extensions partial 0, planned 0; evidence pending 4

Runtime5/5

target conformance pass rate

Perm Probe4/4

0 native; 4 installer-enforced

Trust0

155 scripts scanned; secrets found

Py Compat0

235 files scanned for Python 3.11

Arch Debt0

719 largest lines; 0 watchlist; 6 early; 71 CLI handlers; 18 in entrypoint

Atlas5

11 scanned skills; route collisions

Driftlow

1 metadata events; 0 missed triggers

Daily Ops5

proposal-review; approval 0; release lock false

Weekly Queue5

curator-review; ready 1; top score 88

Waivers0

0 gates covered; human risk decisions

Intake4/4

0 valid submissions; 0 invalid

Claim Guard0

194 public surfaces scanned

Notes0/0

0 open blocker annotations

Registry1.1.0

5 targets; MIT license

Archivepass

580 zip entries; package verification

Installpass

4 adapters; 12 permissions enforced; 0 permission failures

Upgrademinor

declared minor; 0 breaking changes

审查闸门

-
通过

意图画布

intent confidence 100/100; Intent is clear enough to package the first routeable version.

reports/intent-confidence.json 证据
通过

触发实验

13 trigger cases; 0 misroutes; 0 ambiguous

reports/route_scorecard.json 证据
关注

输出实验

5/5 cases; with-skill 100.0; baseline 0.0; file-backed 1; near-neighbor 1; blind A/B 5; exec 10; command 10; model 10; recorded 0; reviewed 0/5; review pending 5

reports/output_quality_scorecard.json 证据
通过

上下文

initial load 998/1000; deferred 530871/120000; top deferred scripts 469568; resource governance governed; quality density 140.3

reports/context_budget.json 证据
通过

运行矩阵

5 / 5 targets pass

reports/conformance_matrix.json 证据
通过

信任报告

0 secrets; 155 scripts; 3 network-capable scripts; 0 help smoke failures

reports/security_trust_report.json 证据
通过

Python 兼容

Python 3.11; 234 files; 0 compatibility issues; 0 syntax; 0 f-string 3.11 hazards

reports/python_compatibility.json 证据
通过

架构维护

232 Python files; 0 hotspots; 0 watchlist files; 6 early watch files; 0 blockers; largest 719 lines; 71 CLI handlers; 18 in entrypoint

reports/architecture_maintainability.json 证据
通过

权限批准

3/3 permissions approved; gaps 0; required file_write, network, subprocess

reports/security_trust_report.json + security/permission_policy.json 证据
通过

权限探针

4/4 targets probed; native 0; metadata fallback 4; installer 4; residual risks 4

reports/runtime_permission_probes.json 证据
通过

组合治理

11 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 0 actionable drift; 22 scoped non-actionable issues

reports/skill_atlas.json 证据
通过

运营回路

1 metadata events; adoption 100.0; missed 0; bad-output 0; risk low; daily proposals 5; daily decision proposal-review; daily release lock true; weekly queue 5 unique; weekly ready 1; weekly top 88; weekly release lock true

reports/adoption_drift_report.json + reports/skillops/daily + reports/skillops/weekly 证据
关注

人工批准

0 active waivers; 1 warning gates still need reviewer decision

reports/review_waivers.json 证据
关注

世界证据

4 pending world-class evidence entries; 1 human pending; 3 external pending; source checks 12/19 pass; 7 blocked; overclaim guard true

reports/world_class_evidence_ledger.json 证据
通过

注册审计

yao-meta-skill 1.1.0; 6/6 compatibility entries pass; install pass with 4 adapters; installer permissions 12 enforced / 0 failures

reports/registry_audit.json + reports/install_simulation.json 证据
通过

发布路线

0 promote; 3 keep current; 0 blocked; upgrade minor declared / minor recommended

reports/promotion_decisions.json + reports/upgrade_check.json + docs/migration-v2.md 证据
+
通过

意图画布

intent confidence 100/100; Intent is clear enough to package the first routeable version.

reports/intent-confidence.json 证据
通过

触发实验

13 trigger cases; 0 misroutes; 0 ambiguous

reports/route_scorecard.json 证据
关注

输出实验

5/5 cases; with-skill 100.0; baseline 0.0; file-backed 1; near-neighbor 1; blind A/B 5; exec 10; command 10; model 10; recorded 0; reviewed 0/5; review pending 5

reports/output_quality_scorecard.json 证据
通过

上下文

initial load 999/1000; deferred 531767/120000; top deferred scripts 469568; resource governance governed; quality density 140.1

reports/context_budget.json 证据
通过

运行矩阵

5 / 5 targets pass

reports/conformance_matrix.json 证据
通过

信任报告

0 secrets; 155 scripts; 3 network-capable scripts; 0 help smoke failures

reports/security_trust_report.json 证据
通过

Python 兼容

Python 3.11; 235 files; 0 compatibility issues; 0 syntax; 0 f-string 3.11 hazards

reports/python_compatibility.json 证据
通过

架构维护

232 Python files; 0 hotspots; 0 watchlist files; 6 early watch files; 0 blockers; largest 719 lines; 71 CLI handlers; 18 in entrypoint

reports/architecture_maintainability.json 证据
通过

权限批准

3/3 permissions approved; gaps 0; required file_write, network, subprocess

reports/security_trust_report.json + security/permission_policy.json 证据
通过

权限探针

4/4 targets probed; native 0; metadata fallback 4; installer 4; residual risks 4

reports/runtime_permission_probes.json 证据
通过

组合治理

11 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 0 actionable drift; 22 scoped non-actionable issues

reports/skill_atlas.json 证据
通过

运营回路

1 metadata events; adoption 100.0; missed 0; bad-output 0; risk low; daily proposals 5; daily decision proposal-review; daily release lock false; weekly queue 5 unique; weekly ready 1; weekly top 88; weekly release lock false

reports/adoption_drift_report.json + reports/skillops/daily + reports/skillops/weekly 证据
关注

人工批准

0 active waivers; 1 warning gates still need reviewer decision

reports/review_waivers.json 证据
关注

世界证据

4 pending world-class evidence entries; 1 human pending; 3 external pending; source checks 12/19 pass; 7 blocked; overclaim guard true

reports/world_class_evidence_ledger.json 证据
通过

注册审计

yao-meta-skill 1.1.0; 6/6 compatibility entries pass; install pass with 4 adapters; installer permissions 12 enforced / 0 failures

reports/registry_audit.json + reports/install_simulation.json 证据
通过

发布路线

0 promote; 3 keep current; 0 blocked; upgrade minor declared / minor recommended

reports/promotion_decisions.json + reports/upgrade_check.json + docs/migration-v2.md 证据
@@ -792,7 +792,7 @@

修复动作

-
关注

输出实验

补足 output eval 覆盖、execution evidence、blind A/B 和 reviewer adjudication。同步补足盲审声明。

没有输出质量和人工盲评证据时，Skill 只能证明会触发，不能证明输出真的更好且经得起审查。
修复位置
evals/output/cases.jsonl + reports/output_quality_scorecard.md + reports/output_review_kit.html + reports/output_review_adjudication.md + reports/output_review_decisions.json
验证命令
python3 scripts/adjudicate_output_review.py --write-template && python3 scripts/yao.py output-review
关注

人工批准

对保留的 warning 写入 reviewer、理由、范围和到期时间，或修掉 warning。

warning 可以被接受，但必须可审计、会过期，并且不能掩盖 blocker。
修复位置
reports/review_waivers.md
验证命令
python3 scripts/render_review_waivers.py .
关注

世界证据

补齐 provider、真人盲评、原生权限执行和真实客户端遥测证据，或明确本次发布不声明 world-class 完成。

世界级结论必须来自已接受的外部/人工证据；计划、metadata fallback、待评审和本地命令都不能替代完成证据。
修复位置
reports/world_class_operator_runbook.html + reports/world_class_evidence_ledger.md + reports/world_class_evidence_intake.md + reports/world_class_submission_review.md
验证命令
python3 scripts/yao.py world-class-runbook . --submissions-dir evidence/world_class/submissions && python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions && python3 scripts/yao.py review-studio .

证据采集

以下条目仍需真实外部或人工证据；提交文件、校验命令和阻断检查必须同时闭环。

pending · external

Provider Holdout

model-executed 10; token-observed 10

提交
evidence/world_class/submissions/provider-holdout.json
模板
evidence/world_class/templates/provider-holdout.intake.json
阻断
0 blocked / 3 pass
修复
1 repair rows; counts as completion: false
阶段
1 blocked phases; counts as completion: false
下一步
Run provider-backed holdout cases with real credentials and commit only aggregate evidence.
阻断检查

暂无阻断检查。

阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: operator with provider credentialsevidence: provider-holdoutSet one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
修复清单
  • #20 · unblock-access · precheckprovider-api-keyoperator with provider credentialsRequired env_any precheck is missing.Set one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key provider-holdout --output-dir evidence/world_class/submissions
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
首要步骤
  1. Set one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.
  2. For OpenAI Responses: python3 scripts/yao.py output-exec --provider-runner openai --provider-model ${YAO_OUTPUT_EVAL_MODEL:-gpt-4.1-mini} --timeout-seconds 60
  3. For DeepSeek Chat Completions: python3 scripts/yao.py output-exec --provider-runner deepseek --provider-model deepseek-v4-flash --provider-api-format chat-completions --provider-thinking disabled --api-key-env DEEPSEEK_API_KEY --timeout-seconds 120
采集契约
来源要求
  • provider-backed model run
  • observed timing
  • observed token metadata
通过条件
  • reports/output_execution_runs.json summary.model_executed_count > 0
  • reports/output_execution_runs.json summary.timing_observed_count > 0
  • reports/output_execution_runs.json summary.token_observed_count > 0
  • reports/skill_os2_audit.json item provider-holdout status becomes pass
证据资产
  • reports/output_execution_runs.json
  • reports/output_execution_runs.md
  • reports/skill_os2_audit.json
  • evidence/world_class/intake.schema.json
  • evidence/world_class/templates/provider-holdout.intake.json
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref1 / 1 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence6 / 6 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Do not commit provider credentials or environment dumps.
  • The output execution report records output hashes and aggregate run metadata, not raw provider prompts.
pending · human

Human Adjudication

0/5 decisions; pending 5

提交
evidence/world_class/submissions/human-adjudication.json
模板
evidence/world_class/templates/human-adjudication.intake.json
阻断
5 blocked / 5 pass
修复
6 repair rows; counts as completion: false
阶段
2 blocked phases; counts as completion: false
下一步
Record real A/B choices, reviewer metadata, and blind-review attestation, then regenerate adjudication.
阻断检查
  • No pending decisionspending_count: 5 / ==0Record a reviewer choice and reason for every pair.
  • Judgments completejudgment_count: 0 / ==pair_countEvery pair needs one valid human judgment.
  • Reviewer metadatareviewer_metadata_present: False / trueRecord reviewer and reviewed_at before adjudication can count.
  • Blind review attestedblind_review_attested: False / trueSet reviewer_attestation only after choices are completed before opening the answer key.
  • Human evidence readyready_for_human_evidence: False / trueComplete all reviewer decisions with metadata and rationale, plus blind-review attestation and integrity fingerprints.
阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: human reviewerevidence: human-adjudicationAssign a real reviewer identity before claiming human adjudication.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-sourceCollect source5 / 5 blockedowners: human reviewerevidence: human-adjudicationSet reviewer_attestation only after choices are completed before opening the answer key.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
修复清单
  • #20 · unblock-access · precheckhuman-reviewerhuman reviewerRequired human precheck is human-required.Assign a real reviewer identity before claiming human adjudication.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkblind_review_attestedhuman reviewerCurrent value False does not satisfy 'true'.Set reviewer_attestation only after choices are completed before opening the answer key.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkjudgment_counthuman reviewerCurrent value 0 does not satisfy '==pair_count'.Every pair needs one valid human judgment.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkpending_counthuman reviewerCurrent value 5 does not satisfy '==0'.Record a reviewer choice and reason for every pair.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkready_for_human_evidencehuman reviewerCurrent value False does not satisfy 'true'.Complete all reviewer decisions with metadata and rationale, plus blind-review attestation and integrity fingerprints.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkreviewer_metadata_presenthuman reviewerCurrent value False does not satisfy 'true'.Record reviewer and reviewed_at before adjudication can count.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key human-adjudication --output-dir evidence/world_class/submissions
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
首要步骤
  1. python3 scripts/yao.py output-review-kit --write-template
  2. Open reports/output_review_kit.md and choose A or B for each pair without opening the answer key.
  3. python3 scripts/adjudicate_output_review.py --write-template
采集契约
来源要求
  • real reviewer identity
  • blind A/B decisions
  • answer key unopened until decisions exist
通过条件
  • reports/output_review_adjudication.json summary.pending_count == 0
  • reports/output_review_adjudication.json summary.judgment_count == summary.pair_count
  • reports/output_review_adjudication.json summary.invalid_decision_count == 0
  • reports/output_review_adjudication.json summary.reviewer_metadata_present is true
  • reports/output_review_adjudication.json summary.blind_review_attested is true
  • reports/output_review_adjudication.json review_integrity.blind_pack_sha256 exists and matches reports/output_review_decisions.json
  • reports/output_review_adjudication.json pairs and reviewer_checklist store prompt_sha256, not raw prompt text
  • reports/output_review_adjudication.json summary.ready_for_human_evidence is true
  • reports/skill_os2_audit.json item human-adjudication status becomes pass
证据资产
  • reports/output_blind_review_pack.md
  • reports/output_review_kit.md
  • reports/output_review_decisions.json
  • reports/output_review_adjudication.json
  • reports/output_review_adjudication.md
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence8 / 8 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Reviewer decisions should not include raw user data or private customer detail.
  • Reviewer reasons must be rubric-based and must not include raw user data or private customer detail.
  • The decision importer rejects raw prompt, output, transcript, message, and answer-key fields.
  • The adjudication evidence stores prompt_sha256 instead of raw prompt text.
pending · external

Native Permission Enforcement

native-enforced targets 0; installer-enforced targets 4

提交
evidence/world_class/submissions/native-permission-enforcement.json
模板
evidence/world_class/templates/native-permission-enforcement.intake.json
阻断
1 blocked / 2 pass
修复
2 repair rows; counts as completion: false
阶段
2 blocked phases; counts as completion: false
下一步
Integrate a real target-client or external installer runtime guard before claiming native permission enforcement.
阻断检查
  • Native enforcementnative_enforcement_count: 0 / >0Collect real target-client or external runtime guard proof.
阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: target client or installer integratorevidence: native-permission-enforcementAttach a real target-client or external installer runtime guard; metadata fallback is not enough.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-sourceCollect source1 / 1 blockedowners: target client or installer integratorevidence: native-permission-enforcementCollect real target-client or external runtime guard proof.python3 scripts/yao.py runtime-permissions . --package-dir dist && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
修复清单
  • #20 · unblock-access · prechecknative-guardtarget client or installer integratorRequired external precheck is external-required.Attach a real target-client or external installer runtime guard; metadata fallback is not enough.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checknative_enforcement_counttarget client or installer integratorCurrent value 0 does not satisfy '>0'.Collect real target-client or external runtime guard proof.python3 scripts/yao.py runtime-permissions . --package-dir dist && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key native-permission-enforcement --output-dir evidence/world_class/submissions
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
首要步骤
  1. Implement or connect a real target client or external installer runtime guard that blocks undeclared network, file_write, or subprocess capabilities.
  2. Update the generated target adapter only when the guard is actually enforced by that target.
  3. python3 scripts/yao.py package . --platform openai --platform claude --platform generic --platform vscode --output-dir dist --zip
采集契约
来源要求
  • real target client or external installer runtime guard
  • native enforcement flag or externally accepted guard proof
  • residual risk retained for fallback targets
通过条件
  • reports/runtime_permission_probes.json summary.native_enforcement_count > 0
  • reports/runtime_permission_probes.json summary.failure_count == 0
  • reports/runtime_permission_probes.json summary.installer_enforcement_pass_count records local installer enforcement but does not replace native evidence
  • reports/skill_os2_audit.json item native-permission-enforcement status becomes pass
证据资产
  • dist/targets/*/adapter.json
  • reports/runtime_permission_probes.json
  • reports/runtime_permission_probes.md
  • reports/install_simulation.json
  • reports/install_simulation.md
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence11 / 11 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Do not mark native_enforcement true for metadata-only fallbacks.
  • Keep residual risks visible for targets that still rely on operator enforcement.
pending · external

Native Client Telemetry

external source events 0; adoption samples 1

提交
evidence/world_class/submissions/native-client-telemetry.json
模板
evidence/world_class/templates/native-client-telemetry.intake.json
阻断
1 blocked / 2 pass
修复
2 repair rows; counts as completion: false
阶段
2 blocked phases; counts as completion: false
下一步
Install a real client against the native host and import production metadata-only events.
阻断检查
  • External eventsexternal_source_events: 0 / >0Import at least one metadata-only event from a real client.
阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: Browser/Chrome/IDE/provider client integratorevidence: native-client-telemetryInstall a real Browser, Chrome, IDE, or provider client that emits metadata-only events.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-sourceCollect source1 / 1 blockedowners: Browser/Chrome/IDE/provider client integratorevidence: native-client-telemetryImport at least one metadata-only event from a real client.python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
修复清单
  • #20 · unblock-access · precheckexternal-clientBrowser/Chrome/IDE/provider client integratorRequired external precheck is external-required.Install a real Browser, Chrome, IDE, or provider client that emits metadata-only events.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkexternal_source_eventsBrowser/Chrome/IDE/provider client integratorCurrent value 0 does not satisfy '>0'.Import at least one metadata-only event from a real client.python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key native-client-telemetry --output-dir evidence/world_class/submissions
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
首要步骤
  1. python3 scripts/telemetry_native_host.py . --write-launcher /tmp/yao-telemetry-host.sh --write-manifest /tmp/yao-telemetry-host.json --allowed-origin chrome-extension://<extension-id>/
  2. Install the generated native messaging manifest for the real client and send at least one accepted skill_activation or skill_output event.
  3. python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl
采集契约
来源要求
  • real external client source
  • metadata-only event
  • local-first import path
通过条件
  • reports/adoption_drift_report.json summary.source_types.external > 0
  • reports/adoption_drift_report.json summary.adoption_sample_count > 0
  • reports/skill_os2_audit.json item native-client-telemetry status becomes pass
证据资产
  • reports/adoption_drift_report.json
  • reports/adoption_drift_report.md
  • reports/telemetry_hook_recipes.json
  • scripts/telemetry_native_host.py
  • evidence/world_class/intake.schema.json
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence6 / 6 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Telemetry must remain metadata-only and local-first.
  • Do not package reports/telemetry_events.jsonl or any raw prompt, output, transcript, note, or message field.
+
关注

输出实验

补足 output eval 覆盖、execution evidence、blind A/B 和 reviewer adjudication。同步补足盲审声明。

没有输出质量和人工盲评证据时，Skill 只能证明会触发，不能证明输出真的更好且经得起审查。
修复位置
evals/output/cases.jsonl + reports/output_quality_scorecard.md + reports/output_review_kit.html + reports/output_review_adjudication.md + reports/output_review_decisions.json
验证命令
python3 scripts/adjudicate_output_review.py --write-template && python3 scripts/yao.py output-review
关注

人工批准

对保留的 warning 写入 reviewer、理由、范围和到期时间，或修掉 warning。

warning 可以被接受，但必须可审计、会过期，并且不能掩盖 blocker。
修复位置
reports/review_waivers.md
验证命令
python3 scripts/render_review_waivers.py .
关注

世界证据

补齐 provider、真人盲评、原生权限执行和真实客户端遥测证据，或明确本次发布不声明 world-class 完成。

世界级结论必须来自已接受的外部/人工证据；计划、metadata fallback、待评审和本地命令都不能替代完成证据。
修复位置
reports/world_class_operator_runbook.html + reports/world_class_evidence_ledger.md + reports/world_class_evidence_intake.md + reports/world_class_submission_review.md
验证命令
python3 scripts/yao.py world-class-runbook . --submissions-dir evidence/world_class/submissions && python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions && python3 scripts/yao.py review-studio .

证据采集

以下条目仍需真实外部或人工证据；提交文件、校验命令和阻断检查必须同时闭环。

pending · external

Provider Holdout

model-executed 10; token-observed 10

提交
evidence/world_class/submissions/provider-holdout.json
模板
evidence/world_class/templates/provider-holdout.intake.json
阻断
0 blocked / 3 pass
修复
1 repair rows; counts as completion: false
阶段
1 blocked phases; counts as completion: false
下一步
Run provider-backed holdout cases with real credentials and commit only aggregate evidence.
阻断检查

暂无阻断检查。

阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: operator with provider credentialsevidence: provider-holdoutSet one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
修复清单
  • #20 · unblock-access · precheckprovider-api-keyoperator with provider credentialsRequired env_any precheck is missing.Set one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key provider-holdout --output-dir evidence/world_class/submissions
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
首要步骤
  1. Set one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.
  2. For OpenAI Responses: python3 scripts/yao.py output-exec --provider-runner openai --provider-model ${YAO_OUTPUT_EVAL_MODEL:-gpt-4.1-mini} --timeout-seconds 60
  3. For DeepSeek Chat Completions: python3 scripts/yao.py output-exec --provider-runner deepseek --provider-model deepseek-v4-flash --provider-api-format chat-completions --provider-thinking disabled --api-key-env DEEPSEEK_API_KEY --timeout-seconds 120
采集契约
来源要求
  • provider-backed model run
  • observed timing
  • observed token metadata
通过条件
  • reports/output_execution_runs.json summary.model_executed_count > 0
  • reports/output_execution_runs.json summary.timing_observed_count > 0
  • reports/output_execution_runs.json summary.token_observed_count > 0
  • reports/skill_os2_audit.json item provider-holdout status becomes pass
证据资产
  • reports/output_execution_runs.json
  • reports/output_execution_runs.md
  • reports/skill_os2_audit.json
  • evidence/world_class/intake.schema.json
  • evidence/world_class/templates/provider-holdout.intake.json
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref1 / 1 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence6 / 6 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Do not commit provider credentials or environment dumps.
  • The output execution report records output hashes and aggregate run metadata, not raw provider prompts.
pending · human

Human Adjudication

0/5 decisions; pending 5

提交
evidence/world_class/submissions/human-adjudication.json
模板
evidence/world_class/templates/human-adjudication.intake.json
阻断
5 blocked / 5 pass
修复
6 repair rows; counts as completion: false
阶段
2 blocked phases; counts as completion: false
下一步
Record real A/B choices, reviewer metadata, and blind-review attestation, then regenerate adjudication.
阻断检查
  • No pending decisionspending_count: 5 / ==0Record a reviewer choice and reason for every pair.
  • Judgments completejudgment_count: 0 / ==pair_countEvery pair needs one valid human judgment.
  • Reviewer metadatareviewer_metadata_present: False / trueRecord reviewer and reviewed_at before adjudication can count.
  • Blind review attestedblind_review_attested: False / trueSet reviewer_attestation only after choices are completed before opening the answer key.
  • Human evidence readyready_for_human_evidence: False / trueComplete all reviewer decisions with metadata and rationale, plus blind-review attestation and integrity fingerprints.
阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: human reviewerevidence: human-adjudicationAssign a real reviewer identity before claiming human adjudication.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-sourceCollect source5 / 5 blockedowners: human reviewerevidence: human-adjudicationSet reviewer_attestation only after choices are completed before opening the answer key.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
修复清单
  • #20 · unblock-access · precheckhuman-reviewerhuman reviewerRequired human precheck is human-required.Assign a real reviewer identity before claiming human adjudication.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkblind_review_attestedhuman reviewerCurrent value False does not satisfy 'true'.Set reviewer_attestation only after choices are completed before opening the answer key.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkjudgment_counthuman reviewerCurrent value 0 does not satisfy '==pair_count'.Every pair needs one valid human judgment.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkpending_counthuman reviewerCurrent value 5 does not satisfy '==0'.Record a reviewer choice and reason for every pair.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkready_for_human_evidencehuman reviewerCurrent value False does not satisfy 'true'.Complete all reviewer decisions with metadata and rationale, plus blind-review attestation and integrity fingerprints.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkreviewer_metadata_presenthuman reviewerCurrent value False does not satisfy 'true'.Record reviewer and reviewed_at before adjudication can count.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key human-adjudication --output-dir evidence/world_class/submissions
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
首要步骤
  1. python3 scripts/yao.py output-review-kit --write-template
  2. Open reports/output_review_kit.md and choose A or B for each pair without opening the answer key.
  3. python3 scripts/adjudicate_output_review.py --write-template
采集契约
来源要求
  • real reviewer identity
  • blind A/B decisions
  • answer key unopened until decisions exist
通过条件
  • reports/output_review_adjudication.json summary.pending_count == 0
  • reports/output_review_adjudication.json summary.judgment_count == summary.pair_count
  • reports/output_review_adjudication.json summary.invalid_decision_count == 0
  • reports/output_review_adjudication.json summary.reviewer_metadata_present is true
  • reports/output_review_adjudication.json summary.blind_review_attested is true
  • reports/output_review_adjudication.json review_integrity.blind_pack_sha256 exists and matches reports/output_review_decisions.json
  • reports/output_review_adjudication.json pairs and reviewer_checklist store prompt_sha256, not raw prompt text
  • reports/output_review_adjudication.json summary.ready_for_human_evidence is true
  • reports/skill_os2_audit.json item human-adjudication status becomes pass
证据资产
  • reports/output_blind_review_pack.md
  • reports/output_review_kit.md
  • reports/output_review_decisions.json
  • reports/output_review_adjudication.json
  • reports/output_review_adjudication.md
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence8 / 8 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Reviewer decisions should not include raw user data or private customer detail.
  • Reviewer reasons must be rubric-based and must not include raw user data or private customer detail.
  • The decision importer rejects raw prompt, output, transcript, message, and answer-key fields.
  • The adjudication evidence stores prompt_sha256 instead of raw prompt text.
pending · external

Native Permission Enforcement

native-enforced targets 0; installer-enforced targets 4

提交
evidence/world_class/submissions/native-permission-enforcement.json
模板
evidence/world_class/templates/native-permission-enforcement.intake.json
阻断
1 blocked / 2 pass
修复
2 repair rows; counts as completion: false
阶段
2 blocked phases; counts as completion: false
下一步
Integrate a real target-client or external installer runtime guard before claiming native permission enforcement.
阻断检查
  • Native enforcementnative_enforcement_count: 0 / >0Collect real target-client or external runtime guard proof.
阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: target client or installer integratorevidence: native-permission-enforcementAttach a real target-client or external installer runtime guard; metadata fallback is not enough.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-sourceCollect source1 / 1 blockedowners: target client or installer integratorevidence: native-permission-enforcementCollect real target-client or external runtime guard proof.python3 scripts/yao.py runtime-permissions . --package-dir dist && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
修复清单
  • #20 · unblock-access · prechecknative-guardtarget client or installer integratorRequired external precheck is external-required.Attach a real target-client or external installer runtime guard; metadata fallback is not enough.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checknative_enforcement_counttarget client or installer integratorCurrent value 0 does not satisfy '>0'.Collect real target-client or external runtime guard proof.python3 scripts/yao.py runtime-permissions . --package-dir dist && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key native-permission-enforcement --output-dir evidence/world_class/submissions
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
首要步骤
  1. Implement or connect a real target client or external installer runtime guard that blocks undeclared network, file_write, or subprocess capabilities.
  2. Update the generated target adapter only when the guard is actually enforced by that target.
  3. python3 scripts/yao.py package . --platform openai --platform claude --platform generic --platform vscode --output-dir dist --zip
采集契约
来源要求
  • real target client or external installer runtime guard
  • native enforcement flag or externally accepted guard proof
  • residual risk retained for fallback targets
通过条件
  • reports/runtime_permission_probes.json summary.native_enforcement_count > 0
  • reports/runtime_permission_probes.json summary.failure_count == 0
  • reports/runtime_permission_probes.json summary.installer_enforcement_pass_count records local installer enforcement but does not replace native evidence
  • reports/skill_os2_audit.json item native-permission-enforcement status becomes pass
证据资产
  • dist/targets/*/adapter.json
  • reports/runtime_permission_probes.json
  • reports/runtime_permission_probes.md
  • reports/install_simulation.json
  • reports/install_simulation.md
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence7 / 8 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Do not mark native_enforcement true for metadata-only fallbacks.
  • Keep residual risks visible for targets that still rely on operator enforcement.
pending · external

Native Client Telemetry

external source events 0; adoption samples 1

提交
evidence/world_class/submissions/native-client-telemetry.json
模板
evidence/world_class/templates/native-client-telemetry.intake.json
阻断
1 blocked / 2 pass
修复
2 repair rows; counts as completion: false
阶段
2 blocked phases; counts as completion: false
下一步
Install a real client against the native host and import production metadata-only events.
阻断检查
  • External eventsexternal_source_events: 0 / >0Import at least one metadata-only event from a real client.
阶段队列
  • #20 · unblock-accessUnblock access1 / 1 blockedowners: Browser/Chrome/IDE/provider client integratorevidence: native-client-telemetryInstall a real Browser, Chrome, IDE, or provider client that emits metadata-only events.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-sourceCollect source1 / 1 blockedowners: Browser/Chrome/IDE/provider client integratorevidence: native-client-telemetryImport at least one metadata-only event from a real client.python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
修复清单
  • #20 · unblock-access · precheckexternal-clientBrowser/Chrome/IDE/provider client integratorRequired external precheck is external-required.Install a real Browser, Chrome, IDE, or provider client that emits metadata-only events.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
  • #40 · collect-source · source-checkexternal_source_eventsBrowser/Chrome/IDE/provider client integratorCurrent value 0 does not satisfy '>0'.Import at least one metadata-only event from a real client.python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
操作命令
  • 准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key native-client-telemetry --output-dir evidence/world_class/submissions
  • 校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
  • 审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
  • 刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
首要步骤
  1. python3 scripts/telemetry_native_host.py . --write-launcher /tmp/yao-telemetry-host.sh --write-manifest /tmp/yao-telemetry-host.json --allowed-origin chrome-extension://<extension-id>/
  2. Install the generated native messaging manifest for the real client and send at least one accepted skill_activation or skill_output event.
  3. python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl
采集契约
来源要求
  • real external client source
  • metadata-only event
  • local-first import path
通过条件
  • reports/adoption_drift_report.json summary.source_types.external > 0
  • reports/adoption_drift_report.json summary.adoption_sample_count > 0
  • reports/skill_os2_audit.json item native-client-telemetry status becomes pass
证据资产
  • reports/adoption_drift_report.json
  • reports/adoption_drift_report.md
  • reports/telemetry_hook_recipes.json
  • scripts/telemetry_native_host.py
  • evidence/world_class/intake.schema.json
资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

  • submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
  • supporting-evidence6 / 6 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.
隐私边界
  • Telemetry must remain metadata-only and local-first.
  • Do not package reports/telemetry_events.jsonl or any raw prompt, output, transcript, note, or message field.
@@ -808,7 +808,7 @@

证据路径

-
  • skill_overviewreports/skill-overview.html
  • review_viewerreports/review-viewer.html
  • output_evalreports/output_quality_scorecard.md
  • output_executionreports/output_execution_runs.md
  • output_blind_reviewreports/output_blind_review_pack.md
  • output_review_kitreports/output_review_kit.md
  • output_review_kit_htmlreports/output_review_kit.html
  • output_review_decisionsreports/output_review_decisions.json
  • output_review_adjudicationreports/output_review_adjudication.md
  • benchmark_reproducibilityreports/benchmark_reproducibility.md
  • skill_os2_coveragereports/skill_os2_coverage.md
  • runtime_conformancereports/conformance_matrix.md
  • trust_reportreports/security_trust_report.md
  • python_compatibilityreports/python_compatibility.md
  • architecture_maintainabilityreports/architecture_maintainability.md
  • permission_policysecurity/permission_policy.md
  • runtime_permissionsreports/runtime_permission_probes.md
  • skill_atlasreports/skill_atlas.html
  • compiled_targetsreports/compiled_targets.md
  • adoption_driftreports/adoption_drift_report.md
  • review_waiversreports/review_waivers.md
  • review_annotationsreports/review_annotations.md
  • adaptation_proposalsreports/adaptation_proposals.md
  • adaptation_approval_ledgerreports/adaptation_approval_ledger.json
  • adaptation_regressionreports/adaptation_regression_report.md
  • world_class_evidence_planreports/world_class_evidence_plan.md
  • world_class_evidence_ledgerreports/world_class_evidence_ledger.md
  • world_class_evidence_intakereports/world_class_evidence_intake.md
  • world_class_evidence_preflightreports/world_class_evidence_preflight.md
  • world_class_evidence_preflight_htmlreports/world_class_evidence_preflight.html
  • world_class_submission_reviewreports/world_class_submission_review.md
  • world_class_operator_runbookreports/world_class_operator_runbook.md
  • world_class_operator_runbook_htmlreports/world_class_operator_runbook.html
  • world_class_claim_guardreports/world_class_claim_guard.md
  • registry_auditreports/registry_audit.md
  • package_verificationreports/package_verification.md
  • install_simulationreports/install_simulation.md
  • upgrade_checkreports/upgrade_check.md
  • migrationdocs/migration-v2.md
  • skill_irskill-ir/examples/yao-meta-skill.json
  • daily_skillopsreports/skillops/daily/2026-06-21.md
  • weekly_curatorreports/skillops/weekly/2026-W25.md
+
  • skill_overviewreports/skill-overview.html
  • review_viewerreports/review-viewer.html
  • output_evalreports/output_quality_scorecard.md
  • output_executionreports/output_execution_runs.md
  • output_blind_reviewreports/output_blind_review_pack.md
  • output_review_kitreports/output_review_kit.md
  • output_review_kit_htmlreports/output_review_kit.html
  • output_review_decisionsreports/output_review_decisions.json
  • output_review_adjudicationreports/output_review_adjudication.md
  • benchmark_reproducibilityreports/benchmark_reproducibility.md
  • skill_os2_coveragereports/skill_os2_coverage.md
  • runtime_conformancereports/conformance_matrix.md
  • trust_reportreports/security_trust_report.md
  • python_compatibilityreports/python_compatibility.md
  • architecture_maintainabilityreports/architecture_maintainability.md
  • permission_policysecurity/permission_policy.md
  • runtime_permissionsreports/runtime_permission_probes.md
  • skill_atlasreports/skill_atlas.html
  • compiled_targetsreports/compiled_targets.md
  • adoption_driftreports/adoption_drift_report.md
  • review_waiversreports/review_waivers.md
  • review_annotationsreports/review_annotations.md
  • adaptation_proposalsreports/adaptation_proposals.md
  • adaptation_approval_ledgerreports/adaptation_approval_ledger.json
  • adaptation_regressionreports/adaptation_regression_report.md
  • world_class_evidence_planreports/world_class_evidence_plan.md
  • world_class_evidence_ledgerreports/world_class_evidence_ledger.md
  • world_class_evidence_intakereports/world_class_evidence_intake.md
  • world_class_evidence_preflightreports/world_class_evidence_preflight.md
  • world_class_evidence_preflight_htmlreports/world_class_evidence_preflight.html
  • world_class_submission_reviewreports/world_class_submission_review.md
  • world_class_operator_runbookreports/world_class_operator_runbook.md
  • world_class_operator_runbook_htmlreports/world_class_operator_runbook.html
  • world_class_claim_guardreports/world_class_claim_guard.md
  • registry_auditreports/registry_audit.md
  • package_verificationreports/package_verification.md
  • install_simulationreports/install_simulation.md
  • upgrade_checkreports/upgrade_check.md
  • migrationdocs/migration-v2.md
  • skill_irskill-ir/examples/yao-meta-skill.json
  • daily_skillopsreports/skillops/daily/2026-07-02.md
  • weekly_curatorreports/skillops/weekly/2026-W27.md
@@ -844,17 +844,17 @@
-

上下文

initial load 998/1000; deferred 530871/120000; top deferred scripts 469568; resource governance governed; quality density 140.3

+

上下文

initial load 999/1000; deferred 531767/120000; top deferred scripts 469568; resource governance governed; quality density 140.1

编译证据

Review reports/compiled_targets.md before packaging to inspect target adapter modes, generated files, preserved semantics, warnings, and unsupported features.

-

信任报告

Secret
0
脚本数
155
网络脚本
3
Help 失败
0
包体哈希
eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383
+

信任报告

Secret
0
脚本数
155
网络脚本
3
Help 失败
0
包体哈希
e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3

安全边界

高风险 secret、远程 inline execution、缺失依赖策略或无法解释的脚本接口应阻断 governed release。

-

Python 兼容

目标 Python
3.11
文件数
234
问题数
0
语法错误
0
F-string 3.11
0
+

Python 兼容

目标 Python
3.11
文件数
235
问题数
0
语法错误
0
F-string 3.11
0

解释器边界

CI 和发布审查以 Python 3.11 兼容为底线；本地更高版本允许的新语法不能绕过兼容门禁。

@@ -874,14 +874,14 @@
-

运营回路

1 metadata events; adoption 100.0; missed 0; bad-output 0; risk low; daily proposals 5; daily decision proposal-review; daily release lock true; weekly queue 5 unique; weekly ready 1; weekly top 88; weekly release lock true

+

运营回路

1 metadata events; adoption 100.0; missed 0; bad-output 0; risk low; daily proposals 5; daily decision proposal-review; daily release lock false; weekly queue 5 unique; weekly ready 1; weekly top 88; weekly release lock false

漂移信号

事件数
1
采用率
100
漏触发
0
Bad Output Count
0
风险带
low
-

日常运维

决策
proposal-review
提案数
5
Approval Count
0
Pending Review Count
0
发布锁
是
世界级
否
Writes Source Files
否
Auto Patch Enabled
否
-

周度队列

决策
curator-review
Week Id
2026-W25
日报数
4
唯一机会
5
待批准复核
1
提案复核
4
Top Score
88
发布锁
是
Writes Source Files
否
Auto Patch Enabled
否
+

日常运维

决策
proposal-review
提案数
5
Approval Count
0
Pending Review Count
0
发布锁
否
世界级
否
Writes Source Files
否
Auto Patch Enabled
否
+

周度队列

决策
curator-review
Week Id
2026-W27
日报数
5
唯一机会
5
待批准复核
1
提案复核
4
Top Score
88
发布锁
否
Writes Source Files
否
Auto Patch Enabled
否
@@ -914,8 +914,8 @@
-

公开声明

本地复现
是
发布锁
是
可公开声明
否
声明阻断
3
Provider 证据
是
人审完成
否
世界级就绪
否
-

声明阻断

  • 阻断human blind-review adjudication is incomplete
  • 阻断world-class evidence is not accepted yet (3 open gaps, 4 ledger pending)
  • 阻断world-class source checks are not all accepted (12/19 pass, 7 blocked)
+

公开声明

本地复现
是
发布锁
否
可公开声明
否
声明阻断
4
Provider 证据
是
人审完成
否
世界级就绪
否
+

声明阻断

  • 阻断release lock is not clean or commit is unavailable
  • 阻断human blind-review adjudication is incomplete
  • 阻断world-class evidence is not accepted yet (3 open gaps, 4 ledger pending)
  • 阻断world-class source checks are not all accepted (12/19 pass, 7 blocked)
@@ -929,7 +929,7 @@
-

声明守卫

台账可声明
否
台账待补
4
声明面
184
违规数
0
Overclaim Guard Active
是
+

声明守卫

台账可声明
否
台账待补
4
声明面
194
违规数
0
Overclaim Guard Active
是

声明边界

claim guard 扫描 README、docs 和 reports 中的完成态表述；ledger 未 ready 时，任何英文完成断言、true 状态声明或中文完成态都会阻断发布审查。

diff --git a/reports/review-studio.json b/reports/review-studio.json index c880ae11..0241ab3b 100644 --- a/reports/review-studio.json +++ b/reports/review-studio.json @@ -65,7 +65,7 @@ "key": "context-budget", "label": "上下文", "status": "pass", - "detail": "initial load 998/1000; deferred 530871/120000; top deferred scripts 469568; resource governance governed; quality density 140.3", + "detail": "initial load 999/1000; deferred 531767/120000; top deferred scripts 469568; resource governance governed; quality density 140.1", "evidence": "reports/context_budget.json", "link": "context_budget.md", "review_action_id": "", @@ -110,7 +110,7 @@ "key": "python-compat", "label": "Python 兼容", "status": "pass", - "detail": "Python 3.11; 234 files; 0 compatibility issues; 0 syntax; 0 f-string 3.11 hazards", + "detail": "Python 3.11; 235 files; 0 compatibility issues; 0 syntax; 0 f-string 3.11 hazards", "evidence": "reports/python_compatibility.json", "link": "python_compatibility.md", "review_action_id": "", @@ -185,7 +185,7 @@ "key": "operations-loop", "label": "运营回路", "status": "pass", - "detail": "1 metadata events; adoption 100.0; missed 0; bad-output 0; risk low; daily proposals 5; daily decision proposal-review; daily release lock true; weekly queue 5 unique; weekly ready 1; weekly top 88; weekly release lock true", + "detail": "1 metadata events; adoption 100.0; missed 0; bad-output 0; risk low; daily proposals 5; daily decision proposal-review; daily release lock false; weekly queue 5 unique; weekly ready 1; weekly top 88; weekly release lock false", "evidence": "reports/adoption_drift_report.json + reports/skillops/daily + reports/skillops/weekly", "link": "adoption_drift_report.md", "review_action_id": "", @@ -1194,8 +1194,8 @@ "artifact_prefill_counts_as_evidence": false, "submission_ref_ready_count": 2, "submission_ref_total_count": 2, - "supporting_evidence_ready_count": 11, - "supporting_evidence_total_count": 11, + "supporting_evidence_ready_count": 7, + "supporting_evidence_total_count": 8, "roles": [ { "role": "submission-ref", @@ -1208,8 +1208,8 @@ { "role": "supporting-evidence", "label": "Supporting evidence", - "ready_count": 11, - "total_count": 11, + "ready_count": 7, + "total_count": 8, "copy_to_artifact_refs": false, "description": "Supporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs." } @@ -1437,8 +1437,8 @@ "upgrade_check": "reports/upgrade_check.md", "migration": "docs/migration-v2.md", "skill_ir": "skill-ir/examples/yao-meta-skill.json", - "daily_skillops": "reports/skillops/daily/2026-06-21.md", - "weekly_curator": "reports/skillops/weekly/2026-W25.md" + "daily_skillops": "reports/skillops/daily/2026-07-02.md", + "weekly_curator": "reports/skillops/weekly/2026-W27.md" }, "data": { "overview": { @@ -1446,7 +1446,7 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "display_name": "Yao Meta Skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "maturity": "governed", "updated_at": "2026-03-31", "core_value": "把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。", @@ -1523,7 +1523,7 @@ "label": "可维护性", "score": 100, "reasons": [ - "SKILL.md 约 339 个词/字。", + "SKILL.md 约 342 个词/字。", "入口文件保持克制，可维护性较好。", "references/ 已承载扩展指导。", "scripts/ 已承载确定性逻辑。", @@ -1544,7 +1544,7 @@ "label": "上下文成本", "score": 42, "reasons": [ - "入口约 339 个词/字，references 约 16934 个词/字。", + "入口约 342 个词/字，references 约 16934 个词/字。", "分数越高代表上下文成本越低。", "上下文成本偏高，建议压缩入口或拆分 references。" ] @@ -1598,7 +1598,7 @@ }, "contract_boundary": { "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "activation": "manual", "execution": "inline", "shell": "bash" @@ -1626,7 +1626,7 @@ "boundary_cards": [ { "label": "Owned", - "body": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "body": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." }, { "label": "Adjacent", @@ -1648,7 +1648,7 @@ "已生成 Output Review Adjudication，可记录盲评决策、一致率和待评审项。" ], "gaps": [ - "上下文成本需要补强：入口约 339 个词/字，references 约 16934 个词/字。" + "上下文成本需要补强：入口约 342 个词/字，references 约 16934 个词/字。" ], "recommendations": [ "先改触发边界，再扩展工作流。", @@ -1713,7 +1713,7 @@ "name": "包体膨胀风险", "impact": 2, "probability": 1, - "signal": "SKILL.md 约 339 个词/字。", + "signal": "SKILL.md 约 342 个词/字。", "response": "先补证据和边界，再增加包体复杂度。" }, { @@ -1964,10 +1964,10 @@ "path": "reports", "label": "Generated evidence and overview artifacts", "kind": "folder", - "file_count": 152 + "file_count": 170 } ], - "file_count": 375, + "file_count": 393, "folder_count": 4, "distribution": [ { @@ -2000,7 +2000,7 @@ }, { "label": "reports", - "value": 152 + "value": 170 } ] }, @@ -2075,7 +2075,7 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "display_name": "Yao Meta Skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "logic_steps": [ "For one-off/no reusable process: `Do not create a skill`; `near-neighbor`; require `repeated use` + `reusable output contract`.", "Capture job, output, exclusions, constraints, standards, and the lightest fit.", @@ -2085,7 +2085,7 @@ ], "usage_steps": [ "Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.", - "Use this skill when the request matches: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "Use this skill when the request matches: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." ], "package_map": [ { @@ -2134,7 +2134,7 @@ "path": "reports", "label": "Generated evidence and overview artifacts", "kind": "folder", - "file_count": 152 + "file_count": 170 } ], "strengths": [ @@ -2152,7 +2152,7 @@ "用户说出类似需求时：Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals." ], "trigger_contract": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "activation": "manual", "execution": "inline", "shell": "bash" @@ -2454,12 +2454,12 @@ "ok": true, "summary": { "reproducibility_ready": true, - "release_lock_ready": true, + "release_lock_ready": false, "methodology_complete": true, "required_artifact_count": 25, "missing_artifact_count": 0, - "evidence_bundle_sha256": "e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64cbe8db9ad", - "source_contract_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "evidence_bundle_sha256": "639707e807e5ec8460f6aa6d1e5ba34874bb3228160555d8a01dbc37620d4873", + "source_contract_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7", "output_case_count": 5, "failure_disclosure_count": 3, @@ -2477,19 +2477,19 @@ "world_class_source_check_count": 19, "world_class_source_pass_count": 12, "world_class_source_blocked_count": 7, - "beta_test_ready": true, - "beta_test_blocker_count": 0, + "beta_test_ready": false, + "beta_test_blocker_count": 1, "beta_test_deferred_evidence_count": 4, "public_claim_ready": false, - "public_claim_blocker_count": 3, + "public_claim_blocker_count": 4, "working_tree_dirty": true, - "changed_file_count": 3, - "source_tree_dirty": false, - "source_changed_file_count": 0, + "changed_file_count": 84, + "source_tree_dirty": true, + "source_changed_file_count": 2, "generated_tree_dirty": true, - "generated_changed_file_count": 3 + "generated_changed_file_count": 82 }, - "commit": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", + "commit": "1710a5a3e0177313954758908735c147796e8981", "missing_artifacts": [], "limitations": [ "The git commit and dirty flags are generation-time context; release lock is blocked by source changes, while generated evidence artifacts are tracked separately.", @@ -2595,7 +2595,7 @@ "interactive_script_count": 0, "package_hash_scope": "source-contract-without-generated-reports", "package_hash_file_count": 247, - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" }, "skill_atlas": { "skill_count": 11, @@ -2619,7 +2619,7 @@ "schema_version": "2.0", "name": "yao-meta-skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "targets": [ "openai", "claude", @@ -2633,7 +2633,7 @@ "trust_level": "local", "license": "MIT", "checksums": { - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7" }, "compatibility": { @@ -2670,7 +2670,7 @@ "install_simulated": true, "install_simulation": "reports/install_simulation.json" }, - "generated_at": "2026-06-21" + "generated_at": "2026-06-13" }, "failures": [], "warnings": [] @@ -2694,7 +2694,7 @@ "ok": true, "summary": { "archive_present": true, - "archive_entry_count": 580, + "archive_entry_count": 598, "nested_skill_entry_count": 0, "archive_extracted": true, "entrypoint_loaded": true, @@ -2755,7 +2755,7 @@ { "field": "description", "from": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes.", - "to": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "to": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." } ], "checksum_changes": [ @@ -2767,7 +2767,7 @@ { "field": "package_sha256", "from": "0000000000000000000000000000000000000000000000000000000000000000", - "to": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "to": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" } ] }, @@ -3661,7 +3661,7 @@ }, "synthesis_highlights": [ "Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.", - "Borrow a review checkpoint wherever trust matters more than raw speed.", + "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", "Borrow the discipline of defining what the skill should not own before growing the package." ], "artifact_design": { @@ -3856,7 +3856,7 @@ "correct_count": 3, "precision": 1.0, "recall": 1.0, - "average_margin": 0.701 + "average_margin": 0.644 }, "team-frontend-review": { "expected_count": 3, @@ -3872,7 +3872,7 @@ "correct_count": 3, "precision": 1.0, "recall": 1.0, - "average_margin": 0.586 + "average_margin": 0.709 }, "no_route": { "expected_count": 4, @@ -3918,18 +3918,18 @@ "expected_route": "yao-meta-skill", "predicted_route": "yao-meta-skill", "correct": true, - "margin": 1.0, + "margin": 0.681, "ambiguous": false, "ranked_routes": [ { "name": "yao-meta-skill", - "score": 1.0, + "score": 0.681, "passed_threshold": true, "threshold": 0.33, "matched_desired_concepts": [ "eval_optimize", + "iterate_existing_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], "matched_negative_concepts": [] @@ -3958,16 +3958,17 @@ "expected_route": "yao-meta-skill", "predicted_route": "yao-meta-skill", "correct": true, - "margin": 0.661, + "margin": 0.51, "ambiguous": false, "ranked_routes": [ { "name": "yao-meta-skill", - "score": 0.661, + "score": 0.51, "passed_threshold": true, "threshold": 0.33, "matched_desired_concepts": [ "eval_optimize", + "iterate_existing_skill", "reuse_package" ], "matched_negative_concepts": [] @@ -3996,16 +3997,19 @@ "expected_route": "yao-meta-skill", "predicted_route": "yao-meta-skill", "correct": true, - "margin": 0.441, + "margin": 0.74, "ambiguous": false, "ranked_routes": [ { "name": "yao-meta-skill", - "score": 0.441, + "score": 0.74, "passed_threshold": true, "threshold": 0.33, "matched_desired_concepts": [ - "reuse_package" + "build_skill", + "multi_asset", + "reuse_package", + "transform_workflow" ], "matched_negative_concepts": [] }, @@ -4052,14 +4056,6 @@ ], "matched_negative_concepts": [] }, - { - "name": "yao-meta-skill", - "score": 0.012, - "passed_threshold": false, - "threshold": 0.33, - "matched_desired_concepts": [], - "matched_negative_concepts": [] - }, { "name": "governed-incident-command", "score": 0.012, @@ -4067,6 +4063,14 @@ "threshold": 0.48, "matched_desired_concepts": [], "matched_negative_concepts": [] + }, + { + "name": "yao-meta-skill", + "score": 0.006, + "passed_threshold": false, + "threshold": 0.33, + "matched_desired_concepts": [], + "matched_negative_concepts": [] } ] }, @@ -4092,14 +4096,6 @@ ], "matched_negative_concepts": [] }, - { - "name": "yao-meta-skill", - "score": 0.008, - "passed_threshold": false, - "threshold": 0.33, - "matched_desired_concepts": [], - "matched_negative_concepts": [] - }, { "name": "governed-incident-command", "score": 0.008, @@ -4107,6 +4103,14 @@ "threshold": 0.48, "matched_desired_concepts": [], "matched_negative_concepts": [] + }, + { + "name": "yao-meta-skill", + "score": 0.004, + "passed_threshold": false, + "threshold": 0.33, + "matched_desired_concepts": [], + "matched_negative_concepts": [] } ] }, @@ -4134,7 +4138,7 @@ }, { "name": "yao-meta-skill", - "score": 0.012, + "score": 0.006, "passed_threshold": false, "threshold": 0.33, "matched_desired_concepts": [], @@ -4172,14 +4176,6 @@ ], "matched_negative_concepts": [] }, - { - "name": "yao-meta-skill", - "score": 0.016, - "passed_threshold": false, - "threshold": 0.33, - "matched_desired_concepts": [], - "matched_negative_concepts": [] - }, { "name": "team-frontend-review", "score": 0.011, @@ -4187,6 +4183,14 @@ "threshold": 0.48, "matched_desired_concepts": [], "matched_negative_concepts": [] + }, + { + "name": "yao-meta-skill", + "score": 0.005, + "passed_threshold": false, + "threshold": 0.33, + "matched_desired_concepts": [], + "matched_negative_concepts": [] } ] }, @@ -4212,7 +4216,7 @@ }, { "name": "yao-meta-skill", - "score": 0.328, + "score": 0.179, "passed_threshold": false, "threshold": 0.33, "matched_desired_concepts": [ @@ -4236,7 +4240,7 @@ "expected_route": "governed-incident-command", "predicted_route": "governed-incident-command", "correct": true, - "margin": 0.252, + "margin": 0.621, "ambiguous": false, "ranked_routes": [ { @@ -4254,12 +4258,11 @@ }, { "name": "yao-meta-skill", - "score": 0.748, + "score": 0.379, "passed_threshold": true, "threshold": 0.33, "matched_desired_concepts": [ "reuse_package", - "team_operationalize", "transform_workflow" ], "matched_negative_concepts": [] @@ -4322,14 +4325,6 @@ "margin": null, "ambiguous": false, "ranked_routes": [ - { - "name": "yao-meta-skill", - "score": 0.008, - "passed_threshold": false, - "threshold": 0.33, - "matched_desired_concepts": [], - "matched_negative_concepts": [] - }, { "name": "governed-incident-command", "score": 0.008, @@ -4338,6 +4333,14 @@ "matched_desired_concepts": [], "matched_negative_concepts": [] }, + { + "name": "yao-meta-skill", + "score": 0.0, + "passed_threshold": false, + "threshold": 0.33, + "matched_desired_concepts": [], + "matched_negative_concepts": [] + }, { "name": "team-frontend-review", "score": 0.0, @@ -4363,7 +4366,7 @@ "ranked_routes": [ { "name": "yao-meta-skill", - "score": 0.024, + "score": 0.016, "passed_threshold": false, "threshold": 0.33, "matched_desired_concepts": [], @@ -6313,27 +6316,48 @@ "benchmark_reproducibility": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02", "skill_dir": ".", - "commit": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", + "commit": "1710a5a3e0177313954758908735c147796e8981", "git_status": { "available": true, "dirty": true, - "changed_file_count": 3, + "changed_file_count": 84, "generated_dirty": true, - "generated_changed_file_count": 3, - "source_dirty": false, - "source_changed_file_count": 0, + "generated_changed_file_count": 82, + "source_dirty": true, + "source_changed_file_count": 2, "sample": [ - " M reports/context_budget.json", - " M reports/context_budget.md", - " M reports/context_budget_summary.json" + " M SKILL.md", + " M evals/improved_description.txt", + " M registry/index.json", + " M registry/packages/yao-meta-skill.json", + " M reports/adaptation_proposals.json", + " M reports/adaptation_proposals.md", + " M reports/adoption_drift_report.json", + " M reports/artifact-design-profile.json", + " M reports/artifact-design-profile.md", + " M reports/baseline-compare.json", + " M reports/benchmark_reproducibility.json", + " M reports/benchmark_reproducibility.md" + ], + "source_sample": [ + " M SKILL.md", + " M evals/improved_description.txt" ], - "source_sample": [], "generated_sample": [ - " M reports/context_budget.json", - " M reports/context_budget.md", - " M reports/context_budget_summary.json" + " M registry/index.json", + " M registry/packages/yao-meta-skill.json", + " M reports/adaptation_proposals.json", + " M reports/adaptation_proposals.md", + " M reports/adoption_drift_report.json", + " M reports/artifact-design-profile.json", + " M reports/artifact-design-profile.md", + " M reports/baseline-compare.json", + " M reports/benchmark_reproducibility.json", + " M reports/benchmark_reproducibility.md", + " M reports/candidate_registry.json", + " M reports/candidate_registry.md" ], "generated_dirty_prefixes": [ "dist/", @@ -6347,12 +6371,12 @@ }, "summary": { "reproducibility_ready": true, - "release_lock_ready": true, + "release_lock_ready": false, "methodology_complete": true, "required_artifact_count": 25, "missing_artifact_count": 0, - "evidence_bundle_sha256": "e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64cbe8db9ad", - "source_contract_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "evidence_bundle_sha256": "639707e807e5ec8460f6aa6d1e5ba34874bb3228160555d8a01dbc37620d4873", + "source_contract_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7", "output_case_count": 5, "failure_disclosure_count": 3, @@ -6370,22 +6394,24 @@ "world_class_source_check_count": 19, "world_class_source_pass_count": 12, "world_class_source_blocked_count": 7, - "beta_test_ready": true, - "beta_test_blocker_count": 0, + "beta_test_ready": false, + "beta_test_blocker_count": 1, "beta_test_deferred_evidence_count": 4, "public_claim_ready": false, - "public_claim_blocker_count": 3, + "public_claim_blocker_count": 4, "working_tree_dirty": true, - "changed_file_count": 3, - "source_tree_dirty": false, - "source_changed_file_count": 0, + "changed_file_count": 84, + "source_tree_dirty": true, + "source_changed_file_count": 2, "generated_tree_dirty": true, - "generated_changed_file_count": 3 + "generated_changed_file_count": 82 }, "beta_test_release": { - "ready": true, + "ready": false, "scope": "beta/public test release without superiority, fully-reviewed, or world-class claims", - "blockers": [], + "blockers": [ + "release lock is not clean or commit is unavailable" + ], "allowed_deferred_evidence": [ { "key": "provider-holdout", @@ -6415,6 +6441,7 @@ "ready": false, "scope": "public benchmark or world-class readiness claim", "blockers": [ + "release lock is not clean or commit is unavailable", "human blind-review adjudication is incomplete", "world-class evidence is not accepted yet (3 open gaps, 4 ledger pending)", "world-class source checks are not all accepted (12/19 pass, 7 blocked)" @@ -6422,12 +6449,12 @@ "policy": "Local reproducibility can pass before public claims; public claims require provider evidence, human adjudication, clean release lock, accepted world-class evidence, and complete source checks." }, "release_lock": { - "ready": true, - "commit": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", + "ready": false, + "commit": "1710a5a3e0177313954758908735c147796e8981", "status_scope": "generation-time status before this report is written", - "source_changed_file_count": 0, - "generated_changed_file_count": 3, - "reason": "only generated evidence artifacts were dirty at generation time" + "source_changed_file_count": 2, + "generated_changed_file_count": 82, + "reason": "source files were dirty at generation time" }, "evidence_bundle": { "algorithm": "sha256(path,label,exists,artifact_sha256)", @@ -6435,7 +6462,7 @@ "existing_count": 25, "missing_count": 0, "missing_paths": [], - "sha256": "e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64cbe8db9ad" + "sha256": "639707e807e5ec8460f6aa6d1e5ba34874bb3228160555d8a01dbc37620d4873" }, "methodology": { "path": "reports/benchmark_methodology.md", @@ -6529,8 +6556,8 @@ "label": "trigger_scorecard", "path": "reports/route_scorecard.json", "exists": true, - "bytes": 16950, - "sha256": "a649547a7d3d5fe02262ca1b188ed4022d8f557770be425f7b7cf762aa76533a" + "bytes": 17042, + "sha256": "53fc22d220dc11453c1e66a17ec2d935e069f9bc936952fe4a53afdb4c71b2cc" }, { "label": "runtime_conformance", @@ -6544,21 +6571,21 @@ "path": "reports/security_trust_report.json", "exists": true, "bytes": 139090, - "sha256": "2ef43f009be68b1dd1a994b841d69b89723cf0b6d7725dc6ad86670d14235135" + "sha256": "ffe2c88bbdbac4907bf8db8f01b9a6b2da78e83acdffb3d975e99424274cd9f7" }, { "label": "python_compatibility", "path": "reports/python_compatibility.json", "exists": true, - "bytes": 30351, - "sha256": "7f3fc8e785bf7007755c0b7f277967fd478ea17a5800ab14b9c23de395576cda" + "bytes": 30497, + "sha256": "4d82942052a2ca87db7acea451096332d4b8bf6420a29fa9f5596e83464d21ea" }, { "label": "registry_audit", "path": "reports/registry_audit.json", "exists": true, - "bytes": 3137, - "sha256": "03cb951dc8ba71843602bffa1148d7a40095a7e109a024193422aaa8a94ba4c7" + "bytes": 3155, + "sha256": "9be6de364d69a2ad40b68b460a35c91476e822eb008cfe4d5e8614c1e989d3d4" }, { "label": "package_verification", @@ -6571,64 +6598,64 @@ "label": "install_simulation", "path": "reports/install_simulation.json", "exists": true, - "bytes": 8793, - "sha256": "6cf79201727352e5f92f4e1fa1697a069ab325d526065f5316ee939e84475ef5" + "bytes": 8947, + "sha256": "65adf51d8c4b1d5ea49edde09004f7e4bfc06127e640baf2de23af695fbe0728" }, { "label": "skill_os2_audit", "path": "reports/skill_os2_audit.json", "exists": true, "bytes": 13824, - "sha256": "1079221b5ac72e19e3e7a0f4970f133d828336f34bac82abf3fb867d18834cfa" + "sha256": "150c0027deaa3c2dd44db5380e9a9b16f123e8a8e3ff6142c711b2b0c432f81c" }, { "label": "world_class_evidence_plan", "path": "reports/world_class_evidence_plan.json", "exists": true, "bytes": 23314, - "sha256": "374ee899a0ba5b84a95337de993439d78e5708be4fdf40dee4292766d0d2bf99" + "sha256": "483ed2b3bd020fa2c5a888300c0d0c1a265eebaf648076581f082f2d527d3669" }, { "label": "world_class_evidence_ledger", "path": "reports/world_class_evidence_ledger.json", "exists": true, "bytes": 26274, - "sha256": "565ae80a82286e912df134c88a269566ecf635f39e55618df4ed4eac69d7c7ea" + "sha256": "b19e7113ec463dbb1cfb87c45d3c51624151aeea1747c8d9c286b6967a385c6a" }, { "label": "world_class_evidence_intake", "path": "reports/world_class_evidence_intake.json", "exists": true, "bytes": 20922, - "sha256": "e36a7e753f4176e7ddd7a74fcc5b064769191baf73b5faa58752f80457ec58ff" + "sha256": "885bff9c1c8546c606f583a982a2283229438dbdcd44f3c7e55aff71f858f8f4" }, { "label": "world_class_evidence_preflight", "path": "reports/world_class_evidence_preflight.json", "exists": true, "bytes": 90852, - "sha256": "9e4f0bc489ca958b5a9e2854f17b847de8ee1f081de40e8c8b86cec138afe633" + "sha256": "a0b00eed5450b33ec29fea6652d41db216931452eeddd0d99f2d90aa20c44d95" }, { "label": "world_class_submission_review", "path": "reports/world_class_submission_review.json", "exists": true, "bytes": 17292, - "sha256": "3c944b4bd27f61483e8e02aa3f3b6c8cdb4aba8dc986241e8cbe60ba64489123" + "sha256": "e8df5c72ade7ad0704b6ee5dd6aedefea98f79a9a370cd4d80655401862e87f4" }, { "label": "world_class_operator_runbook", "path": "reports/world_class_operator_runbook.json", "exists": true, "bytes": 83712, - "sha256": "d8678010fcfd5fb8f28d95925478496749f5498ce13a171c80f79372b874351d" + "sha256": "bb67813244d64f8e5e416620bb9d7856d95d85a1ec7f9581b2a97b9fed47e02e" }, { "label": "world_class_operator_runbook_markdown", "path": "reports/world_class_operator_runbook.md", "exists": true, "bytes": 25504, - "sha256": "fe3a73bf2eb9a018ec03881c95d23d30abcbf8bc36cbb264f68daa08238d8e5f" + "sha256": "e8d4dee7541323f0e6fa6be4903e7c482b1d00ef1db5594f0c44212088b53d85" }, { "label": "world_class_operator_runbook_html", @@ -6641,8 +6668,8 @@ "label": "world_class_claim_guard", "path": "reports/world_class_claim_guard.json", "exists": true, - "bytes": 18759, - "sha256": "556700f05a65bda9b55d829c109d1aec54ae5b4da80ffbefe1196319df86f117" + "bytes": 20102, + "sha256": "5eb808496802e8f972e5aa738d99f3b4a37ff34c8ab495f25ff2db3e7237239f" } ], "missing_artifacts": [], @@ -6783,7 +6810,7 @@ "skill_os2_coverage": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "item_count": 21, @@ -7531,11 +7558,11 @@ "exists": true }, { - "path": "reports/skillops/daily/2026-06-21.json", + "path": "reports/skillops/daily/2026-07-02.json", "exists": true }, { - "path": "reports/skillops/daily/2026-06-21.md", + "path": "reports/skillops/daily/2026-07-02.md", "exists": true } ], @@ -7558,11 +7585,11 @@ "exists": true }, { - "path": "reports/skillops/weekly/2026-W25.json", + "path": "reports/skillops/weekly/2026-W27.json", "exists": true }, { - "path": "reports/skillops/weekly/2026-W25.md", + "path": "reports/skillops/weekly/2026-W27.md", "exists": true } ], @@ -7594,7 +7621,7 @@ "compiled_targets": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-06-13", "skill_dir": ".", "summary": { "target_count": 5, @@ -7631,10 +7658,10 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", @@ -8216,7 +8243,7 @@ "native_surface": "OpenAI-style interface metadata plus neutral Agent Skills source", "activation": { "policy": "Use frontmatter description for catalog routing and targets/openai/agents/openai.yaml for display name, default prompt, and compatibility metadata.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -8474,7 +8501,7 @@ "native_surface": "OpenAI-style interface metadata plus neutral Agent Skills source", "activation": { "policy": "Use frontmatter description for catalog routing and targets/openai/agents/openai.yaml for display name, default prompt, and compatibility metadata.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -8588,10 +8615,10 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", @@ -9173,7 +9200,7 @@ "native_surface": "Claude-compatible neutral source folder with adapter notes", "activation": { "policy": "Use SKILL.md frontmatter description as the primary activation contract and adapter.json for review metadata.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -9431,7 +9458,7 @@ "native_surface": "Claude-compatible neutral source folder with adapter notes", "activation": { "policy": "Use SKILL.md frontmatter description as the primary activation contract and adapter.json for review metadata.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -9545,10 +9572,10 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", @@ -10130,7 +10157,7 @@ "native_surface": "Agent Skills compatible neutral package", "activation": { "policy": "Use SKILL.md name and description; consumers decide automatic or manual activation.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -10381,7 +10408,7 @@ "native_surface": "Agent Skills compatible neutral package", "activation": { "policy": "Use SKILL.md name and description; consumers decide automatic or manual activation.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -10486,10 +10513,10 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", @@ -11071,7 +11098,7 @@ "native_surface": "Agent Skills standard source tree", "activation": { "policy": "Use SKILL.md frontmatter name and description for progressive disclosure.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -11322,7 +11349,7 @@ "native_surface": "Agent Skills standard source tree", "activation": { "policy": "Use SKILL.md frontmatter name and description for progressive disclosure.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -11427,10 +11454,10 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", @@ -12012,7 +12039,7 @@ "native_surface": "VS Code/Copilot Agent Skills project or user scope", "activation": { "policy": "Use folder name plus SKILL.md name/description; keep description under platform limits.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -12267,7 +12294,7 @@ "native_surface": "VS Code/Copilot Agent Skills project or user scope", "activation": { "policy": "Use folder name plus SKILL.md name/description; keep description under platform limits.", - "trigger_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "trigger_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "manual_activation_supported": true, "automatic_activation_note": "Depends on the target client route/catalog implementation." }, @@ -12603,7 +12630,7 @@ "schema_version": "1.0", "ok": true, "skill_dir": ".", - "package_dir": "dist", + "package_dir": "tests/tmp_review_studio/dist", "expected_capabilities": [ "file_write", "network", @@ -12643,7 +12670,7 @@ { "target": "openai", "status": "pass", - "adapter": "dist/targets/openai/adapter.json", + "adapter": "tests/tmp_review_studio/dist/targets/openai/adapter.json", "permission_model": "metadata-only", "native_enforcement": false, "metadata_fallback_explicit": true, @@ -12740,7 +12767,7 @@ { "target": "claude", "status": "pass", - "adapter": "dist/targets/claude/adapter.json", + "adapter": "tests/tmp_review_studio/dist/targets/claude/adapter.json", "permission_model": "neutral-source-plus-adapter", "native_enforcement": false, "metadata_fallback_explicit": true, @@ -12822,7 +12849,7 @@ { "target": "generic", "status": "pass", - "adapter": "dist/targets/generic/adapter.json", + "adapter": "tests/tmp_review_studio/dist/targets/generic/adapter.json", "permission_model": "agent-skills-compatible-metadata", "native_enforcement": false, "metadata_fallback_explicit": true, @@ -12904,7 +12931,7 @@ { "target": "vscode", "status": "pass", - "adapter": "dist/targets/vscode/adapter.json", + "adapter": "tests/tmp_review_studio/dist/targets/vscode/adapter.json", "permission_model": "vscode-workspace-trust-plus-metadata", "native_enforcement": false, "metadata_fallback_explicit": true, @@ -13015,7 +13042,7 @@ "interactive_script_count": 0, "package_hash_scope": "source-contract-without-generated-reports", "package_hash_file_count": 247, - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" }, "failures": [], "warnings": [], @@ -16585,7 +16612,7 @@ "root": ".", "summary": { "target_python": "3.11", - "file_count": 234, + "file_count": 235, "issue_count": 0, "syntax_error_count": 0, "fstring_311_violation_count": 0, @@ -16614,6 +16641,12 @@ "issue_count": 0, "issues": [] }, + { + "path": "reports/blind-human-review-2026-06-29/generate_pack.py", + "ok": true, + "issue_count": 0, + "issues": [] + }, { "path": "scripts/adaptation_patch_safety.py", "ok": true, @@ -18201,11 +18234,11 @@ "stats": { "context_budget_tier": "production", "context_budget_limit": 1000, - "skill_body_tokens": 805, - "other_text_tokens": 1169049, - "estimated_initial_load_tokens": 998, - "estimated_total_text_tokens": 1169854, - "deferred_resource_tokens": 530871, + "skill_body_tokens": 806, + "other_text_tokens": 1200438, + "estimated_initial_load_tokens": 999, + "estimated_total_text_tokens": 1201244, + "deferred_resource_tokens": 531767, "deferred_resource_warn_threshold": 120000, "deferred_resource_dirs": [ { @@ -18220,7 +18253,7 @@ }, { "path": "evals", - "estimated_tokens": 21835, + "estimated_tokens": 22731, "file_count": 27 }, { @@ -18262,10 +18295,10 @@ ], "summary": "Large deferred resources are indexed and backed by evidence." }, - "relevant_file_count": 571, + "relevant_file_count": 590, "unused_resource_dirs": [], "quality_signal_points": 140, - "quality_density": 140.3 + "quality_density": 140.1 } }, "promotion": { @@ -18279,9 +18312,9 @@ { "name": "yao-meta-skill", "winner_label": "Current", - "winner_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "winner_tokens": 53, - "current_tokens": 53, + "winner_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "winner_tokens": 54, + "current_tokens": 54, "winner_holdout_fp": 0, "winner_holdout_fn": 0, "current_holdout_fp": 0, @@ -18312,44 +18345,44 @@ "calibration": { "holdout": { "threshold": 0.33, - "mean_positive_score": 0.908, - "mean_non_trigger_score": 0.003, - "mean_near_neighbor_score": 0.006, - "min_positive_score": 0.767, - "max_non_trigger_score": 0.023, - "score_gap": 0.744, - "threshold_margin": 0.307, - "positive_threshold_buffer": 0.437, - "negative_threshold_buffer": 0.307, + "mean_positive_score": 0.704, + "mean_non_trigger_score": 0.0, + "mean_near_neighbor_score": 0.0, + "min_positive_score": 0.465, + "max_non_trigger_score": 0.0, + "score_gap": 0.465, + "threshold_margin": 0.135, + "positive_threshold_buffer": 0.135, + "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, "risk_band": "healthy" }, "blind_holdout": { "threshold": 0.33, - "mean_positive_score": 0.669, - "mean_non_trigger_score": 0.116, - "mean_near_neighbor_score": 0.152, - "min_positive_score": 0.665, - "max_non_trigger_score": 0.305, - "score_gap": 0.36, - "threshold_margin": 0.025, - "positive_threshold_buffer": 0.335, - "negative_threshold_buffer": 0.025, - "boundary_case_count": 1, - "boundary_case_rate": 0.167, - "risk_band": "tight" + "mean_positive_score": 0.377, + "mean_non_trigger_score": 0.024, + "mean_near_neighbor_score": 0.006, + "min_positive_score": 0.374, + "max_non_trigger_score": 0.084, + "score_gap": 0.29, + "threshold_margin": 0.044, + "positive_threshold_buffer": 0.044, + "negative_threshold_buffer": 0.246, + "boundary_case_count": 2, + "boundary_case_rate": 0.333, + "risk_band": "watch" }, "adversarial_holdout": { "threshold": 0.33, - "mean_positive_score": 1.0, + "mean_positive_score": 0.81, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, - "min_positive_score": 1.0, + "min_positive_score": 0.808, "max_non_trigger_score": 0.0, - "score_gap": 1.0, + "score_gap": 0.808, "threshold_margin": 0.33, - "positive_threshold_buffer": 0.67, + "positive_threshold_buffer": 0.478, "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, @@ -18440,7 +18473,7 @@ "route_metrics": { "precision": 1.0, "recall": 1.0, - "average_margin": 0.701, + "average_margin": 0.644, "misroute_count": 0, "ambiguous_case_count": 0 } @@ -18463,8 +18496,8 @@ "label": "Current", "ranking_state": "selected_by_dev", "promotion_state": "kept_current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "tokens": 53, + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "tokens": 54, "dev_errors": 0, "holdout_errors": 0, "reason_tags": [] @@ -18972,7 +19005,7 @@ "route_metrics": { "precision": 1.0, "recall": 1.0, - "average_margin": 0.586, + "average_margin": 0.709, "misroute_count": 0, "ambiguous_case_count": 0 } @@ -19114,7 +19147,7 @@ { "name": "yao-meta-skill", "path": ".", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "owner": "Yao Team", "version": "1.1.0", "status": "active", @@ -19327,22 +19360,28 @@ "templates/complex_skill.md.j2" ], "token_set": [ - "creation", - "distribution", + "checks", "docs", "evals", "evaluate", - "improvement", - "notes", - "or", + "exclude", + "improve", + "install", + "migration", + "no-skill", "package", - "packaging", "prompts", - "refactor", - "team-ready", - "transcripts", - "workflow", - "workflows" + "release", + "routing", + "scripts", + "sops", + "summary", + "tests", + "translation", + "trigger", + "workflows", + "优化已有", + "评测" ], "atlas_scope": "release", "actionable": true, @@ -20093,7 +20132,7 @@ "adoption_drift": { "ok": true, "schema_version": "2.0", - "generated_at": "2026-06-20T12:35:58Z", + "generated_at": "2026-07-02T03:42:30Z", "skill_dir": ".", "privacy_contract": { "storage": "local-first", @@ -20177,7 +20216,7 @@ "daily_skillops": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02T03:41:56Z", "skill_dir": ".", "decision": "proposal-review", "source_supplied": false, @@ -20190,7 +20229,7 @@ "local_blueprint_ready": true, "public_world_class_ready": false, "world_class_pending_count": 4, - "release_lock_ready": true, + "release_lock_ready": false, "evidence_consistency_ok": true, "writes_source_files": false, "auto_patch_enabled": false, @@ -20207,7 +20246,7 @@ "local_blueprint_ready": true, "public_world_class_ready": false, "world_class_pending_count": 4, - "release_lock_ready": true, + "release_lock_ready": false, "evidence_consistency_ok": true, "writes_source_files": false, "auto_patch_enabled": false, @@ -20656,7 +20695,7 @@ "local_blueprint_ready": true, "public_world_class_ready": false, "world_class_pending_count": 4, - "release_lock_ready": true, + "release_lock_ready": false, "evidence_consistency_ok": true }, "actions": [ @@ -20683,20 +20722,20 @@ "benchmark_reproducibility": "reports/benchmark_reproducibility.json" }, "artifacts": { - "json": "reports/skillops/daily/2026-06-21.json", - "markdown": "reports/skillops/daily/2026-06-21.md" + "json": "reports/skillops/daily/2026-07-02.json", + "markdown": "reports/skillops/daily/2026-07-02.md" }, - "source_path": "reports/skillops/daily/2026-06-21.json" + "source_path": "reports/skillops/daily/2026-07-02.json" }, "weekly_curator": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02T03:41:56Z", "skill_dir": ".", "decision": "curator-review", - "week_id": "2026-W25", - "daily_report_count": 4, - "opportunity_count": 20, + "week_id": "2026-W27", + "daily_report_count": 5, + "opportunity_count": 25, "unique_opportunity_count": 5, "ready_for_approval_review_count": 1, "proposal_review_count": 4, @@ -20705,7 +20744,7 @@ "top_score": 88, "skill_count": 11, "actionable_portfolio_issue_count": 0, - "release_lock_ready": true, + "release_lock_ready": false, "evidence_consistency_ok": true, "public_world_class_ready": false, "world_class_pending_count": 4, @@ -20714,9 +20753,9 @@ "failure_count": 0, "summary": { "decision": "curator-review", - "week_id": "2026-W25", - "daily_report_count": 4, - "opportunity_count": 20, + "week_id": "2026-W27", + "daily_report_count": 5, + "opportunity_count": 25, "unique_opportunity_count": 5, "ready_for_approval_review_count": 1, "proposal_review_count": 4, @@ -20725,7 +20764,7 @@ "top_score": 88, "skill_count": 11, "actionable_portfolio_issue_count": 0, - "release_lock_ready": true, + "release_lock_ready": false, "evidence_consistency_ok": true, "public_world_class_ready": false, "world_class_pending_count": 4, @@ -20782,7 +20821,8 @@ "reports/skillops/daily/2026-06-16.json", "reports/skillops/daily/2026-06-17.json", "reports/skillops/daily/2026-06-20.json", - "reports/skillops/daily/2026-06-21.json" + "reports/skillops/daily/2026-06-21.json", + "reports/skillops/daily/2026-07-02.json" ], "opportunity_summary": { "action_type_counts": { @@ -20980,7 +21020,7 @@ "no_route_opportunity_count": 0 }, "release_state": { - "release_lock_ready": true, + "release_lock_ready": false, "evidence_consistency_ok": true, "public_world_class_ready": false, "world_class_pending_count": 4 @@ -21006,10 +21046,10 @@ "world_class_ledger": "reports/world_class_evidence_ledger.json" }, "artifacts": { - "json": "reports/skillops/weekly/2026-W25.json", - "markdown": "reports/skillops/weekly/2026-W25.md" + "json": "reports/skillops/weekly/2026-W27.json", + "markdown": "reports/skillops/weekly/2026-W27.md" }, - "source_path": "reports/skillops/weekly/2026-W25.json" + "source_path": "reports/skillops/weekly/2026-W27.json" }, "review_waivers": { "schema_version": "1.0", @@ -21158,7 +21198,7 @@ "adaptation_proposals": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02T03:41:56Z", "skill_dir": ".", "source_patterns": "reports/user_patterns.json", "pattern_count": 5, @@ -21556,7 +21596,7 @@ "world_class_evidence_ledger": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-20", + "generated_at": "2026-07-02", "skill_dir": ".", "decision": "evidence-pending", "ready_to_claim_world_class": false, @@ -22158,7 +22198,7 @@ "world_class_evidence_intake": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-20", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "schema_present": true, @@ -22556,7 +22596,7 @@ "world_class_evidence_preflight": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "evidence_item_count": 4, @@ -23357,8 +23397,8 @@ "artifact_prefill_counts_as_evidence": false, "submission_ref_total_count": 2, "submission_ref_ready_count": 2, - "supporting_evidence_total_count": 11, - "supporting_evidence_ready_count": 11, + "supporting_evidence_total_count": 8, + "supporting_evidence_ready_count": 7, "roles": [ { "role": "submission-ref", @@ -24501,8 +24541,8 @@ "artifact_prefill_counts_as_evidence": false, "submission_ref_total_count": 7, "submission_ref_ready_count": 7, - "supporting_evidence_total_count": 31, - "supporting_evidence_ready_count": 31, + "supporting_evidence_total_count": 28, + "supporting_evidence_ready_count": 27, "roles": [ { "role": "submission-ref", @@ -24534,7 +24574,7 @@ "world_class_submission_review": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-20", + "generated_at": "2026-07-02", "skill_dir": ".", "decision": "awaiting-submissions", "ready_to_claim_world_class": false, @@ -24976,7 +25016,7 @@ "world_class_operator_runbook": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-20", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "evidence_item_count": 4, @@ -26667,15 +26707,15 @@ "world_class_claim_guard": { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "ledger_ready_to_claim_world_class": false, "ledger_pending_count": 4, - "claim_surface_count": 184, - "json_claim_surface_count": 91, - "metadata_claim_surface_count": 92, - "package_claim_surface_count": 17, + "claim_surface_count": 194, + "json_claim_surface_count": 92, + "metadata_claim_surface_count": 93, + "package_claim_surface_count": 10, "violation_count": 0, "overclaim_guard_active": true, "decision": "claim-guard-pass-evidence-pending" @@ -26720,34 +26760,6 @@ "path": "agents/interface.yaml", "violation_count": 0 }, - { - "path": "dist/manifest.json", - "violation_count": 0 - }, - { - "path": "dist/targets/claude/README.md", - "violation_count": 0 - }, - { - "path": "dist/targets/claude/adapter.json", - "violation_count": 0 - }, - { - "path": "dist/targets/generic/adapter.json", - "violation_count": 0 - }, - { - "path": "dist/targets/openai/adapter.json", - "violation_count": 0 - }, - { - "path": "dist/targets/vscode/README.md", - "violation_count": 0 - }, - { - "path": "dist/targets/vscode/adapter.json", - "violation_count": 0 - }, { "path": "docs/README.fr-FR.md", "violation_count": 0 @@ -26880,6 +26892,58 @@ "path": "reports/benchmark_reproducibility.md", "violation_count": 0 }, + { + "path": "reports/blind-human-review-2026-06-29/DO_NOT_OPEN_answer_key.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/DO_NOT_OPEN_answer_key.json", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/README.md", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/adjudication.md", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/blind-pack.json", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/case-01-support-triage.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/case-02-monthly-revenue-reconciliation.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/case-03-webinar-repurposing.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/case-04-incident-postmortem.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/case-05-pr-review-followup.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/index.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/review-decisions-template.json", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/review-decisions.recorded.json", + "violation_count": 0 + }, { "path": "reports/candidate_registry.json", "violation_count": 0 @@ -27304,6 +27368,14 @@ "path": "reports/skillops/daily/2026-06-21.md", "violation_count": 0 }, + { + "path": "reports/skillops/daily/2026-07-02.json", + "violation_count": 0 + }, + { + "path": "reports/skillops/daily/2026-07-02.md", + "violation_count": 0 + }, { "path": "reports/skillops/weekly/2026-W25.json", "violation_count": 0 @@ -27312,6 +27384,14 @@ "path": "reports/skillops/weekly/2026-W25.md", "violation_count": 0 }, + { + "path": "reports/skillops/weekly/2026-W27.json", + "violation_count": 0 + }, + { + "path": "reports/skillops/weekly/2026-W27.md", + "violation_count": 0 + }, { "path": "reports/system-model.json", "violation_count": 0 @@ -27464,7 +27544,7 @@ "schema_version": "2.0", "name": "yao-meta-skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "targets": [ "openai", "claude", @@ -27478,7 +27558,7 @@ "trust_level": "local", "license": "MIT", "checksums": { - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7" }, "compatibility": { @@ -27515,11 +27595,11 @@ "install_simulated": true, "install_simulation": "reports/install_simulation.json" }, - "generated_at": "2026-06-21" + "generated_at": "2026-06-13" }, "index": { "schema_version": "2.0", - "generated_at": "2026-06-21", + "generated_at": "2026-06-13", "package_count": 1, "packages": [ { @@ -27535,7 +27615,7 @@ "vscode" ], "package_metadata": "registry/packages/yao-meta-skill.json", - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" } ] }, @@ -28223,14 +28303,14 @@ "install_simulation": { "ok": true, "schema_version": "2.0", - "generated_at": "2026-06-21", + "generated_at": "2026-06-13", "skill_dir": ".", - "package_dir": "dist", - "install_root": "dist/install-simulation/simulate-yao-meta-skill", - "installed_skill_dir": "dist/install-simulation/simulate-yao-meta-skill/yao-meta-skill", + "package_dir": "tests/tmp_review_studio/dist", + "install_root": "tests/tmp_review_studio/install-root/simulate-yao-meta-skill", + "installed_skill_dir": "tests/tmp_review_studio/install-root/simulate-yao-meta-skill/yao-meta-skill", "summary": { "archive_present": true, - "archive_entry_count": 580, + "archive_entry_count": 598, "nested_skill_entry_count": 0, "archive_extracted": true, "entrypoint_loaded": true, @@ -28249,7 +28329,7 @@ { "id": "archive-present", "status": "pass", - "detail": "Package archive exists: dist/yao-meta-skill.zip" + "detail": "Package archive exists: tests/tmp_review_studio/dist/yao-meta-skill.zip" }, { "id": "archive-safe-paths", @@ -28500,16 +28580,16 @@ "failures": [], "warnings": [], "artifacts": { - "archive": "dist/yao-meta-skill.zip", - "package_manifest": "dist/manifest.json", - "json": "reports/install_simulation.json", - "markdown": "reports/install_simulation.md" + "archive": "tests/tmp_review_studio/dist/yao-meta-skill.zip", + "package_manifest": "tests/tmp_review_studio/dist/manifest.json", + "json": "tests/tmp_review_studio/install_simulation.json", + "markdown": "tests/tmp_review_studio/install_simulation.md" } }, "upgrade_check": { "ok": true, "schema_version": "2.0", - "generated_at": "2026-06-21", + "generated_at": "2026-06-13", "previous": { "name": "yao-meta-skill", "version": "1.0.0", @@ -28571,7 +28651,7 @@ { "field": "description", "from": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes.", - "to": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "to": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." } ], "checksum_changes": [ @@ -28583,7 +28663,7 @@ { "field": "package_sha256", "from": "0000000000000000000000000000000000000000000000000000000000000000", - "to": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "to": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" } ] }, @@ -28598,8 +28678,8 @@ "artifacts": { "previous_package": "registry/examples/yao-meta-skill-1.0.0.json", "current_package": "reports/registry_audit.json", - "json": "reports/upgrade_check.json", - "markdown": "reports/upgrade_check.md" + "json": "tests/tmp_review_studio/upgrade_check.json", + "markdown": "tests/tmp_review_studio/upgrade_check.md" } }, "manifest": { @@ -28628,7 +28708,7 @@ }, "frontmatter": { "name": "yao-meta-skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "metadata": { "author": "Yao Team" } diff --git a/reports/review-viewer.html b/reports/review-viewer.html index dc661185..f0c857ba 100644 --- a/reports/review-viewer.html +++ b/reports/review-viewer.html @@ -188,7 +188,7 @@ th {

Review Viewer

Yao Meta Skill

-

Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

+

Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.

maturity: governed archetype: governed @@ -200,7 +200,7 @@ th {

Architecture at a glance

-
Inputs
workflow, prompt, transcript, docs, or notes
Boundary
Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.
Logic
For one-off/no reusable process: `Do not create a skill`; `near-neighbor`; require `repeated use` + `reusable output contract`.; Capture job, output, exclusions, constraints, standards, and the lightest fit.; Scan references in order: external benchmark, user source, local fit; surface only uncertainty or conflict.
Usage
Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.; Use this skill when the request matches: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.
Next
Review the top iteration directions before growing the package.
+
Inputs
workflow, prompt, transcript, docs, or notes
Boundary
Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.
Logic
For one-off/no reusable process: `Do not create a skill`; `near-neighbor`; require `repeated use` + `reusable output contract`.; Capture job, output, exclusions, constraints, standards, and the lightest fit.; Scan references in order: external benchmark, user source, local fit; surface only uncertainty or conflict.
Usage
Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.; Use this skill when the request matches: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.
Next
Review the top iteration directions before growing the package.
@@ -210,7 +210,7 @@ th {

How to use it

-
  • Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.
  • Use this skill when the request matches: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.
+
  • Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.
  • Use this skill when the request matches: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.
@@ -232,13 +232,13 @@ th {

Compare view

-

Winner: Current

{'priority': ['fewest false positives', 'fewest false negatives', 'highest near-neighbor pass rate', 'highest negative pass rate', 'highest precision', 'highest recall', 'shortest description']}

VariantTokensDev ErrorsHoldout ErrorsStrategy
Baseline800existing
Current5300current
Current5300current
+

Winner: Current

{'priority': ['fewest false positives', 'fewest false negatives', 'highest near-neighbor pass rate', 'highest negative pass rate', 'highest precision', 'highest recall', 'shortest description']}

VariantTokensDev ErrorsHoldout ErrorsStrategy
Baseline800existing
Current5400current
Current5400current

Variant diff studio

-

Baseline

existing

Create and improve agent skills.

tokens 8 (0)dev 0 (0)holdout 0 (0)

Adds relative to baseline

  • Create and improve agent skills.

Drops from baseline

No dropped cues.

Current

current

Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

tokens 53 (+45)dev 0 (0)holdout 0 (0)

Adds relative to baseline

  • Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes.
  • Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

Drops from baseline

  • Create and improve agent skills.

Winner — Current

current

Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

tokens 53 (+45)dev 0 (0)holdout 0 (0)

Adds relative to baseline

  • Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes.
  • Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

Drops from baseline

  • Create and improve agent skills.
+

Baseline

existing

Create and improve agent skills.

tokens 8 (0)dev 0 (0)holdout 0 (0)

Adds relative to baseline

  • Create and improve agent skills.

Drops from baseline

No dropped cues.

Current

current

Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.

tokens 54 (+46)dev 0 (0)holdout 0 (0)

Adds relative to baseline

  • Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts.
  • Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测.
  • Exclude no-skill summary/translation/docs.

Drops from baseline

  • Create and improve agent skills.

Winner — Current

current

Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.

tokens 54 (+46)dev 0 (0)holdout 0 (0)

Adds relative to baseline

  • Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts.
  • Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测.
  • Exclude no-skill summary/translation/docs.

Drops from baseline

  • Create and improve agent skills.
@@ -310,11 +310,11 @@ th {

Reference synthesis

-

Official skill anatomy and context discipline

Borrow now

  • Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.

Avoid

  • Do not let packaging or platform concerns swallow the core job boundary.

Human-in-the-loop verification

Borrow now

  • Borrow a review checkpoint wherever trust matters more than raw speed.

Avoid

  • Do not force every skill through heavyweight review when the risk is low.

Boundary-first design

Borrow now

  • Borrow the discipline of defining what the skill should not own before growing the package.

Avoid

  • Do not expand execution assets until route boundaries stay clean.
+

Official skill anatomy and context discipline

Borrow now

  • Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.

Avoid

  • Do not let packaging or platform concerns swallow the core job boundary.

Hypothesis-test-learn loop

Borrow now

  • Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.

Avoid

  • Do not create experimental overhead that exceeds the skill's real risk tier.

Boundary-first design

Borrow now

  • Borrow the discipline of defining what the skill should not own before growing the package.

Avoid

  • Do not expand execution assets until route boundaries stay clean.

Borrow now

-
  • Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.
  • Borrow a review checkpoint wherever trust matters more than raw speed.
  • Borrow the discipline of defining what the skill should not own before growing the package.
  • Borrow the way it turns a messy workflow into a repeatable operating path.
+
  • Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.
  • Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.
  • Borrow the discipline of defining what the skill should not own before growing the package.
  • Borrow the way it turns a messy workflow into a repeatable operating path.

Use the recommendation by default. Only surface the underlying benchmark tradeoffs when intent is uncertain or a real design conflict needs a deliberate call.

diff --git a/reports/review-viewer.json b/reports/review-viewer.json index 63b1b8e5..6cb5d0a7 100644 --- a/reports/review-viewer.json +++ b/reports/review-viewer.json @@ -4,7 +4,7 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "display_name": "Yao Meta Skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "maturity": "governed", "updated_at": "2026-03-31", "core_value": "把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。", @@ -81,7 +81,7 @@ "label": "可维护性", "score": 100, "reasons": [ - "SKILL.md 约 339 个词/字。", + "SKILL.md 约 342 个词/字。", "入口文件保持克制，可维护性较好。", "references/ 已承载扩展指导。", "scripts/ 已承载确定性逻辑。", @@ -102,7 +102,7 @@ "label": "上下文成本", "score": 42, "reasons": [ - "入口约 339 个词/字，references 约 16934 个词/字。", + "入口约 342 个词/字，references 约 16934 个词/字。", "分数越高代表上下文成本越低。", "上下文成本偏高，建议压缩入口或拆分 references。" ] @@ -156,7 +156,7 @@ }, "contract_boundary": { "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "activation": "manual", "execution": "inline", "shell": "bash" @@ -184,7 +184,7 @@ "boundary_cards": [ { "label": "Owned", - "body": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "body": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." }, { "label": "Adjacent", @@ -206,7 +206,7 @@ "已生成 Output Review Adjudication，可记录盲评决策、一致率和待评审项。" ], "gaps": [ - "上下文成本需要补强：入口约 339 个词/字，references 约 16934 个词/字。" + "上下文成本需要补强：入口约 342 个词/字，references 约 16934 个词/字。" ], "recommendations": [ "先改触发边界，再扩展工作流。", @@ -271,7 +271,7 @@ "name": "包体膨胀风险", "impact": 2, "probability": 1, - "signal": "SKILL.md 约 339 个词/字。", + "signal": "SKILL.md 约 342 个词/字。", "response": "先补证据和边界，再增加包体复杂度。" }, { @@ -522,10 +522,10 @@ "path": "reports", "label": "Generated evidence and overview artifacts", "kind": "folder", - "file_count": 152 + "file_count": 170 } ], - "file_count": 375, + "file_count": 393, "folder_count": 4, "distribution": [ { @@ -558,7 +558,7 @@ }, { "label": "reports", - "value": 152 + "value": 170 } ] }, @@ -633,7 +633,7 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "display_name": "Yao Meta Skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "logic_steps": [ "For one-off/no reusable process: `Do not create a skill`; `near-neighbor`; require `repeated use` + `reusable output contract`.", "Capture job, output, exclusions, constraints, standards, and the lightest fit.", @@ -643,7 +643,7 @@ ], "usage_steps": [ "Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.", - "Use this skill when the request matches: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "Use this skill when the request matches: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." ], "package_map": [ { @@ -692,7 +692,7 @@ "path": "reports", "label": "Generated evidence and overview artifacts", "kind": "folder", - "file_count": 152 + "file_count": 170 } ], "strengths": [ @@ -710,7 +710,7 @@ "用户说出类似需求时：Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals." ], "trigger_contract": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "activation": "manual", "execution": "inline", "shell": "bash" @@ -1012,12 +1012,12 @@ "ok": true, "summary": { "reproducibility_ready": true, - "release_lock_ready": true, + "release_lock_ready": false, "methodology_complete": true, "required_artifact_count": 25, "missing_artifact_count": 0, - "evidence_bundle_sha256": "e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64cbe8db9ad", - "source_contract_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "evidence_bundle_sha256": "639707e807e5ec8460f6aa6d1e5ba34874bb3228160555d8a01dbc37620d4873", + "source_contract_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7", "output_case_count": 5, "failure_disclosure_count": 3, @@ -1035,19 +1035,19 @@ "world_class_source_check_count": 19, "world_class_source_pass_count": 12, "world_class_source_blocked_count": 7, - "beta_test_ready": true, - "beta_test_blocker_count": 0, + "beta_test_ready": false, + "beta_test_blocker_count": 1, "beta_test_deferred_evidence_count": 4, "public_claim_ready": false, - "public_claim_blocker_count": 3, + "public_claim_blocker_count": 4, "working_tree_dirty": true, - "changed_file_count": 3, - "source_tree_dirty": false, - "source_changed_file_count": 0, + "changed_file_count": 84, + "source_tree_dirty": true, + "source_changed_file_count": 2, "generated_tree_dirty": true, - "generated_changed_file_count": 3 + "generated_changed_file_count": 82 }, - "commit": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", + "commit": "1710a5a3e0177313954758908735c147796e8981", "missing_artifacts": [], "limitations": [ "The git commit and dirty flags are generation-time context; release lock is blocked by source changes, while generated evidence artifacts are tracked separately.", @@ -1153,7 +1153,7 @@ "interactive_script_count": 0, "package_hash_scope": "source-contract-without-generated-reports", "package_hash_file_count": 247, - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" }, "skill_atlas": { "skill_count": 11, @@ -1177,7 +1177,7 @@ "schema_version": "2.0", "name": "yao-meta-skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "targets": [ "openai", "claude", @@ -1191,7 +1191,7 @@ "trust_level": "local", "license": "MIT", "checksums": { - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7" }, "compatibility": { @@ -1228,7 +1228,7 @@ "install_simulated": true, "install_simulation": "reports/install_simulation.json" }, - "generated_at": "2026-06-21" + "generated_at": "2026-06-13" }, "failures": [], "warnings": [] @@ -1252,7 +1252,7 @@ "ok": true, "summary": { "archive_present": true, - "archive_entry_count": 580, + "archive_entry_count": 598, "nested_skill_entry_count": 0, "archive_extracted": true, "entrypoint_loaded": true, @@ -1313,7 +1313,7 @@ { "field": "description", "from": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes.", - "to": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "to": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." } ], "checksum_changes": [ @@ -1325,7 +1325,7 @@ { "field": "package_sha256", "from": "0000000000000000000000000000000000000000000000000000000000000000", - "to": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "to": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" } ] }, @@ -2219,7 +2219,7 @@ }, "synthesis_highlights": [ "Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.", - "Borrow a review checkpoint wherever trust matters more than raw speed.", + "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", "Borrow the discipline of defining what the skill should not own before growing the package." ], "artifact_design": { @@ -2512,7 +2512,7 @@ "adversarial_errors": 0 }, "current": { - "tokens": 53, + "tokens": 54, "dev_errors": 0, "holdout_errors": 0, "blind_errors": 0, @@ -2520,7 +2520,7 @@ "adversarial_errors": 0 }, "winner": { - "tokens": 53, + "tokens": 54, "dev_errors": 0, "holdout_errors": 0, "blind_errors": 0, @@ -2602,13 +2602,13 @@ ] }, "compare": { - "current_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "current_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "current_candidate": { "id": "current", "label": "Current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "strategy": "current", - "estimated_tokens": 53, + "estimated_tokens": 54, "dev": { "false_positives": 0, "false_negatives": 0, @@ -2624,7 +2624,7 @@ -1.0, -1.0, -1.0, - 53 + 54 ], "holdout": { "false_positives": 0, @@ -2929,7 +2929,8 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ "runbook", @@ -2982,7 +2983,8 @@ ], "transform_workflow": [ "workflow", - "workflow fragments" + "workflow fragments", + "scripts" ] }, "negative": {} @@ -3206,7 +3208,8 @@ "transform_workflow": [ "rough notes", "transcript", - "onboarding flow" + "onboarding flow", + "scripts" ] }, "negative": {} @@ -4089,7 +4092,8 @@ ], "transform_workflow": [ "process", - "internal process note" + "internal process note", + "scripts" ] }, "negative": {} @@ -4156,7 +4160,7 @@ { "prompt": "Turn this recurring onboarding process into a reusable skill library entry with validation notes.", "family": "workflow_to_skill", - "score": 0.546, + "score": 0.841, "predicted_trigger": true, "expected_trigger": true, "passed": true, @@ -4170,6 +4174,7 @@ "iterate_existing_skill" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" @@ -4177,13 +4182,16 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.528, + "semantic_coverage": 0.849, "support_score": 0, "lexical_support": 0.0, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { + "build_skill": [ + "skill library entry" + ], "eval_optimize": [ "validation" ], @@ -4246,10 +4254,12 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ - "quarterly release routine" + "quarterly release routine", + "scripts" ] }, "negative": {} @@ -4290,17 +4300,20 @@ "concept_evidence": { "positive": { "build_skill": [ - "turn this playbook into an agent capability package" + "turn this playbook into an agent capability package", + "agent capability" ], "reuse_package": [ "package", - "team library" + "team library", + "release" ], "team_operationalize": [ "team" ], "transform_workflow": [ "prompt history", + "playbook", "transcript", "release notes" ] @@ -4403,7 +4416,8 @@ "concept_evidence": { "positive": { "reuse_package": [ - "package" + "package", + "release" ], "transform_workflow": [ "checklist" @@ -4438,6 +4452,7 @@ "iterate_existing_skill" ], "matched_desired_concepts": [ + "reuse_package", "transform_workflow" ], "extra_positive_concepts": [], @@ -4448,13 +4463,16 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.189, + "semantic_coverage": 0.377, "support_score": 0, "lexical_support": 0.111, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { + "reuse_package": [ + "release" + ], "transform_workflow": [ "workflow" ] @@ -4751,9 +4769,9 @@ "winner": { "id": "current", "label": "Current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "strategy": "current", - "estimated_tokens": 53, + "estimated_tokens": 54, "dev": { "false_positives": 0, "false_negatives": 0, @@ -4769,7 +4787,7 @@ -1.0, -1.0, -1.0, - 53 + 54 ], "holdout": { "false_positives": 0, @@ -4924,17 +4942,19 @@ { "prompt": "Convert this operations checklist into a reusable skill.", "family": "workflow_to_skill", - "score": 0.673, + "score": 0.379, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -4943,9 +4963,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.25, + "lexical_support": 0.125, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -4960,34 +4980,37 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Add trigger evals to this skill before sharing it with the team.", "family": "iterate_existing_skill", - "score": 0.407, + "score": 0.333, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", - "team_operationalize" + "iterate_existing_skill" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.333, - "support_score": 0.16, + "semantic_coverage": 0.286, + "support_score": 0.04, "lexical_support": 0.25, "negative_penalty": 0, "coverage_boost": 0.04, @@ -5015,31 +5038,32 @@ { "prompt": "We have a messy release runbook, export process, and a prompt history; turn all of that into one reusable skill package with evals and packaging checks.", "family": "complex_multi_asset", - "score": 1.0, + "score": 0.816, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.25, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.208, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -5055,7 +5079,8 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ "runbook", @@ -5070,32 +5095,33 @@ { "prompt": "Turn these workflow fragments, transcripts, and prompts into one production-ready skill with scripts and references.", "family": "complex_multi_asset", - "score": 0.7, + "score": 0.454, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, - "support_score": 0.06, - "lexical_support": 0.4, + "semantic_coverage": 0.411, + "support_score": 0, + "lexical_support": 0.2, "negative_penalty": 0, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "multi_asset": [ @@ -5107,7 +5133,8 @@ ], "transform_workflow": [ "workflow", - "workflow fragments" + "workflow fragments", + "scripts" ] }, "negative": {} @@ -5117,31 +5144,33 @@ { "prompt": "Tighten the trigger boundary on this existing skill and prepare it for team distribution.", "family": "iterate_existing_skill", - "score": 0.742, + "score": 0.514, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", - "reuse_package", - "team_operationalize" + "iterate_existing_skill", + "reuse_package" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, - "support_score": 0.16, - "lexical_support": 0.357, + "semantic_coverage": 0.464, + "support_score": 0.04, + "lexical_support": 0.214, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -5167,31 +5196,34 @@ { "prompt": "Turn the SOP, transcript, and cleanup notes into a maintained skill for the team library.", "family": "paraphrase_trigger", - "score": 0.763, + "score": 0.39, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0, - "lexical_support": 0.357, + "semantic_coverage": 0.357, + "support_score": 0.04, + "lexical_support": 0.143, "negative_penalty": 0, - "coverage_boost": 0.06, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "reuse_package": [ @@ -5208,37 +5240,40 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Codify this workflow into a shared capability package and validate the route boundary before distribution.", "family": "paraphrase_trigger", - "score": 1.0, + "score": 0.815, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.34, - "lexical_support": 0.267, + "semantic_coverage": 0.804, + "support_score": 0.04, + "lexical_support": 0.067, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -5270,33 +5305,35 @@ { "prompt": "After the long handoff notes below, build a skill package for team reuse. The materials include a transcript, rough notes, and an onboarding flow, and the result should include references, scripts, and validation.", "family": "long_context_trigger", - "score": 1.0, + "score": 0.871, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill", - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.4, - "lexical_support": 0.231, + "semantic_coverage": 0.857, + "support_score": 0.04, + "lexical_support": 0.154, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -5324,7 +5361,8 @@ "transform_workflow": [ "rough notes", "transcript", - "onboarding flow" + "onboarding flow", + "scripts" ] }, "negative": {} @@ -5343,10 +5381,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -5358,7 +5398,7 @@ "exclusive_negative_concepts": [ "brainstorm_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0.24, @@ -5387,10 +5427,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -5402,7 +5444,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.083, + "lexical_support": 0.0, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -5426,17 +5468,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [ - "reuse_package" - ], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset", + "reuse_package" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "brainstorm_only", "no_build_directive" @@ -5445,11 +5488,11 @@ "brainstorm_only", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.06, + "semantic_coverage": 0.232, + "support_score": 0, "lexical_support": 0.133, "negative_penalty": 0.66, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "multi_asset": [ @@ -5481,10 +5524,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -5496,7 +5541,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.167, + "lexical_support": 0.0, "negative_penalty": 0.35, "coverage_boost": 0.0, "concept_evidence": { @@ -5520,10 +5565,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -5535,9 +5582,9 @@ "exclusive_negative_concepts": [ "summarize_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.222, + "lexical_support": 0.0, "negative_penalty": 0.3, "coverage_boost": 0.0, "concept_evidence": { @@ -5564,10 +5611,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package" @@ -5581,9 +5630,9 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.087, + "lexical_support": 0.0, "negative_penalty": 0.68, "coverage_boost": 0.0, "concept_evidence": { @@ -5608,36 +5657,37 @@ { "prompt": "I pasted a very long process note. For now I only need a recap and a short summary, not a skill package.", "family": "long_context_summary_only", - "score": 0.306, + "score": 0.236, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "summarize_only" ], "exclusive_negative_concepts": [ "summarize_only" ], - "semantic_coverage": 0.667, - "support_score": 0.34, + "semantic_coverage": 0.661, + "support_score": 0, "lexical_support": 0.222, "negative_penalty": 0.3, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -5657,8 +5707,7 @@ ] } } - }, - "boundary_case": true + } } ], "near_neighbor": [ @@ -5672,10 +5721,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -5690,9 +5741,9 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.1, "negative_penalty": 0.68, "coverage_boost": 0.04, "concept_evidence": { @@ -5725,10 +5776,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -5740,9 +5793,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.143, + "lexical_support": 0.071, "negative_penalty": 0.42, "coverage_boost": 0.0, "concept_evidence": { @@ -5769,17 +5822,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -5787,11 +5841,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, + "semantic_coverage": 0.482, + "support_score": 0, "lexical_support": 0.3, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -5822,17 +5876,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "iterate_existing_skill", "reuse_package" ], - "extra_positive_concepts": [ - "iterate_existing_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline", "no_build_directive" @@ -5841,11 +5896,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.16, - "lexical_support": 0.267, + "semantic_coverage": 0.321, + "support_score": 0, + "lexical_support": 0.2, "negative_penalty": 0.64, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "iterate_existing_skill": [ @@ -5876,17 +5931,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -5894,11 +5950,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.286, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.143, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -5930,17 +5986,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "future_outline", @@ -5950,11 +6007,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.19, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.048, "negative_penalty": 0.82, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -5993,17 +6050,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline", "no_build_directive" @@ -6012,11 +6070,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.263, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.158, "negative_penalty": 0.64, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -6157,30 +6215,32 @@ { "prompt": "Build a reusable skill from this long internal process note, and make sure it has references, scripts, and a trigger description.", "family": "complex_multi_asset", - "score": 0.767, + "score": 0.465, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0.06, + "semantic_coverage": 0.411, + "support_score": 0.04, "lexical_support": 0.211, "negative_penalty": 0, "coverage_boost": 0.06, @@ -6198,7 +6258,8 @@ ], "transform_workflow": [ "process", - "internal process note" + "internal process note", + "scripts" ] }, "negative": {} @@ -6208,32 +6269,34 @@ { "prompt": "Take this existing skill draft, tighten the trigger boundary, add near-neighbor evals, and package it for distribution.", "family": "iterate_existing_skill", - "score": 1.0, + "score": 0.684, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", + "iterate_existing_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.16, - "lexical_support": 0.333, + "semantic_coverage": 0.643, + "support_score": 0.04, + "lexical_support": 0.278, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -6264,19 +6327,22 @@ { "prompt": "Turn this recurring onboarding process into a reusable skill library entry with validation notes.", "family": "workflow_to_skill", - "score": 0.936, + "score": 0.805, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" @@ -6284,13 +6350,16 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, + "semantic_coverage": 0.804, "support_score": 0, - "lexical_support": 0.214, + "lexical_support": 0.071, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { + "build_skill": [ + "skill library entry" + ], "eval_optimize": [ "validation" ], @@ -6309,31 +6378,32 @@ { "prompt": "Codify this quarterly release routine into a reusable skill package with scripts, references, and packaging checks.", "family": "paraphrase_trigger", - "score": 1.0, + "score": 0.874, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill", - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.4, + "semantic_coverage": 0.857, + "support_score": 0, "lexical_support": 0.312, "negative_penalty": 0, "coverage_boost": 0.06, @@ -6352,10 +6422,12 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ - "quarterly release routine" + "quarterly release routine", + "scripts" ] }, "negative": {} @@ -6365,47 +6437,52 @@ { "prompt": "Below is a noisy transcript plus release notes and prompt history. Ignore the chatter and turn this playbook into an agent capability package for the team library.", "family": "long_context_trigger", - "score": 0.839, + "score": 0.691, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0.34, - "lexical_support": 0.24, + "semantic_coverage": 0.661, + "support_score": 0.04, + "lexical_support": 0.16, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ - "turn this playbook into an agent capability package" + "turn this playbook into an agent capability package", + "agent capability" ], "reuse_package": [ "package", - "team library" + "team library", + "release" ], "team_operationalize": [ "team" ], "transform_workflow": [ "prompt history", + "playbook", "transcript", "release notes" ] @@ -6426,10 +6503,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -6444,7 +6523,7 @@ "no_build_directive", "summarize_only" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0.72, @@ -6480,10 +6559,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -6498,15 +6579,16 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.091, + "lexical_support": 0.182, "negative_penalty": 0.68, "coverage_boost": 0.04, "concept_evidence": { "positive": { "reuse_package": [ - "package" + "package", + "release" ], "transform_workflow": [ "checklist" @@ -6534,12 +6616,15 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "reuse_package", "transform_workflow" ], "extra_positive_concepts": [], @@ -6550,13 +6635,16 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.222, + "lexical_support": 0.167, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { + "reuse_package": [ + "release" + ], "transform_workflow": [ "workflow" ] @@ -6584,18 +6672,19 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "no_build_directive", "translate_only" @@ -6604,11 +6693,11 @@ "no_build_directive", "translate_only" ], - "semantic_coverage": 0.667, - "support_score": 0.34, + "semantic_coverage": 0.661, + "support_score": 0, "lexical_support": 0.188, "negative_penalty": 0.77, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -6646,15 +6735,17 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], - "matched_desired_concepts": [], - "extra_positive_concepts": [ + "matched_desired_concepts": [ "build_skill" ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "brainstorm_only", "future_outline" @@ -6663,8 +6754,8 @@ "brainstorm_only", "future_outline" ], - "semantic_coverage": 0.0, - "support_score": 0.34, + "semantic_coverage": 0.304, + "support_score": 0, "lexical_support": 0.077, "negative_penalty": 0.46, "coverage_boost": 0.0, @@ -6695,10 +6786,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -6740,17 +6833,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -6758,11 +6852,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, + "semantic_coverage": 0.482, + "support_score": 0, "lexical_support": 0.308, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -6788,25 +6882,26 @@ { "prompt": "From the long workflow below, create a template only and a checklist for discussion only. No skill package.", "family": "long_context_near_neighbor", - "score": 0.023, + "score": 0.0, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -6814,11 +6909,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.667, - "support_score": 0.34, - "lexical_support": 0.438, + "semantic_coverage": 0.661, + "support_score": 0, + "lexical_support": 0.375, "negative_penalty": 0.6, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -6991,17 +7086,19 @@ { "prompt": "Convert this operations checklist into a reusable skill.", "family": "workflow_to_skill", - "score": 0.673, + "score": 0.379, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -7010,9 +7107,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.25, + "lexical_support": 0.125, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -7027,34 +7124,37 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Add trigger evals to this skill before sharing it with the team.", "family": "iterate_existing_skill", - "score": 0.407, + "score": 0.333, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", - "team_operationalize" + "iterate_existing_skill" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.333, - "support_score": 0.16, + "semantic_coverage": 0.286, + "support_score": 0.04, "lexical_support": 0.25, "negative_penalty": 0, "coverage_boost": 0.04, @@ -7082,31 +7182,32 @@ { "prompt": "We have a messy release runbook, export process, and a prompt history; turn all of that into one reusable skill package with evals and packaging checks.", "family": "complex_multi_asset", - "score": 1.0, + "score": 0.816, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.25, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.208, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -7122,7 +7223,8 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ "runbook", @@ -7137,32 +7239,33 @@ { "prompt": "Turn these workflow fragments, transcripts, and prompts into one production-ready skill with scripts and references.", "family": "complex_multi_asset", - "score": 0.7, + "score": 0.454, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, - "support_score": 0.06, - "lexical_support": 0.4, + "semantic_coverage": 0.411, + "support_score": 0, + "lexical_support": 0.2, "negative_penalty": 0, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "multi_asset": [ @@ -7174,7 +7277,8 @@ ], "transform_workflow": [ "workflow", - "workflow fragments" + "workflow fragments", + "scripts" ] }, "negative": {} @@ -7184,31 +7288,33 @@ { "prompt": "Tighten the trigger boundary on this existing skill and prepare it for team distribution.", "family": "iterate_existing_skill", - "score": 0.742, + "score": 0.514, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", - "reuse_package", - "team_operationalize" + "iterate_existing_skill", + "reuse_package" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, - "support_score": 0.16, - "lexical_support": 0.357, + "semantic_coverage": 0.464, + "support_score": 0.04, + "lexical_support": 0.214, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -7234,31 +7340,34 @@ { "prompt": "Turn the SOP, transcript, and cleanup notes into a maintained skill for the team library.", "family": "paraphrase_trigger", - "score": 0.763, + "score": 0.39, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", - "team_operationalize", "transform_workflow" ], - "extra_positive_concepts": [], + "extra_positive_concepts": [ + "team_operationalize" + ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0, - "lexical_support": 0.357, + "semantic_coverage": 0.357, + "support_score": 0.04, + "lexical_support": 0.143, "negative_penalty": 0, - "coverage_boost": 0.06, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "reuse_package": [ @@ -7275,37 +7384,40 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Codify this workflow into a shared capability package and validate the route boundary before distribution.", "family": "paraphrase_trigger", - "score": 1.0, + "score": 0.815, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.34, - "lexical_support": 0.267, + "semantic_coverage": 0.804, + "support_score": 0.04, + "lexical_support": 0.067, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -7337,33 +7449,35 @@ { "prompt": "After the long handoff notes below, build a skill package for team reuse. The materials include a transcript, rough notes, and an onboarding flow, and the result should include references, scripts, and validation.", "family": "long_context_trigger", - "score": 1.0, + "score": 0.871, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill", - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.4, - "lexical_support": 0.231, + "semantic_coverage": 0.857, + "support_score": 0.04, + "lexical_support": 0.154, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -7391,7 +7505,8 @@ "transform_workflow": [ "rough notes", "transcript", - "onboarding flow" + "onboarding flow", + "scripts" ] }, "negative": {} @@ -7410,10 +7525,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -7425,7 +7542,7 @@ "exclusive_negative_concepts": [ "brainstorm_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0.24, @@ -7454,10 +7571,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -7469,7 +7588,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.083, + "lexical_support": 0.0, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -7493,17 +7612,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [ - "reuse_package" - ], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset", + "reuse_package" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "brainstorm_only", "no_build_directive" @@ -7512,11 +7632,11 @@ "brainstorm_only", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.06, + "semantic_coverage": 0.232, + "support_score": 0, "lexical_support": 0.133, "negative_penalty": 0.66, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "multi_asset": [ @@ -7548,10 +7668,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -7563,7 +7685,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.167, + "lexical_support": 0.0, "negative_penalty": 0.35, "coverage_boost": 0.0, "concept_evidence": { @@ -7587,10 +7709,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -7602,9 +7726,9 @@ "exclusive_negative_concepts": [ "summarize_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.222, + "lexical_support": 0.0, "negative_penalty": 0.3, "coverage_boost": 0.0, "concept_evidence": { @@ -7631,10 +7755,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package" @@ -7648,9 +7774,9 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.087, + "lexical_support": 0.0, "negative_penalty": 0.68, "coverage_boost": 0.0, "concept_evidence": { @@ -7675,36 +7801,37 @@ { "prompt": "I pasted a very long process note. For now I only need a recap and a short summary, not a skill package.", "family": "long_context_summary_only", - "score": 0.306, + "score": 0.236, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "summarize_only" ], "exclusive_negative_concepts": [ "summarize_only" ], - "semantic_coverage": 0.667, - "support_score": 0.34, + "semantic_coverage": 0.661, + "support_score": 0, "lexical_support": 0.222, "negative_penalty": 0.3, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -7724,8 +7851,7 @@ ] } } - }, - "boundary_case": true + } } ], "near_neighbor": [ @@ -7739,10 +7865,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -7757,9 +7885,9 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.1, "negative_penalty": 0.68, "coverage_boost": 0.04, "concept_evidence": { @@ -7792,10 +7920,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -7807,9 +7937,9 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.143, + "lexical_support": 0.071, "negative_penalty": 0.42, "coverage_boost": 0.0, "concept_evidence": { @@ -7836,17 +7966,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -7854,11 +7985,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, + "semantic_coverage": 0.482, + "support_score": 0, "lexical_support": 0.3, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -7889,17 +8020,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "iterate_existing_skill", "reuse_package" ], - "extra_positive_concepts": [ - "iterate_existing_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline", "no_build_directive" @@ -7908,11 +8040,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.16, - "lexical_support": 0.267, + "semantic_coverage": 0.321, + "support_score": 0, + "lexical_support": 0.2, "negative_penalty": 0.64, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "iterate_existing_skill": [ @@ -7943,17 +8075,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -7961,11 +8094,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.286, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.143, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -7997,17 +8130,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "future_outline", @@ -8017,11 +8151,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.19, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.048, "negative_penalty": 0.82, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -8060,17 +8194,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline", "no_build_directive" @@ -8079,11 +8214,11 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, - "lexical_support": 0.263, + "semantic_coverage": 0.482, + "support_score": 0, + "lexical_support": 0.158, "negative_penalty": 0.64, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -8224,30 +8359,32 @@ { "prompt": "Build a reusable skill from this long internal process note, and make sure it has references, scripts, and a trigger description.", "family": "complex_multi_asset", - "score": 0.767, + "score": 0.465, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "multi_asset" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0.06, + "semantic_coverage": 0.411, + "support_score": 0.04, "lexical_support": 0.211, "negative_penalty": 0, "coverage_boost": 0.06, @@ -8265,7 +8402,8 @@ ], "transform_workflow": [ "process", - "internal process note" + "internal process note", + "scripts" ] }, "negative": {} @@ -8275,32 +8413,34 @@ { "prompt": "Take this existing skill draft, tighten the trigger boundary, add near-neighbor evals, and package it for distribution.", "family": "iterate_existing_skill", - "score": 1.0, + "score": 0.684, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "eval_optimize", + "iterate_existing_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "iterate_existing_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 1.0, - "support_score": 0.16, - "lexical_support": 0.333, + "semantic_coverage": 0.643, + "support_score": 0.04, + "lexical_support": 0.278, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -8331,19 +8471,22 @@ { "prompt": "Turn this recurring onboarding process into a reusable skill library entry with validation notes.", "family": "workflow_to_skill", - "score": 0.936, + "score": 0.805, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" @@ -8351,13 +8494,16 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, + "semantic_coverage": 0.804, "support_score": 0, - "lexical_support": 0.214, + "lexical_support": 0.071, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { + "build_skill": [ + "skill library entry" + ], "eval_optimize": [ "validation" ], @@ -8376,31 +8522,32 @@ { "prompt": "Codify this quarterly release routine into a reusable skill package with scripts, references, and packaging checks.", "family": "paraphrase_trigger", - "score": 1.0, + "score": 0.874, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill", - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.4, + "semantic_coverage": 0.857, + "support_score": 0, "lexical_support": 0.312, "negative_penalty": 0, "coverage_boost": 0.06, @@ -8419,10 +8566,12 @@ "reuse_package": [ "reusable", "package", - "packaging" + "packaging", + "release" ], "transform_workflow": [ - "quarterly release routine" + "quarterly release routine", + "scripts" ] }, "negative": {} @@ -8432,47 +8581,52 @@ { "prompt": "Below is a noisy transcript plus release notes and prompt history. Ignore the chatter and turn this playbook into an agent capability package for the team library.", "family": "long_context_trigger", - "score": 0.839, + "score": 0.691, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], "extra_positive_concepts": [ - "build_skill" + "team_operationalize" ], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.733, - "support_score": 0.34, - "lexical_support": 0.24, + "semantic_coverage": 0.661, + "support_score": 0.04, + "lexical_support": 0.16, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ - "turn this playbook into an agent capability package" + "turn this playbook into an agent capability package", + "agent capability" ], "reuse_package": [ "package", - "team library" + "team library", + "release" ], "team_operationalize": [ "team" ], "transform_workflow": [ "prompt history", + "playbook", "transcript", "release notes" ] @@ -8493,10 +8647,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -8511,7 +8667,7 @@ "no_build_directive", "summarize_only" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0.72, @@ -8547,10 +8703,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -8565,15 +8723,16 @@ "explain_only", "no_build_directive" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.091, + "lexical_support": 0.182, "negative_penalty": 0.68, "coverage_boost": 0.04, "concept_evidence": { "positive": { "reuse_package": [ - "package" + "package", + "release" ], "transform_workflow": [ "checklist" @@ -8601,12 +8760,15 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "reuse_package", "transform_workflow" ], "extra_positive_concepts": [], @@ -8617,13 +8779,16 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.222, + "lexical_support": 0.167, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { + "reuse_package": [ + "release" + ], "transform_workflow": [ "workflow" ] @@ -8651,18 +8816,19 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "no_build_directive", "translate_only" @@ -8671,11 +8837,11 @@ "no_build_directive", "translate_only" ], - "semantic_coverage": 0.667, - "support_score": 0.34, + "semantic_coverage": 0.661, + "support_score": 0, "lexical_support": 0.188, "negative_penalty": 0.77, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -8713,15 +8879,17 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], - "matched_desired_concepts": [], - "extra_positive_concepts": [ + "matched_desired_concepts": [ "build_skill" ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "brainstorm_only", "future_outline" @@ -8730,8 +8898,8 @@ "brainstorm_only", "future_outline" ], - "semantic_coverage": 0.0, - "support_score": 0.34, + "semantic_coverage": 0.304, + "support_score": 0, "lexical_support": 0.077, "negative_penalty": 0.46, "coverage_boost": 0.0, @@ -8762,10 +8930,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -8807,17 +8977,18 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -8825,11 +8996,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.333, - "support_score": 0.34, + "semantic_coverage": 0.482, + "support_score": 0, "lexical_support": 0.308, "negative_penalty": 0.6, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "build_skill": [ @@ -8855,25 +9026,26 @@ { "prompt": "From the long workflow below, create a template only and a checklist for discussion only. No skill package.", "family": "long_context_near_neighbor", - "score": 0.023, + "score": 0.0, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -8881,11 +9053,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.667, - "support_score": 0.34, - "lexical_support": 0.438, + "semantic_coverage": 0.661, + "support_score": 0, + "lexical_support": 0.375, "negative_penalty": 0.6, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "build_skill": [ @@ -8988,17 +9160,19 @@ { "prompt": "Operationalize this support escalation routine as a maintained skill and keep trigger eval coverage.", "family": "blind_operationalize", - "score": 0.665, + "score": 0.38, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -9007,7 +9181,7 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0, @@ -9024,22 +9198,25 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Take these SOP scraps, prompt notes, and transcript excerpts and formalize them into a reusable capability package.", "family": "blind_multi_asset", - "score": 0.673, + "score": 0.374, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -9048,9 +9225,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.25, + "lexical_support": 0.062, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -9066,40 +9243,42 @@ }, "negative": {} } - } + }, + "boundary_case": true } ], "should_not_trigger": [ { "prompt": "Turn this workflow note into a wiki reference only, do not create any skill files, and keep it out of agent execution.", "family": "blind_document_only", - "score": 0.16, + "score": 0.084, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [ - "transform_workflow" - ], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset", + "transform_workflow" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only" ], "exclusive_negative_concepts": [], - "semantic_coverage": 0.333, - "support_score": 0.06, - "lexical_support": 0.227, + "semantic_coverage": 0.232, + "support_score": 0, + "lexical_support": 0.136, "negative_penalty": 0.18, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "multi_asset": [ @@ -9127,10 +9306,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -9142,7 +9323,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.3, + "lexical_support": 0.2, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -9167,24 +9348,26 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline" ], "exclusive_negative_concepts": [ "future_outline" ], - "semantic_coverage": 0.0, - "support_score": 0.06, - "lexical_support": 0.231, + "semantic_coverage": 0.054, + "support_score": 0, + "lexical_support": 0.154, "negative_penalty": 0.22, "coverage_boost": 0.0, "concept_evidence": { @@ -9204,17 +9387,19 @@ { "prompt": "Review this process and suggest improvements for later; no packaging yet.", "family": "blind_brainstorm_only", - "score": 0.305, + "score": 0.013, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -9227,9 +9412,9 @@ "exclusive_negative_concepts": [ "future_outline" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.273, + "lexical_support": 0.182, "negative_penalty": 0.22, "coverage_boost": 0.04, "concept_evidence": { @@ -9247,8 +9432,7 @@ ] } } - }, - "boundary_case": true + } } ] } @@ -9327,17 +9511,19 @@ { "prompt": "Operationalize this support escalation routine as a maintained skill and keep trigger eval coverage.", "family": "blind_operationalize", - "score": 0.665, + "score": 0.38, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -9346,7 +9532,7 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, "lexical_support": 0.143, "negative_penalty": 0, @@ -9363,22 +9549,25 @@ }, "negative": {} } - } + }, + "boundary_case": true }, { "prompt": "Take these SOP scraps, prompt notes, and transcript excerpts and formalize them into a reusable capability package.", "family": "blind_multi_asset", - "score": 0.673, + "score": 0.374, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -9387,9 +9576,9 @@ "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.25, + "lexical_support": 0.062, "negative_penalty": 0, "coverage_boost": 0.04, "concept_evidence": { @@ -9405,40 +9594,42 @@ }, "negative": {} } - } + }, + "boundary_case": true } ], "should_not_trigger": [ { "prompt": "Turn this workflow note into a wiki reference only, do not create any skill files, and keep it out of agent execution.", "family": "blind_document_only", - "score": 0.16, + "score": 0.084, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [ - "transform_workflow" - ], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset", + "transform_workflow" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only" ], "exclusive_negative_concepts": [], - "semantic_coverage": 0.333, - "support_score": 0.06, - "lexical_support": 0.227, + "semantic_coverage": 0.232, + "support_score": 0, + "lexical_support": 0.136, "negative_penalty": 0.18, - "coverage_boost": 0.0, + "coverage_boost": 0.04, "concept_evidence": { "positive": { "multi_asset": [ @@ -9466,10 +9657,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -9481,7 +9674,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.3, + "lexical_support": 0.2, "negative_penalty": 0.26, "coverage_boost": 0.0, "concept_evidence": { @@ -9506,24 +9699,26 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" - ], - "matched_desired_concepts": [], - "extra_positive_concepts": [ + "iterate_existing_skill", "multi_asset" ], + "matched_desired_concepts": [ + "multi_asset" + ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "future_outline" ], "exclusive_negative_concepts": [ "future_outline" ], - "semantic_coverage": 0.0, - "support_score": 0.06, - "lexical_support": 0.231, + "semantic_coverage": 0.054, + "support_score": 0, + "lexical_support": 0.154, "negative_penalty": 0.22, "coverage_boost": 0.0, "concept_evidence": { @@ -9543,17 +9738,19 @@ { "prompt": "Review this process and suggest improvements for later; no packaging yet.", "family": "blind_brainstorm_only", - "score": 0.305, + "score": 0.013, "predicted_trigger": false, "expected_trigger": false, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package", @@ -9566,9 +9763,9 @@ "exclusive_negative_concepts": [ "future_outline" ], - "semantic_coverage": 0.667, + "semantic_coverage": 0.357, "support_score": 0, - "lexical_support": 0.273, + "lexical_support": 0.182, "negative_penalty": 0.22, "coverage_boost": 0.04, "concept_evidence": { @@ -9586,8 +9783,7 @@ ] } } - }, - "boundary_case": true + } } ] } @@ -10107,7 +10303,8 @@ "judge_detail": { "mode": "judge-rubric", "focused_positive_concepts": [ - "transform_workflow" + "transform_workflow", + "multi_asset" ], "strong_positive_concepts": [ "transform_workflow" @@ -10196,7 +10393,9 @@ "passed": true, "judge_detail": { "mode": "judge-rubric", - "focused_positive_concepts": [], + "focused_positive_concepts": [ + "multi_asset" + ], "strong_positive_concepts": [], "trigger_action_hits": [], "input_hits": [ @@ -10454,7 +10653,8 @@ "judge_detail": { "mode": "judge-rubric", "focused_positive_concepts": [ - "transform_workflow" + "transform_workflow", + "multi_asset" ], "strong_positive_concepts": [ "transform_workflow" @@ -10543,7 +10743,9 @@ "passed": true, "judge_detail": { "mode": "judge-rubric", - "focused_positive_concepts": [], + "focused_positive_concepts": [ + "multi_asset" + ], "strong_positive_concepts": [], "trigger_action_hits": [], "input_hits": [ @@ -11053,31 +11255,32 @@ { "prompt": "Ignore the noisy meeting chatter and operationalize this support escalation workflow into a maintained skill package with trigger hardening.", "family": "adversarial_noisy_operationalize", - "score": 1.0, + "score": 0.812, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.211, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.158, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -11104,31 +11307,32 @@ { "prompt": "Codify this routine into a reusable capability from docs, transcripts, and prompt history, then stress test the route boundary.", "family": "adversarial_multi_asset_boundary", - "score": 1.0, + "score": 0.808, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.263, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.105, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -11163,18 +11367,19 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -11182,11 +11387,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.667, - "support_score": 0.06, - "lexical_support": 0.357, + "semantic_coverage": 0.411, + "support_score": 0, + "lexical_support": 0.286, "negative_penalty": 0.6, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "multi_asset": [ @@ -11220,10 +11425,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -11240,9 +11447,9 @@ "summarize_only", "translate_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.125, + "lexical_support": 0.062, "negative_penalty": 1.25, "coverage_boost": 0.0, "concept_evidence": { @@ -11281,10 +11488,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -11298,7 +11507,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.1, "negative_penalty": 0.64, "coverage_boost": 0.0, "concept_evidence": { @@ -11327,10 +11536,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package" @@ -11346,9 +11557,9 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.273, + "lexical_support": 0.091, "negative_penalty": 0.88, "coverage_boost": 0.0, "concept_evidence": { @@ -11448,31 +11659,32 @@ { "prompt": "Ignore the noisy meeting chatter and operationalize this support escalation workflow into a maintained skill package with trigger hardening.", "family": "adversarial_noisy_operationalize", - "score": 1.0, + "score": 0.812, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.211, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.158, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -11499,31 +11711,32 @@ { "prompt": "Codify this routine into a reusable capability from docs, transcripts, and prompt history, then stress test the route boundary.", "family": "adversarial_multi_asset_boundary", - "score": 1.0, + "score": 0.808, "predicted_trigger": true, "expected_trigger": true, "passed": true, "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "build_skill", "eval_optimize", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "build_skill" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [], "exclusive_negative_concepts": [], - "semantic_coverage": 0.933, - "support_score": 0.34, - "lexical_support": 0.263, + "semantic_coverage": 0.804, + "support_score": 0, + "lexical_support": 0.105, "negative_penalty": 0, "coverage_boost": 0.06, "concept_evidence": { @@ -11558,18 +11771,19 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ + "multi_asset", "reuse_package", "transform_workflow" ], - "extra_positive_concepts": [ - "multi_asset" - ], + "extra_positive_concepts": [], "matched_negative_concepts": [ "document_only", "no_build_directive" @@ -11577,11 +11791,11 @@ "exclusive_negative_concepts": [ "no_build_directive" ], - "semantic_coverage": 0.667, - "support_score": 0.06, - "lexical_support": 0.357, + "semantic_coverage": 0.411, + "support_score": 0, + "lexical_support": 0.286, "negative_penalty": 0.6, - "coverage_boost": 0.04, + "coverage_boost": 0.06, "concept_evidence": { "positive": { "multi_asset": [ @@ -11615,10 +11829,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "transform_workflow" @@ -11635,9 +11851,9 @@ "summarize_only", "translate_only" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.125, + "lexical_support": 0.062, "negative_penalty": 1.25, "coverage_boost": 0.0, "concept_evidence": { @@ -11676,10 +11892,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [], "extra_positive_concepts": [], @@ -11693,7 +11911,7 @@ ], "semantic_coverage": 0.0, "support_score": 0, - "lexical_support": 0.2, + "lexical_support": 0.1, "negative_penalty": 0.64, "coverage_boost": 0.0, "concept_evidence": { @@ -11722,10 +11940,12 @@ "score_detail": { "mode": "semantic-intent", "desired_positive_concepts": [ + "build_skill", "transform_workflow", "reuse_package", "eval_optimize", - "team_operationalize" + "iterate_existing_skill", + "multi_asset" ], "matched_desired_concepts": [ "reuse_package" @@ -11741,9 +11961,9 @@ "future_outline", "no_build_directive" ], - "semantic_coverage": 0.333, + "semantic_coverage": 0.179, "support_score": 0, - "lexical_support": 0.273, + "lexical_support": 0.091, "negative_penalty": 0.88, "coverage_boost": 0.0, "concept_evidence": { @@ -12172,9 +12392,9 @@ { "id": "current", "label": "Current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "strategy": "current", - "estimated_tokens": 53, + "estimated_tokens": 54, "dev": { "false_positives": 0, "false_negatives": 0, @@ -12190,7 +12410,7 @@ -1.0, -1.0, -1.0, - 53 + 54 ], "holdout": { "false_positives": 0, @@ -12524,37 +12744,37 @@ }, "winner_calibration": { "threshold": 0.33, - "mean_positive_score": 0.908, - "mean_non_trigger_score": 0.003, - "mean_near_neighbor_score": 0.006, - "min_positive_score": 0.767, - "max_non_trigger_score": 0.023, - "score_gap": 0.744, - "threshold_margin": 0.307, - "positive_threshold_buffer": 0.437, - "negative_threshold_buffer": 0.307, + "mean_positive_score": 0.704, + "mean_non_trigger_score": 0.0, + "mean_near_neighbor_score": 0.0, + "min_positive_score": 0.465, + "max_non_trigger_score": 0.0, + "score_gap": 0.465, + "threshold_margin": 0.135, + "positive_threshold_buffer": 0.135, + "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, "risk_band": "healthy" }, "current_calibration": { "threshold": 0.33, - "mean_positive_score": 0.908, - "mean_non_trigger_score": 0.003, - "mean_near_neighbor_score": 0.006, - "min_positive_score": 0.767, - "max_non_trigger_score": 0.023, - "score_gap": 0.744, - "threshold_margin": 0.307, - "positive_threshold_buffer": 0.437, - "negative_threshold_buffer": 0.307, + "mean_positive_score": 0.704, + "mean_non_trigger_score": 0.0, + "mean_near_neighbor_score": 0.0, + "min_positive_score": 0.465, + "max_non_trigger_score": 0.0, + "score_gap": 0.465, + "threshold_margin": 0.135, + "positive_threshold_buffer": 0.135, + "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, "risk_band": "healthy" }, "baseline_calibration": { "threshold": 0.33, - "mean_positive_score": 0.648, + "mean_positive_score": 0.707, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, "min_positive_score": 0.416, @@ -12625,33 +12845,33 @@ }, "winner_calibration": { "threshold": 0.33, - "mean_positive_score": 0.669, - "mean_non_trigger_score": 0.116, - "mean_near_neighbor_score": 0.152, - "min_positive_score": 0.665, - "max_non_trigger_score": 0.305, - "score_gap": 0.36, - "threshold_margin": 0.025, - "positive_threshold_buffer": 0.335, - "negative_threshold_buffer": 0.025, - "boundary_case_count": 1, - "boundary_case_rate": 0.167, - "risk_band": "tight" + "mean_positive_score": 0.377, + "mean_non_trigger_score": 0.024, + "mean_near_neighbor_score": 0.006, + "min_positive_score": 0.374, + "max_non_trigger_score": 0.084, + "score_gap": 0.29, + "threshold_margin": 0.044, + "positive_threshold_buffer": 0.044, + "negative_threshold_buffer": 0.246, + "boundary_case_count": 2, + "boundary_case_rate": 0.333, + "risk_band": "watch" }, "current_calibration": { "threshold": 0.33, - "mean_positive_score": 0.669, - "mean_non_trigger_score": 0.116, - "mean_near_neighbor_score": 0.152, - "min_positive_score": 0.665, - "max_non_trigger_score": 0.305, - "score_gap": 0.36, - "threshold_margin": 0.025, - "positive_threshold_buffer": 0.335, - "negative_threshold_buffer": 0.025, - "boundary_case_count": 1, - "boundary_case_rate": 0.167, - "risk_band": "tight" + "mean_positive_score": 0.377, + "mean_non_trigger_score": 0.024, + "mean_near_neighbor_score": 0.006, + "min_positive_score": 0.374, + "max_non_trigger_score": 0.084, + "score_gap": 0.29, + "threshold_margin": 0.044, + "positive_threshold_buffer": 0.044, + "negative_threshold_buffer": 0.246, + "boundary_case_count": 2, + "boundary_case_rate": 0.333, + "risk_band": "watch" }, "baseline_calibration": { "threshold": 0.33, @@ -12800,14 +13020,14 @@ }, "winner_calibration": { "threshold": 0.33, - "mean_positive_score": 1.0, + "mean_positive_score": 0.81, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, - "min_positive_score": 1.0, + "min_positive_score": 0.808, "max_non_trigger_score": 0.0, - "score_gap": 1.0, + "score_gap": 0.808, "threshold_margin": 0.33, - "positive_threshold_buffer": 0.67, + "positive_threshold_buffer": 0.478, "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, @@ -12815,14 +13035,14 @@ }, "current_calibration": { "threshold": 0.33, - "mean_positive_score": 1.0, + "mean_positive_score": 0.81, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, - "min_positive_score": 1.0, + "min_positive_score": 0.808, "max_non_trigger_score": 0.0, - "score_gap": 1.0, + "score_gap": 0.808, "threshold_margin": 0.33, - "positive_threshold_buffer": 0.67, + "positive_threshold_buffer": 0.478, "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, @@ -12877,8 +13097,8 @@ }, "summary": { "winner_label": "Current", - "winner_tokens": 53, - "current_tokens": 53, + "winner_tokens": 54, + "current_tokens": 54, "winner_dev_total_errors": 0, "current_dev_total_errors": 0, "winner_holdout_total_errors": 0, @@ -12891,7 +13111,7 @@ "current_adversarial_holdout_total_errors": 0, "winner_judge_blind_agreement_rate": 1.0, "winner_adversarial_risk_band": "healthy", - "winner_adversarial_score_gap": 1.0, + "winner_adversarial_score_gap": 0.808, "candidate_count": 6, "baseline_tokens": 8, "baseline_dev_total_errors": 1, @@ -12912,9 +13132,9 @@ { "name": "yao-meta-skill", "winner_label": "Current", - "winner_description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "winner_tokens": 53, - "current_tokens": 53, + "winner_description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "winner_tokens": 54, + "current_tokens": 54, "winner_holdout_fp": 0, "winner_holdout_fn": 0, "current_holdout_fp": 0, @@ -12945,44 +13165,44 @@ "calibration": { "holdout": { "threshold": 0.33, - "mean_positive_score": 0.908, - "mean_non_trigger_score": 0.003, - "mean_near_neighbor_score": 0.006, - "min_positive_score": 0.767, - "max_non_trigger_score": 0.023, - "score_gap": 0.744, - "threshold_margin": 0.307, - "positive_threshold_buffer": 0.437, - "negative_threshold_buffer": 0.307, + "mean_positive_score": 0.704, + "mean_non_trigger_score": 0.0, + "mean_near_neighbor_score": 0.0, + "min_positive_score": 0.465, + "max_non_trigger_score": 0.0, + "score_gap": 0.465, + "threshold_margin": 0.135, + "positive_threshold_buffer": 0.135, + "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, "risk_band": "healthy" }, "blind_holdout": { "threshold": 0.33, - "mean_positive_score": 0.669, - "mean_non_trigger_score": 0.116, - "mean_near_neighbor_score": 0.152, - "min_positive_score": 0.665, - "max_non_trigger_score": 0.305, - "score_gap": 0.36, - "threshold_margin": 0.025, - "positive_threshold_buffer": 0.335, - "negative_threshold_buffer": 0.025, - "boundary_case_count": 1, - "boundary_case_rate": 0.167, - "risk_band": "tight" + "mean_positive_score": 0.377, + "mean_non_trigger_score": 0.024, + "mean_near_neighbor_score": 0.006, + "min_positive_score": 0.374, + "max_non_trigger_score": 0.084, + "score_gap": 0.29, + "threshold_margin": 0.044, + "positive_threshold_buffer": 0.044, + "negative_threshold_buffer": 0.246, + "boundary_case_count": 2, + "boundary_case_rate": 0.333, + "risk_band": "watch" }, "adversarial_holdout": { "threshold": 0.33, - "mean_positive_score": 1.0, + "mean_positive_score": 0.81, "mean_non_trigger_score": 0.0, "mean_near_neighbor_score": 0.0, - "min_positive_score": 1.0, + "min_positive_score": 0.808, "max_non_trigger_score": 0.0, - "score_gap": 1.0, + "score_gap": 0.808, "threshold_margin": 0.33, - "positive_threshold_buffer": 0.67, + "positive_threshold_buffer": 0.478, "negative_threshold_buffer": 0.33, "boundary_case_count": 0, "boundary_case_rate": 0.0, @@ -13073,7 +13293,7 @@ "route_metrics": { "precision": 1.0, "recall": 1.0, - "average_margin": 0.701, + "average_margin": 0.644, "misroute_count": 0, "ambiguous_case_count": 0 } @@ -13096,8 +13316,8 @@ "label": "Current", "ranking_state": "selected_by_dev", "promotion_state": "kept_current", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", - "tokens": 53, + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", + "tokens": 54, "dev_errors": 0, "holdout_errors": 0, "reason_tags": [] @@ -13605,7 +13825,7 @@ "route_metrics": { "precision": 1.0, "recall": 1.0, - "average_margin": 0.586, + "average_margin": 0.709, "misroute_count": 0, "ambiguous_case_count": 0 } @@ -13886,7 +14106,7 @@ }, "reference_synthesis": { "skill_name": "yao-meta-skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "intent_confidence": { "score": 100, "band": "high", @@ -13936,39 +14156,41 @@ }, { "source_type": "research", - "name": "Human-in-the-loop verification", + "name": "Hypothesis-test-learn loop", "evidence_mode": "curated-pattern-track", "matched_keywords": [ - "review", - "govern" + "test", + "iteration" ], - "borrow": "Borrow a review checkpoint wherever trust matters more than raw speed.", - "avoid": "Do not force every skill through heavyweight review when the risk is low.", - "why_relevant": "This track matches: review, govern." + "borrow": "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", + "avoid": "Do not create experimental overhead that exceeds the skill's real risk tier.", + "why_relevant": "This track matches: test, iteration." }, { "source_type": "principles", "name": "Boundary-first design", "evidence_mode": "curated-pattern-track", "matched_keywords": [ - "route" + "route", + "trigger", + "exclude" ], "borrow": "Borrow the discipline of defining what the skill should not own before growing the package.", "avoid": "Do not expand execution assets until route boundaries stay clean.", - "why_relevant": "This track matches: route." + "why_relevant": "This track matches: route, trigger, exclude." } ], "synthesis": { "borrow_now": [ "Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.", - "Borrow a review checkpoint wherever trust matters more than raw speed.", + "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", "Borrow the discipline of defining what the skill should not own before growing the package.", "Borrow the way it turns a messy workflow into a repeatable operating path.", "Borrow the clear execution entrypoints and command structure." ], "avoid_now": [ "Do not let packaging or platform concerns swallow the core job boundary.", - "Do not force every skill through heavyweight review when the risk is low.", + "Do not create experimental overhead that exceeds the skill's real risk tier.", "Do not expand execution assets until route boundaries stay clean.", "Do not import process overhead that only exists for that project's scale.", "Do not copy repo-specific commands or environment assumptions verbatim." @@ -14004,10 +14226,10 @@ "score": 4 }, { - "name": "Human-in-the-loop verification", + "name": "Hypothesis-test-learn loop", "source_type": "research", - "borrow": "Borrow a review checkpoint wherever trust matters more than raw speed.", - "avoid": "Do not force every skill through heavyweight review when the risk is low.", + "borrow": "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", + "avoid": "Do not create experimental overhead that exceeds the skill's real risk tier.", "gates": { "recurrence": true, "generativity": true, @@ -14121,11 +14343,11 @@ "summary": "Start by borrowing this pattern: Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts. Avoid this for the first pass: Do not let packaging or platform concerns swallow the core job boundary.", "borrow_now": [ "Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.", - "Borrow a review checkpoint wherever trust matters more than raw speed." + "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed." ], "avoid_for_now": [ "Do not let packaging or platform concerns swallow the core job boundary.", - "Do not force every skill through heavyweight review when the risk is low." + "Do not create experimental overhead that exceeds the skill's real risk tier." ], "why": "There is a real design conflict to resolve: The stated preference leans lightweight or speed-first, while the benchmark mix leans toward governance, review, or heavier evaluation structure.", "user_decision_required": true @@ -14291,7 +14513,7 @@ "reviewer_note": "Use this report before deepening the package and again before approving example outputs." }, "artifact_design": { - "skill_name": "yao-meta-skill-operator-ux-worktree", + "skill_name": "yao-meta-skill-trigger-coverage-merge", "design_system": "metric editorial", "primary_artifact": { "key": "dashboard", @@ -14411,7 +14633,7 @@ "reviewer_note": "Use this profile to judge whether the generated artifacts feel designed for their job, not merely rendered." }, "prompt_quality": { - "skill_name": "yao-meta-skill-operator-ux-worktree", + "skill_name": "yao-meta-skill-trigger-coverage-merge", "relevance": "prompt-aware", "primary_task_family": { "key": "execution_operation", diff --git a/reports/route_scorecard.json b/reports/route_scorecard.json index 9a15898c..e5bb5ba2 100644 --- a/reports/route_scorecard.json +++ b/reports/route_scorecard.json @@ -15,7 +15,7 @@ "correct_count": 3, "precision": 1.0, "recall": 1.0, - "average_margin": 0.701 + "average_margin": 0.644 }, "team-frontend-review": { "expected_count": 3, @@ -31,7 +31,7 @@ "correct_count": 3, "precision": 1.0, "recall": 1.0, - "average_margin": 0.586 + "average_margin": 0.709 }, "no_route": { "expected_count": 4, @@ -77,18 +77,18 @@ "expected_route": "yao-meta-skill", "predicted_route": "yao-meta-skill", "correct": true, - "margin": 1.0, + "margin": 0.681, "ambiguous": false, "ranked_routes": [ { "name": "yao-meta-skill", - "score": 1.0, + "score": 0.681, "passed_threshold": true, "threshold": 0.33, "matched_desired_concepts": [ "eval_optimize", + "iterate_existing_skill", "reuse_package", - "team_operationalize", "transform_workflow" ], "matched_negative_concepts": [] @@ -117,16 +117,17 @@ "expected_route": "yao-meta-skill", "predicted_route": "yao-meta-skill", "correct": true, - "margin": 0.661, + "margin": 0.51, "ambiguous": false, "ranked_routes": [ { "name": "yao-meta-skill", - "score": 0.661, + "score": 0.51, "passed_threshold": true, "threshold": 0.33, "matched_desired_concepts": [ "eval_optimize", + "iterate_existing_skill", "reuse_package" ], "matched_negative_concepts": [] @@ -155,16 +156,19 @@ "expected_route": "yao-meta-skill", "predicted_route": "yao-meta-skill", "correct": true, - "margin": 0.441, + "margin": 0.74, "ambiguous": false, "ranked_routes": [ { "name": "yao-meta-skill", - "score": 0.441, + "score": 0.74, "passed_threshold": true, "threshold": 0.33, "matched_desired_concepts": [ - "reuse_package" + "build_skill", + "multi_asset", + "reuse_package", + "transform_workflow" ], "matched_negative_concepts": [] }, @@ -211,14 +215,6 @@ ], "matched_negative_concepts": [] }, - { - "name": "yao-meta-skill", - "score": 0.012, - "passed_threshold": false, - "threshold": 0.33, - "matched_desired_concepts": [], - "matched_negative_concepts": [] - }, { "name": "governed-incident-command", "score": 0.012, @@ -226,6 +222,14 @@ "threshold": 0.48, "matched_desired_concepts": [], "matched_negative_concepts": [] + }, + { + "name": "yao-meta-skill", + "score": 0.006, + "passed_threshold": false, + "threshold": 0.33, + "matched_desired_concepts": [], + "matched_negative_concepts": [] } ] }, @@ -251,14 +255,6 @@ ], "matched_negative_concepts": [] }, - { - "name": "yao-meta-skill", - "score": 0.008, - "passed_threshold": false, - "threshold": 0.33, - "matched_desired_concepts": [], - "matched_negative_concepts": [] - }, { "name": "governed-incident-command", "score": 0.008, @@ -266,6 +262,14 @@ "threshold": 0.48, "matched_desired_concepts": [], "matched_negative_concepts": [] + }, + { + "name": "yao-meta-skill", + "score": 0.004, + "passed_threshold": false, + "threshold": 0.33, + "matched_desired_concepts": [], + "matched_negative_concepts": [] } ] }, @@ -293,7 +297,7 @@ }, { "name": "yao-meta-skill", - "score": 0.012, + "score": 0.006, "passed_threshold": false, "threshold": 0.33, "matched_desired_concepts": [], @@ -331,14 +335,6 @@ ], "matched_negative_concepts": [] }, - { - "name": "yao-meta-skill", - "score": 0.016, - "passed_threshold": false, - "threshold": 0.33, - "matched_desired_concepts": [], - "matched_negative_concepts": [] - }, { "name": "team-frontend-review", "score": 0.011, @@ -346,6 +342,14 @@ "threshold": 0.48, "matched_desired_concepts": [], "matched_negative_concepts": [] + }, + { + "name": "yao-meta-skill", + "score": 0.005, + "passed_threshold": false, + "threshold": 0.33, + "matched_desired_concepts": [], + "matched_negative_concepts": [] } ] }, @@ -371,7 +375,7 @@ }, { "name": "yao-meta-skill", - "score": 0.328, + "score": 0.179, "passed_threshold": false, "threshold": 0.33, "matched_desired_concepts": [ @@ -395,7 +399,7 @@ "expected_route": "governed-incident-command", "predicted_route": "governed-incident-command", "correct": true, - "margin": 0.252, + "margin": 0.621, "ambiguous": false, "ranked_routes": [ { @@ -413,12 +417,11 @@ }, { "name": "yao-meta-skill", - "score": 0.748, + "score": 0.379, "passed_threshold": true, "threshold": 0.33, "matched_desired_concepts": [ "reuse_package", - "team_operationalize", "transform_workflow" ], "matched_negative_concepts": [] @@ -481,14 +484,6 @@ "margin": null, "ambiguous": false, "ranked_routes": [ - { - "name": "yao-meta-skill", - "score": 0.008, - "passed_threshold": false, - "threshold": 0.33, - "matched_desired_concepts": [], - "matched_negative_concepts": [] - }, { "name": "governed-incident-command", "score": 0.008, @@ -497,6 +492,14 @@ "matched_desired_concepts": [], "matched_negative_concepts": [] }, + { + "name": "yao-meta-skill", + "score": 0.0, + "passed_threshold": false, + "threshold": 0.33, + "matched_desired_concepts": [], + "matched_negative_concepts": [] + }, { "name": "team-frontend-review", "score": 0.0, @@ -522,7 +525,7 @@ "ranked_routes": [ { "name": "yao-meta-skill", - "score": 0.024, + "score": 0.016, "passed_threshold": false, "threshold": 0.33, "matched_desired_concepts": [], diff --git a/reports/route_scorecard.md b/reports/route_scorecard.md index ba62c9f4..61d1be22 100644 --- a/reports/route_scorecard.md +++ b/reports/route_scorecard.md @@ -9,9 +9,9 @@ | Route | Expected | Predicted | Precision | Recall | Avg Margin | | --- | ---: | ---: | ---: | ---: | ---: | -| `yao-meta-skill` | 3 | 3 | 1.0 | 1.0 | 0.701 | +| `yao-meta-skill` | 3 | 3 | 1.0 | 1.0 | 0.644 | | `team-frontend-review` | 3 | 3 | 1.0 | 1.0 | 0.805 | -| `governed-incident-command` | 3 | 3 | 1.0 | 1.0 | 0.586 | +| `governed-incident-command` | 3 | 3 | 1.0 | 1.0 | 0.709 | | `no_route` | 4 | 4 | 1.0 | 1.0 | - | ## Confusion Matrix diff --git a/reports/runtime_permission_probes.json b/reports/runtime_permission_probes.json index 9c2ffd6e..59c9a1a4 100644 --- a/reports/runtime_permission_probes.json +++ b/reports/runtime_permission_probes.json @@ -2,7 +2,7 @@ "schema_version": "1.0", "ok": true, "skill_dir": ".", - "package_dir": "dist", + "package_dir": "tests/tmp_review_studio/dist", "expected_capabilities": [ "file_write", "network", @@ -42,7 +42,7 @@ { "target": "openai", "status": "pass", - "adapter": "dist/targets/openai/adapter.json", + "adapter": "tests/tmp_review_studio/dist/targets/openai/adapter.json", "permission_model": "metadata-only", "native_enforcement": false, "metadata_fallback_explicit": true, @@ -139,7 +139,7 @@ { "target": "claude", "status": "pass", - "adapter": "dist/targets/claude/adapter.json", + "adapter": "tests/tmp_review_studio/dist/targets/claude/adapter.json", "permission_model": "neutral-source-plus-adapter", "native_enforcement": false, "metadata_fallback_explicit": true, @@ -221,7 +221,7 @@ { "target": "generic", "status": "pass", - "adapter": "dist/targets/generic/adapter.json", + "adapter": "tests/tmp_review_studio/dist/targets/generic/adapter.json", "permission_model": "agent-skills-compatible-metadata", "native_enforcement": false, "metadata_fallback_explicit": true, @@ -303,7 +303,7 @@ { "target": "vscode", "status": "pass", - "adapter": "dist/targets/vscode/adapter.json", + "adapter": "tests/tmp_review_studio/dist/targets/vscode/adapter.json", "permission_model": "vscode-workspace-trust-plus-metadata", "native_enforcement": false, "metadata_fallback_explicit": true, diff --git a/reports/security_trust_report.json b/reports/security_trust_report.json index ac99d3eb..61b7fd84 100644 --- a/reports/security_trust_report.json +++ b/reports/security_trust_report.json @@ -23,7 +23,7 @@ "interactive_script_count": 0, "package_hash_scope": "source-contract-without-generated-reports", "package_hash_file_count": 247, - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" }, "failures": [], "warnings": [], diff --git a/reports/security_trust_report.md b/reports/security_trust_report.md index 07e078c2..54d8aafe 100644 --- a/reports/security_trust_report.md +++ b/reports/security_trust_report.md @@ -16,7 +16,7 @@ - Interactive scripts: `0` - Package hash scope: `source-contract-without-generated-reports` - Package hash files: `247` -- Package SHA256: `eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383` +- Package SHA256: `e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3` ## Failures diff --git a/reports/skill-interpretation.html b/reports/skill-interpretation.html index 83867beb..4cc68623 100644 --- a/reports/skill-interpretation.html +++ b/reports/skill-interpretation.html @@ -761,7 +761,7 @@

YAO Skill 生成审计报告YAO Skill Generation Audit

yao-meta-skill

技能审计报告Yao Meta Skill Audit Report

-

这份报告默认使用中文简体，把新 Skill 的定位、指标、原理、契约、质量、风险、资产和迭代路线整理为一份可审计的 HTML 报告。Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

+

这份报告默认使用中文简体，把新 Skill 的定位、指标、原理、契约、质量、风险、资产和迭代路线整理为一份可审计的 HTML 报告。Skill-specific source text is authored in Chinese; switch to Simplified Chinese for the exact wording.

技能名称：yao-meta-skillSkill name: yao-meta-skill成熟度：治理Maturity: governed格式：Agent SkillsFormat: agent-skills更新时间：2026-03-31Updated: 2026-03-31
openaiopenaiclaudeclaudegenericgenericvscodevscode
@@ -770,7 +770,7 @@
  • 把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。Turn one-off experience into a reusable, evaluable, and portable skill package.
  • Skill 作者、复用团队和后续 reviewer。Skill authors, reuse teams, and later reviewers.
  • 创建完成后建议先打开 reports/skill-interpretation.html，再继续扩展包体。After creation, open reports/skill-interpretation.html before expanding the package further.
-
完整度Completeness100SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
触发清晰Trigger clarity100frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
证据充分Evidence depth100已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
上下文成本Context cost42入口约 339 个词/字，references 约 16934 个词/字。Entrypoint is about 339 words/characters; references are about 16934.
+
完整度Completeness100SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
触发清晰Trigger clarity100frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
证据充分Evidence depth100已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
上下文成本Context cost42入口约 342 个词/字，references 约 16934 个词/字。Entrypoint is about 342 words/characters; references are about 16934.
@@ -782,7 +782,7 @@

作用定位Role

-
  • 原始说明可切换到英文查看；默认中文报告保留结论与结构说明。Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.
  • 把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。Turn one-off experience into a reusable, evaluable, and portable skill package.
  • 交付结果：SKILL.md, agents/interface.yaml, skill-ir/examples/yao-meta-skill.json, reports/compiled_targets.md, reports/output_quality_scorecard.md, reports/output_execution_runs.md, reports/output_blind_review_pack.md, reports/output_review_kit.md, reports/output_review_adjudication.md, reports/benchmark_reproducibility.md, reports/output_blind_answer_key.json, reports/conformance_matrix.md, reports/security_trust_report.md, reports/runtime_permission_probes.md, reports/skill_atlas.html, reports/registry_audit.md, reports/package_verification.md, reports/install_simulation.md, reports/upgrade_check.md, reports/adoption_drift_report.md, reports/review_waivers.md, reports/world_class_evidence_plan.md, reports/world_class_evidence_ledger.md, reports/review_annotations.md, reports/review-studio.html, reports/skill-interpretation.html, reports/skill-overview.html, reports/skill-interpretation.jsonDeliverables: SKILL.md, agents/interface.yaml, skill-ir/examples/yao-meta-skill.json, reports/compiled_targets.md, reports/output_quality_scorecard.md, reports/output_execution_runs.md, reports/output_blind_review_pack.md, reports/output_review_kit.md, reports/output_review_adjudication.md, reports/benchmark_reproducibility.md, reports/output_blind_answer_key.json, reports/conformance_matrix.md, reports/security_trust_report.md, reports/runtime_permission_probes.md, reports/skill_atlas.html, reports/registry_audit.md, reports/package_verification.md, reports/install_simulation.md, reports/upgrade_check.md, reports/adoption_drift_report.md, reports/review_waivers.md, reports/world_class_evidence_plan.md, reports/world_class_evidence_ledger.md, reports/review_annotations.md, reports/review-studio.html, reports/skill-interpretation.html, reports/skill-overview.html, reports/skill-interpretation.json
+
  • Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.Skill-specific source text is authored in Chinese; switch to Simplified Chinese for the exact wording.
  • 把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。Turn one-off experience into a reusable, evaluable, and portable skill package.
  • 交付结果：SKILL.md, agents/interface.yaml, skill-ir/examples/yao-meta-skill.json, reports/compiled_targets.md, reports/output_quality_scorecard.md, reports/output_execution_runs.md, reports/output_blind_review_pack.md, reports/output_review_kit.md, reports/output_review_adjudication.md, reports/benchmark_reproducibility.md, reports/output_blind_answer_key.json, reports/conformance_matrix.md, reports/security_trust_report.md, reports/runtime_permission_probes.md, reports/skill_atlas.html, reports/registry_audit.md, reports/package_verification.md, reports/install_simulation.md, reports/upgrade_check.md, reports/adoption_drift_report.md, reports/review_waivers.md, reports/world_class_evidence_plan.md, reports/world_class_evidence_ledger.md, reports/review_annotations.md, reports/review-studio.html, reports/skill-interpretation.html, reports/skill-overview.html, reports/skill-interpretation.jsonDeliverables: SKILL.md, agents/interface.yaml, skill-ir/examples/yao-meta-skill.json, reports/compiled_targets.md, reports/output_quality_scorecard.md, reports/output_execution_runs.md, reports/output_blind_review_pack.md, reports/output_review_kit.md, reports/output_review_adjudication.md, reports/benchmark_reproducibility.md, reports/output_blind_answer_key.json, reports/conformance_matrix.md, reports/security_trust_report.md, reports/runtime_permission_probes.md, reports/skill_atlas.html, reports/registry_audit.md, reports/package_verification.md, reports/install_simulation.md, reports/upgrade_check.md, reports/adoption_drift_report.md, reports/review_waivers.md, reports/world_class_evidence_plan.md, reports/world_class_evidence_ledger.md, reports/review_annotations.md, reports/review-studio.html, reports/skill-interpretation.html, reports/skill-overview.html, reports/skill-interpretation.json
交付流程Delivery Flow输入材料Input materialSkill 包体Skill package可复用能力Reusable capability
交付流程把用户输入、生成的包体和可复用能力放在一条线上。The delivery flow places user input, generated package, and reusable capability on one path.
@@ -807,12 +807,12 @@

成熟度条Maturity Bar

这里把每个指标压缩为状态、分数和第一条证据，先给出整体判断，再进入下方明细。This compresses each metric into status, score, and the first evidence item before the detailed cards below.

-
  1. 稳定Stable完整度Completeness100SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
  2. 稳定Stable触发清晰Trigger clarity100frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
  3. 稳定Stable证据充分Evidence depth100已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
  4. 稳定Stable可维护性Maintainability100SKILL.md 约 339 个词/字。SKILL.md is about 339 words/characters.
  5. 稳定Stable可迁移性Portability100agents/interface.yaml 已存在。agents/interface.yaml exists.
  6. 关注Watch上下文成本Context cost42入口约 339 个词/字，references 约 16934 个词/字。Entrypoint is about 339 words/characters; references are about 16934.
+
  1. 稳定Stable完整度Completeness100SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
  2. 稳定Stable触发清晰Trigger clarity100frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
  3. 稳定Stable证据充分Evidence depth100已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
  4. 稳定Stable可维护性Maintainability100SKILL.md 约 342 个词/字。SKILL.md is about 342 words/characters.
  5. 稳定Stable可迁移性Portability100agents/interface.yaml 已存在。agents/interface.yaml exists.
  6. 关注Watch上下文成本Context cost42入口约 342 个词/字，references 约 16934 个词/字。Entrypoint is about 342 words/characters; references are about 16934.
指标明细Metric Details
-
完整度Completeness100
  • SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
  • README.md 已存在，便于人工阅读。README.md exists for human-readable usage.
  • agents/interface.yaml 已存在，便于跨平台适配。agents/interface.yaml exists for cross-platform adaptation.
触发清晰Trigger clarity100
  • frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
  • description 有足够长度说明任务边界。The description is long enough to explain the task boundary.
  • description 已包含使用场景或排除边界信号。The description includes usage-scenario or exclusion-boundary signals.
证据充分Evidence depth100
  • 已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
  • skill-ir.json 已存在。skill-ir.json exists.
  • compiled_targets.json 已存在。compiled_targets.json exists.
可维护性Maintainability100
  • SKILL.md 约 339 个词/字。SKILL.md is about 339 words/characters.
  • 入口文件保持克制，可维护性较好。The entrypoint stays restrained, which supports maintainability.
  • references/ 已承载扩展指导。references/ carries extended guidance.
可迁移性Portability100
  • agents/interface.yaml 已存在。agents/interface.yaml exists.
  • manifest.json 已存在。manifest.json exists.
  • 目标平台或 adapter target 已声明。Target platforms or adapter targets are declared.
上下文成本Context cost42
  • 入口约 339 个词/字，references 约 16934 个词/字。Entrypoint is about 339 words/characters; references are about 16934.
  • 分数越高代表上下文成本越低。A higher score means lower context cost.
  • 上下文成本偏高，建议压缩入口或拆分 references。Context cost is high; compress the entrypoint or split references further.
+
完整度Completeness100
  • SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
  • README.md 已存在，便于人工阅读。README.md exists for human-readable usage.
  • agents/interface.yaml 已存在，便于跨平台适配。agents/interface.yaml exists for cross-platform adaptation.
触发清晰Trigger clarity100
  • frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
  • description 有足够长度说明任务边界。The description is long enough to explain the task boundary.
  • description 已包含使用场景或排除边界信号。The description includes usage-scenario or exclusion-boundary signals.
证据充分Evidence depth100
  • 已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
  • skill-ir.json 已存在。skill-ir.json exists.
  • compiled_targets.json 已存在。compiled_targets.json exists.
可维护性Maintainability100
  • SKILL.md 约 342 个词/字。SKILL.md is about 342 words/characters.
  • 入口文件保持克制，可维护性较好。The entrypoint stays restrained, which supports maintainability.
  • references/ 已承载扩展指导。references/ carries extended guidance.
可迁移性Portability100
  • agents/interface.yaml 已存在。agents/interface.yaml exists.
  • manifest.json 已存在。manifest.json exists.
  • 目标平台或 adapter target 已声明。Target platforms or adapter targets are declared.
上下文成本Context cost42
  • 入口约 342 个词/字，references 约 16934 个词/字。Entrypoint is about 342 words/characters; references are about 16934.
  • 分数越高代表上下文成本越低。A higher score means lower context cost.
  • 上下文成本偏高，建议压缩入口或拆分 references。Context cost is high; compress the entrypoint or split references further.
@@ -849,7 +849,7 @@

调用方式How To Use

-
  1. 当你需要把工作流或笔记沉淀成结构精简、触发清晰且带必要 eval 的可复用 Skill 时使用 $yao-meta-skill。Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.
  2. 当用户请求与该 Skill 的触发描述匹配时使用。Use this skill when the request matches the frontmatter description.
+
  1. 当你需要把工作流或笔记沉淀成结构精简、触发清晰且带必要 eval 的可复用 Skill 时使用 $yao-meta-skill。Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.
  2. Use this skill when the request matches: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.Use this skill when the request matches the frontmatter description.
@@ -865,7 +865,7 @@

触发描述Trigger

-

该 Skill 的触发描述来自 SKILL.md frontmatter；默认中文报告先呈现能力边界，原始英文描述可切换到英文查看。Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

+

Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.Skill-specific source text is authored in Chinese; switch to Simplified Chinese for the exact wording.

输入材料Inputs

  • 原始说明可切换到英文查看；默认中文报告保留结论与结构说明。rough workflow notes, SOPs, runbooks, prompts, transcripts, documents, or repeated task descriptions
  • 原始说明可切换到英文查看；默认中文报告保留结论与结构说明。an existing skill directory that needs refactor, evaluation, packaging, or governance hardening
  • 原始说明可切换到英文查看；默认中文报告保留结论与结构说明。target platform requirements such as OpenAI, Claude, generic Agent Skills, or team distribution
  • 原始说明可切换到英文查看；默认中文报告保留结论与结构说明。benchmark references, local constraints, desired maturity tier, and review standards
@@ -888,7 +888,7 @@
- +
类型Type证据Evidence建议Action
强项Strength触发面保持精简，并锚定在 frontmatter description。The trigger surface stays lean and anchored in the frontmatter description.保留并复用Keep
强项Strength已生成 Skill IR，核心语义可先于平台打包被审查和迁移。Skill IR is generated so core semantics can be reviewed and migrated before platform packaging.保留并复用Keep
强项Strength已生成目标编译报告，可审查 IR 到 OpenAI、Claude、generic 等目标契约的映射。Target compilation evidence is generated to review how IR maps to OpenAI, Claude, generic, and other target contracts.保留并复用Keep
缺口Gap上下文成本需要补强：入口约 339 个词/字，references 约 16934 个词/字。Context cost needs improvement: Entrypoint is about 339 words/characters; references are about 16934.纳入下一轮修复Fix next
强项Strength触发面保持精简，并锚定在 frontmatter description。The trigger surface stays lean and anchored in the frontmatter description.保留并复用Keep
强项Strength已生成 Skill IR，核心语义可先于平台打包被审查和迁移。Skill IR is generated so core semantics can be reviewed and migrated before platform packaging.保留并复用Keep
强项Strength已生成目标编译报告，可审查 IR 到 OpenAI、Claude、generic 等目标契约的映射。Target compilation evidence is generated to review how IR maps to OpenAI, Claude, generic, and other target contracts.保留并复用Keep
缺口Gap上下文成本需要补强：入口约 342 个词/字，references 约 16934 个词/字。Context cost needs improvement: Entrypoint is about 342 words/characters; references are about 16934.纳入下一轮修复Fix next
@@ -925,7 +925,7 @@
- +
风险Risk信号Signal应对Response
误触发风险Trigger riskfrontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
输出漂移风险Output drift risk已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
证据不足风险Evidence gap risk已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
包体膨胀风险Package bloat riskSKILL.md 约 339 个词/字。SKILL.md is about 339 words/characters.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
跨平台迁移风险Portability riskagents/interface.yaml 已存在。agents/interface.yaml exists.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
误触发风险Trigger riskfrontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
输出漂移风险Output drift risk已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
证据不足风险Evidence gap risk已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
包体膨胀风险Package bloat riskSKILL.md 约 342 个词/字。SKILL.md is about 342 words/characters.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
跨平台迁移风险Portability riskagents/interface.yaml 已存在。agents/interface.yaml exists.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
@@ -940,7 +940,7 @@

让 reviewer 快速确认关键文件、目录和资产分布。Lets reviewers confirm key files, directories, and asset distribution quickly.

-
资产分布Asset Distribution375项375 itemsSKILL.mdSKILL.mdREADME.mdREADME.mdagents/interface.yamlagents/interface.yamlmanifest.jsonmanifest.jsonreferencesreferencesscriptsscripts
资产分布图展示当前包体的文件和目录重心。The asset distribution chart shows where files and directories are concentrated.
+
资产分布Asset Distribution393项393 itemsSKILL.mdSKILL.mdREADME.mdREADME.mdagents/interface.yamlagents/interface.yamlmanifest.jsonmanifest.jsonreferencesreferencesscriptsscripts
资产分布图展示当前包体的文件和目录重心。The asset distribution chart shows where files and directories are concentrated.
diff --git a/reports/skill-interpretation.json b/reports/skill-interpretation.json index 2206d482..11bfb6c4 100644 --- a/reports/skill-interpretation.json +++ b/reports/skill-interpretation.json @@ -3,7 +3,7 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "display_name": "Yao Meta Skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "maturity": "governed", "updated_at": "2026-03-31", "core_value": "把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。", @@ -81,7 +81,7 @@ "label": "可维护性", "score": 100, "reasons": [ - "SKILL.md 约 339 个词/字。", + "SKILL.md 约 342 个词/字。", "入口文件保持克制，可维护性较好。", "references/ 已承载扩展指导。", "scripts/ 已承载确定性逻辑。", @@ -102,7 +102,7 @@ "label": "上下文成本", "score": 42, "reasons": [ - "入口约 339 个词/字，references 约 16934 个词/字。", + "入口约 342 个词/字，references 约 16934 个词/字。", "分数越高代表上下文成本越低。", "上下文成本偏高，建议压缩入口或拆分 references。" ] @@ -156,7 +156,7 @@ }, "contract_boundary": { "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "activation": "manual", "execution": "inline", "shell": "bash" @@ -184,7 +184,7 @@ "boundary_cards": [ { "label": "Owned", - "body": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "body": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." }, { "label": "Adjacent", @@ -206,7 +206,7 @@ "已生成 Output Review Adjudication，可记录盲评决策、一致率和待评审项。" ], "gaps": [ - "上下文成本需要补强：入口约 339 个词/字，references 约 16934 个词/字。" + "上下文成本需要补强：入口约 342 个词/字，references 约 16934 个词/字。" ], "recommendations": [ "先改触发边界，再扩展工作流。", @@ -271,7 +271,7 @@ "name": "包体膨胀风险", "impact": 2, "probability": 1, - "signal": "SKILL.md 约 339 个词/字。", + "signal": "SKILL.md 约 342 个词/字。", "response": "先补证据和边界，再增加包体复杂度。" }, { @@ -522,10 +522,10 @@ "path": "reports", "label": "Generated evidence and overview artifacts", "kind": "folder", - "file_count": 152 + "file_count": 170 } ], - "file_count": 375, + "file_count": 393, "folder_count": 4, "distribution": [ { @@ -558,7 +558,7 @@ }, { "label": "reports", - "value": 152 + "value": 170 } ] }, @@ -637,7 +637,7 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "display_name": "Yao Meta Skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "logic_steps": [ "For one-off/no reusable process: `Do not create a skill`; `near-neighbor`; require `repeated use` + `reusable output contract`.", "Capture job, output, exclusions, constraints, standards, and the lightest fit.", @@ -647,7 +647,7 @@ ], "usage_steps": [ "Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.", - "Use this skill when the request matches: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "Use this skill when the request matches: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." ], "package_map": [ { @@ -696,7 +696,7 @@ "path": "reports", "label": "Generated evidence and overview artifacts", "kind": "folder", - "file_count": 152 + "file_count": 170 } ], "strengths": [ @@ -714,7 +714,7 @@ "用户说出类似需求时：Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals." ], "trigger_contract": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "activation": "manual", "execution": "inline", "shell": "bash" @@ -1016,12 +1016,12 @@ "ok": true, "summary": { "reproducibility_ready": true, - "release_lock_ready": true, + "release_lock_ready": false, "methodology_complete": true, "required_artifact_count": 25, "missing_artifact_count": 0, - "evidence_bundle_sha256": "e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64cbe8db9ad", - "source_contract_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "evidence_bundle_sha256": "639707e807e5ec8460f6aa6d1e5ba34874bb3228160555d8a01dbc37620d4873", + "source_contract_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7", "output_case_count": 5, "failure_disclosure_count": 3, @@ -1039,19 +1039,19 @@ "world_class_source_check_count": 19, "world_class_source_pass_count": 12, "world_class_source_blocked_count": 7, - "beta_test_ready": true, - "beta_test_blocker_count": 0, + "beta_test_ready": false, + "beta_test_blocker_count": 1, "beta_test_deferred_evidence_count": 4, "public_claim_ready": false, - "public_claim_blocker_count": 3, + "public_claim_blocker_count": 4, "working_tree_dirty": true, - "changed_file_count": 3, - "source_tree_dirty": false, - "source_changed_file_count": 0, + "changed_file_count": 84, + "source_tree_dirty": true, + "source_changed_file_count": 2, "generated_tree_dirty": true, - "generated_changed_file_count": 3 + "generated_changed_file_count": 82 }, - "commit": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", + "commit": "1710a5a3e0177313954758908735c147796e8981", "missing_artifacts": [], "limitations": [ "The git commit and dirty flags are generation-time context; release lock is blocked by source changes, while generated evidence artifacts are tracked separately.", @@ -1157,7 +1157,7 @@ "interactive_script_count": 0, "package_hash_scope": "source-contract-without-generated-reports", "package_hash_file_count": 247, - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" }, "skill_atlas": { "skill_count": 11, @@ -1181,7 +1181,7 @@ "schema_version": "2.0", "name": "yao-meta-skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "targets": [ "openai", "claude", @@ -1195,7 +1195,7 @@ "trust_level": "local", "license": "MIT", "checksums": { - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7" }, "compatibility": { @@ -1232,7 +1232,7 @@ "install_simulated": true, "install_simulation": "reports/install_simulation.json" }, - "generated_at": "2026-06-21" + "generated_at": "2026-06-13" }, "failures": [], "warnings": [] @@ -1256,7 +1256,7 @@ "ok": true, "summary": { "archive_present": true, - "archive_entry_count": 580, + "archive_entry_count": 598, "nested_skill_entry_count": 0, "archive_extracted": true, "entrypoint_loaded": true, @@ -1317,7 +1317,7 @@ { "field": "description", "from": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes.", - "to": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "to": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." } ], "checksum_changes": [ @@ -1329,7 +1329,7 @@ { "field": "package_sha256", "from": "0000000000000000000000000000000000000000000000000000000000000000", - "to": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "to": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" } ] }, @@ -2223,7 +2223,7 @@ }, "synthesis_highlights": [ "Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.", - "Borrow a review checkpoint wherever trust matters more than raw speed.", + "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", "Borrow the discipline of defining what the skill should not own before growing the package." ], "artifact_design": { diff --git a/reports/skill-os-2-review.md b/reports/skill-os-2-review.md index 2ad915dc..3f222929 100644 --- a/reports/skill-os-2-review.md +++ b/reports/skill-os-2-review.md @@ -153,7 +153,7 @@ Next move: add real client or installer permission enforcement integration. | World-Class Evidence Ledger | `4` evidence entries remain pending: `1` human-required and `3` external-required; anti-overclaim guards block planned work, metadata fallbacks, pending review, and local command runners from counting as completion | | Benchmark Reproducibility | local reproducibility ready with `25` required artifacts, `0` missing artifacts, `23` reproduction commands, and `3` disclosed failure cases; human review, native permission enforcement, native telemetry, and provider ledger approval remain explicit limitations | | IR-first Packaging | `openai`, `claude`, `generic`, and `vscode` adapters include compiler contracts, permission contracts, target-native behavior contracts, IR provenance, semantic parity checks, and install-scope notes where applicable | -| Context Budget | initial load `998/1000`; deferred `530167/120000`; resource governance governed | +| Context Budget | initial load `999/1000`; deferred `530167/120000`; resource governance governed | | CI | `make ci-test` target count is `83` after adding the dedicated Operator UX verifier | ## Next Highest-Leverage Moves diff --git a/reports/skill-overview.html b/reports/skill-overview.html index 2e12b329..d124b1c7 100644 --- a/reports/skill-overview.html +++ b/reports/skill-overview.html @@ -761,7 +761,7 @@

YAO Skill 生成审计报告YAO Skill Generation Audit

yao-meta-skill

技能审计报告Yao Meta Skill Audit Report

-

这份报告默认使用中文简体，把新 Skill 的定位、指标、原理、契约、质量、风险、资产和迭代路线整理为一份可审计的 HTML 报告。Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

+

这份报告默认使用中文简体，把新 Skill 的定位、指标、原理、契约、质量、风险、资产和迭代路线整理为一份可审计的 HTML 报告。Skill-specific source text is authored in Chinese; switch to Simplified Chinese for the exact wording.

技能名称：yao-meta-skillSkill name: yao-meta-skill成熟度：治理Maturity: governed格式：Agent SkillsFormat: agent-skills更新时间：2026-03-31Updated: 2026-03-31
openaiopenaiclaudeclaudegenericgenericvscodevscode
@@ -770,7 +770,7 @@
  • 把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。Turn one-off experience into a reusable, evaluable, and portable skill package.
  • Skill 作者、复用团队和后续 reviewer。Skill authors, reuse teams, and later reviewers.
  • 创建完成后建议先打开 reports/skill-overview.html，再继续扩展包体。After creation, open reports/skill-overview.html before expanding the package further.
-
完整度Completeness100SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
触发清晰Trigger clarity100frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
证据充分Evidence depth100已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
上下文成本Context cost42入口约 339 个词/字，references 约 16934 个词/字。Entrypoint is about 339 words/characters; references are about 16934.
+
完整度Completeness100SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
触发清晰Trigger clarity100frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
证据充分Evidence depth100已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
上下文成本Context cost42入口约 342 个词/字，references 约 16934 个词/字。Entrypoint is about 342 words/characters; references are about 16934.
@@ -782,7 +782,7 @@

作用定位Role

-
  • 原始说明可切换到英文查看；默认中文报告保留结论与结构说明。Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.
  • 把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。Turn one-off experience into a reusable, evaluable, and portable skill package.
  • 交付结果：SKILL.md, agents/interface.yaml, skill-ir/examples/yao-meta-skill.json, reports/compiled_targets.md, reports/output_quality_scorecard.md, reports/output_execution_runs.md, reports/output_blind_review_pack.md, reports/output_review_kit.md, reports/output_review_adjudication.md, reports/benchmark_reproducibility.md, reports/output_blind_answer_key.json, reports/conformance_matrix.md, reports/security_trust_report.md, reports/runtime_permission_probes.md, reports/skill_atlas.html, reports/registry_audit.md, reports/package_verification.md, reports/install_simulation.md, reports/upgrade_check.md, reports/adoption_drift_report.md, reports/review_waivers.md, reports/world_class_evidence_plan.md, reports/world_class_evidence_ledger.md, reports/review_annotations.md, reports/review-studio.html, reports/skill-interpretation.html, reports/skill-overview.htmlDeliverables: SKILL.md, agents/interface.yaml, skill-ir/examples/yao-meta-skill.json, reports/compiled_targets.md, reports/output_quality_scorecard.md, reports/output_execution_runs.md, reports/output_blind_review_pack.md, reports/output_review_kit.md, reports/output_review_adjudication.md, reports/benchmark_reproducibility.md, reports/output_blind_answer_key.json, reports/conformance_matrix.md, reports/security_trust_report.md, reports/runtime_permission_probes.md, reports/skill_atlas.html, reports/registry_audit.md, reports/package_verification.md, reports/install_simulation.md, reports/upgrade_check.md, reports/adoption_drift_report.md, reports/review_waivers.md, reports/world_class_evidence_plan.md, reports/world_class_evidence_ledger.md, reports/review_annotations.md, reports/review-studio.html, reports/skill-interpretation.html, reports/skill-overview.html
+
  • Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.Skill-specific source text is authored in Chinese; switch to Simplified Chinese for the exact wording.
  • 把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。Turn one-off experience into a reusable, evaluable, and portable skill package.
  • 交付结果：SKILL.md, agents/interface.yaml, skill-ir/examples/yao-meta-skill.json, reports/compiled_targets.md, reports/output_quality_scorecard.md, reports/output_execution_runs.md, reports/output_blind_review_pack.md, reports/output_review_kit.md, reports/output_review_adjudication.md, reports/benchmark_reproducibility.md, reports/output_blind_answer_key.json, reports/conformance_matrix.md, reports/security_trust_report.md, reports/runtime_permission_probes.md, reports/skill_atlas.html, reports/registry_audit.md, reports/package_verification.md, reports/install_simulation.md, reports/upgrade_check.md, reports/adoption_drift_report.md, reports/review_waivers.md, reports/world_class_evidence_plan.md, reports/world_class_evidence_ledger.md, reports/review_annotations.md, reports/review-studio.html, reports/skill-interpretation.html, reports/skill-overview.htmlDeliverables: SKILL.md, agents/interface.yaml, skill-ir/examples/yao-meta-skill.json, reports/compiled_targets.md, reports/output_quality_scorecard.md, reports/output_execution_runs.md, reports/output_blind_review_pack.md, reports/output_review_kit.md, reports/output_review_adjudication.md, reports/benchmark_reproducibility.md, reports/output_blind_answer_key.json, reports/conformance_matrix.md, reports/security_trust_report.md, reports/runtime_permission_probes.md, reports/skill_atlas.html, reports/registry_audit.md, reports/package_verification.md, reports/install_simulation.md, reports/upgrade_check.md, reports/adoption_drift_report.md, reports/review_waivers.md, reports/world_class_evidence_plan.md, reports/world_class_evidence_ledger.md, reports/review_annotations.md, reports/review-studio.html, reports/skill-interpretation.html, reports/skill-overview.html
交付流程Delivery Flow输入材料Input materialSkill 包体Skill package可复用能力Reusable capability
交付流程把用户输入、生成的包体和可复用能力放在一条线上。The delivery flow places user input, generated package, and reusable capability on one path.
@@ -807,12 +807,12 @@

成熟度条Maturity Bar

这里把每个指标压缩为状态、分数和第一条证据，先给出整体判断，再进入下方明细。This compresses each metric into status, score, and the first evidence item before the detailed cards below.

-
  1. 稳定Stable完整度Completeness100SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
  2. 稳定Stable触发清晰Trigger clarity100frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
  3. 稳定Stable证据充分Evidence depth100已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
  4. 稳定Stable可维护性Maintainability100SKILL.md 约 339 个词/字。SKILL.md is about 339 words/characters.
  5. 稳定Stable可迁移性Portability100agents/interface.yaml 已存在。agents/interface.yaml exists.
  6. 关注Watch上下文成本Context cost42入口约 339 个词/字，references 约 16934 个词/字。Entrypoint is about 339 words/characters; references are about 16934.
+
  1. 稳定Stable完整度Completeness100SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
  2. 稳定Stable触发清晰Trigger clarity100frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
  3. 稳定Stable证据充分Evidence depth100已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
  4. 稳定Stable可维护性Maintainability100SKILL.md 约 342 个词/字。SKILL.md is about 342 words/characters.
  5. 稳定Stable可迁移性Portability100agents/interface.yaml 已存在。agents/interface.yaml exists.
  6. 关注Watch上下文成本Context cost42入口约 342 个词/字，references 约 16934 个词/字。Entrypoint is about 342 words/characters; references are about 16934.
指标明细Metric Details
-
完整度Completeness100
  • SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
  • README.md 已存在，便于人工阅读。README.md exists for human-readable usage.
  • agents/interface.yaml 已存在，便于跨平台适配。agents/interface.yaml exists for cross-platform adaptation.
触发清晰Trigger clarity100
  • frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
  • description 有足够长度说明任务边界。The description is long enough to explain the task boundary.
  • description 已包含使用场景或排除边界信号。The description includes usage-scenario or exclusion-boundary signals.
证据充分Evidence depth100
  • 已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
  • skill-ir.json 已存在。skill-ir.json exists.
  • compiled_targets.json 已存在。compiled_targets.json exists.
可维护性Maintainability100
  • SKILL.md 约 339 个词/字。SKILL.md is about 339 words/characters.
  • 入口文件保持克制，可维护性较好。The entrypoint stays restrained, which supports maintainability.
  • references/ 已承载扩展指导。references/ carries extended guidance.
可迁移性Portability100
  • agents/interface.yaml 已存在。agents/interface.yaml exists.
  • manifest.json 已存在。manifest.json exists.
  • 目标平台或 adapter target 已声明。Target platforms or adapter targets are declared.
上下文成本Context cost42
  • 入口约 339 个词/字，references 约 16934 个词/字。Entrypoint is about 339 words/characters; references are about 16934.
  • 分数越高代表上下文成本越低。A higher score means lower context cost.
  • 上下文成本偏高，建议压缩入口或拆分 references。Context cost is high; compress the entrypoint or split references further.
+
完整度Completeness100
  • SKILL.md 已存在，是 Skill 的入口。SKILL.md exists and acts as the skill entrypoint.
  • README.md 已存在，便于人工阅读。README.md exists for human-readable usage.
  • agents/interface.yaml 已存在，便于跨平台适配。agents/interface.yaml exists for cross-platform adaptation.
触发清晰Trigger clarity100
  • frontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.
  • description 有足够长度说明任务边界。The description is long enough to explain the task boundary.
  • description 已包含使用场景或排除边界信号。The description includes usage-scenario or exclusion-boundary signals.
证据充分Evidence depth100
  • 已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.
  • skill-ir.json 已存在。skill-ir.json exists.
  • compiled_targets.json 已存在。compiled_targets.json exists.
可维护性Maintainability100
  • SKILL.md 约 342 个词/字。SKILL.md is about 342 words/characters.
  • 入口文件保持克制，可维护性较好。The entrypoint stays restrained, which supports maintainability.
  • references/ 已承载扩展指导。references/ carries extended guidance.
可迁移性Portability100
  • agents/interface.yaml 已存在。agents/interface.yaml exists.
  • manifest.json 已存在。manifest.json exists.
  • 目标平台或 adapter target 已声明。Target platforms or adapter targets are declared.
上下文成本Context cost42
  • 入口约 342 个词/字，references 约 16934 个词/字。Entrypoint is about 342 words/characters; references are about 16934.
  • 分数越高代表上下文成本越低。A higher score means lower context cost.
  • 上下文成本偏高，建议压缩入口或拆分 references。Context cost is high; compress the entrypoint or split references further.
@@ -849,7 +849,7 @@

调用方式How To Use

-
  1. 当你需要把工作流或笔记沉淀成结构精简、触发清晰且带必要 eval 的可复用 Skill 时使用 $yao-meta-skill。Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.
  2. 当用户请求与该 Skill 的触发描述匹配时使用。Use this skill when the request matches the frontmatter description.
+
  1. 当你需要把工作流或笔记沉淀成结构精简、触发清晰且带必要 eval 的可复用 Skill 时使用 $yao-meta-skill。Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.
  2. Use this skill when the request matches: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.Use this skill when the request matches the frontmatter description.
@@ -865,7 +865,7 @@

触发描述Trigger

-

该 Skill 的触发描述来自 SKILL.md frontmatter；默认中文报告先呈现能力边界，原始英文描述可切换到英文查看。Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

+

Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.Skill-specific source text is authored in Chinese; switch to Simplified Chinese for the exact wording.

输入材料Inputs

  • 原始说明可切换到英文查看；默认中文报告保留结论与结构说明。rough workflow notes, SOPs, runbooks, prompts, transcripts, documents, or repeated task descriptions
  • 原始说明可切换到英文查看；默认中文报告保留结论与结构说明。an existing skill directory that needs refactor, evaluation, packaging, or governance hardening
  • 原始说明可切换到英文查看；默认中文报告保留结论与结构说明。target platform requirements such as OpenAI, Claude, generic Agent Skills, or team distribution
  • 原始说明可切换到英文查看；默认中文报告保留结论与结构说明。benchmark references, local constraints, desired maturity tier, and review standards
@@ -888,7 +888,7 @@
路径Path作用Role类型Type
SKILL.mdSkill 入口文件Skill entrypoint文件file
README.md人类可读使用说明Human-readable usage guide文件file
agents/interface.yaml跨平台接口元数据Neutral interface metadata文件file
manifest.json生命周期与打包元数据Lifecycle and portability metadata文件file
references扩展指导与复用资料Extended guidance and reusable notes目录folder
scripts确定性脚本或本地工具Deterministic helpers or local tooling目录folder
evals触发与质量检查Trigger and quality checks目录folder
reports生成的证据与总结报告Generated evidence and overview artifacts目录folder
- +
类型Type证据Evidence建议Action
强项Strength触发面保持精简，并锚定在 frontmatter description。The trigger surface stays lean and anchored in the frontmatter description.保留并复用Keep
强项Strength已生成 Skill IR，核心语义可先于平台打包被审查和迁移。Skill IR is generated so core semantics can be reviewed and migrated before platform packaging.保留并复用Keep
强项Strength已生成目标编译报告，可审查 IR 到 OpenAI、Claude、generic 等目标契约的映射。Target compilation evidence is generated to review how IR maps to OpenAI, Claude, generic, and other target contracts.保留并复用Keep
缺口Gap上下文成本需要补强：入口约 339 个词/字，references 约 16934 个词/字。Context cost needs improvement: Entrypoint is about 339 words/characters; references are about 16934.纳入下一轮修复Fix next
强项Strength触发面保持精简，并锚定在 frontmatter description。The trigger surface stays lean and anchored in the frontmatter description.保留并复用Keep
强项Strength已生成 Skill IR，核心语义可先于平台打包被审查和迁移。Skill IR is generated so core semantics can be reviewed and migrated before platform packaging.保留并复用Keep
强项Strength已生成目标编译报告，可审查 IR 到 OpenAI、Claude、generic 等目标契约的映射。Target compilation evidence is generated to review how IR maps to OpenAI, Claude, generic, and other target contracts.保留并复用Keep
缺口Gap上下文成本需要补强：入口约 342 个词/字，references 约 16934 个词/字。Context cost needs improvement: Entrypoint is about 342 words/characters; references are about 16934.纳入下一轮修复Fix next
@@ -925,7 +925,7 @@
- +
风险Risk信号Signal应对Response
误触发风险Trigger riskfrontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
输出漂移风险Output drift risk已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
证据不足风险Evidence gap risk已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
包体膨胀风险Package bloat riskSKILL.md 约 339 个词/字。SKILL.md is about 339 words/characters.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
跨平台迁移风险Portability riskagents/interface.yaml 已存在。agents/interface.yaml exists.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
误触发风险Trigger riskfrontmatter description 已存在，具备基础路由面。The frontmatter description exists, giving the skill a basic routing surface.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
输出漂移风险Output drift risk已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
证据不足风险Evidence gap risk已生成 20 / 20 类报告证据。Generated 20 / 20 evidence report types.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
包体膨胀风险Package bloat riskSKILL.md 约 342 个词/字。SKILL.md is about 342 words/characters.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
跨平台迁移风险Portability riskagents/interface.yaml 已存在。agents/interface.yaml exists.先补证据和边界，再增加包体复杂度。Improve evidence and boundaries before adding package complexity.
@@ -940,7 +940,7 @@

让 reviewer 快速确认关键文件、目录和资产分布。Lets reviewers confirm key files, directories, and asset distribution quickly.

-
资产分布Asset Distribution375项375 itemsSKILL.mdSKILL.mdREADME.mdREADME.mdagents/interface.yamlagents/interface.yamlmanifest.jsonmanifest.jsonreferencesreferencesscriptsscripts
资产分布图展示当前包体的文件和目录重心。The asset distribution chart shows where files and directories are concentrated.
+
资产分布Asset Distribution393项393 itemsSKILL.mdSKILL.mdREADME.mdREADME.mdagents/interface.yamlagents/interface.yamlmanifest.jsonmanifest.jsonreferencesreferencesscriptsscripts
资产分布图展示当前包体的文件和目录重心。The asset distribution chart shows where files and directories are concentrated.
diff --git a/reports/skill-overview.json b/reports/skill-overview.json index 4b37aa1a..92cabd03 100644 --- a/reports/skill-overview.json +++ b/reports/skill-overview.json @@ -3,7 +3,7 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "display_name": "Yao Meta Skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "maturity": "governed", "updated_at": "2026-03-31", "core_value": "把一次性经验沉淀为可复用、可评估、可迁移的 Skill 包体。", @@ -80,7 +80,7 @@ "label": "可维护性", "score": 100, "reasons": [ - "SKILL.md 约 339 个词/字。", + "SKILL.md 约 342 个词/字。", "入口文件保持克制，可维护性较好。", "references/ 已承载扩展指导。", "scripts/ 已承载确定性逻辑。", @@ -101,7 +101,7 @@ "label": "上下文成本", "score": 42, "reasons": [ - "入口约 339 个词/字，references 约 16934 个词/字。", + "入口约 342 个词/字，references 约 16934 个词/字。", "分数越高代表上下文成本越低。", "上下文成本偏高，建议压缩入口或拆分 references。" ] @@ -155,7 +155,7 @@ }, "contract_boundary": { "trigger": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "activation": "manual", "execution": "inline", "shell": "bash" @@ -183,7 +183,7 @@ "boundary_cards": [ { "label": "Owned", - "body": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "body": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." }, { "label": "Adjacent", @@ -205,7 +205,7 @@ "已生成 Output Review Adjudication，可记录盲评决策、一致率和待评审项。" ], "gaps": [ - "上下文成本需要补强：入口约 339 个词/字，references 约 16934 个词/字。" + "上下文成本需要补强：入口约 342 个词/字，references 约 16934 个词/字。" ], "recommendations": [ "先改触发边界，再扩展工作流。", @@ -270,7 +270,7 @@ "name": "包体膨胀风险", "impact": 2, "probability": 1, - "signal": "SKILL.md 约 339 个词/字。", + "signal": "SKILL.md 约 342 个词/字。", "response": "先补证据和边界，再增加包体复杂度。" }, { @@ -521,10 +521,10 @@ "path": "reports", "label": "Generated evidence and overview artifacts", "kind": "folder", - "file_count": 152 + "file_count": 170 } ], - "file_count": 375, + "file_count": 393, "folder_count": 4, "distribution": [ { @@ -557,7 +557,7 @@ }, { "label": "reports", - "value": 152 + "value": 170 } ] }, @@ -632,7 +632,7 @@ "name": "yao-meta-skill", "title": "Yao Meta Skill", "display_name": "Yao Meta Skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "logic_steps": [ "For one-off/no reusable process: `Do not create a skill`; `near-neighbor`; require `repeated use` + `reusable output contract`.", "Capture job, output, exclusions, constraints, standards, and the lightest fit.", @@ -642,7 +642,7 @@ ], "usage_steps": [ "Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals.", - "Use this skill when the request matches: Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "Use this skill when the request matches: Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." ], "package_map": [ { @@ -691,7 +691,7 @@ "path": "reports", "label": "Generated evidence and overview artifacts", "kind": "folder", - "file_count": 152 + "file_count": 170 } ], "strengths": [ @@ -709,7 +709,7 @@ "用户说出类似需求时：Use $yao-meta-skill to turn my workflow or notes into a reusable skill with lean structure, clear triggering, and the right evals." ], "trigger_contract": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "activation": "manual", "execution": "inline", "shell": "bash" @@ -1011,12 +1011,12 @@ "ok": true, "summary": { "reproducibility_ready": true, - "release_lock_ready": true, + "release_lock_ready": false, "methodology_complete": true, "required_artifact_count": 25, "missing_artifact_count": 0, - "evidence_bundle_sha256": "e819a616c7276f6433bbcf38f30696cc0f19b6c035adf5c8cf09e64cbe8db9ad", - "source_contract_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "evidence_bundle_sha256": "639707e807e5ec8460f6aa6d1e5ba34874bb3228160555d8a01dbc37620d4873", + "source_contract_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7", "output_case_count": 5, "failure_disclosure_count": 3, @@ -1034,19 +1034,19 @@ "world_class_source_check_count": 19, "world_class_source_pass_count": 12, "world_class_source_blocked_count": 7, - "beta_test_ready": true, - "beta_test_blocker_count": 0, + "beta_test_ready": false, + "beta_test_blocker_count": 1, "beta_test_deferred_evidence_count": 4, "public_claim_ready": false, - "public_claim_blocker_count": 3, + "public_claim_blocker_count": 4, "working_tree_dirty": true, - "changed_file_count": 3, - "source_tree_dirty": false, - "source_changed_file_count": 0, + "changed_file_count": 84, + "source_tree_dirty": true, + "source_changed_file_count": 2, "generated_tree_dirty": true, - "generated_changed_file_count": 3 + "generated_changed_file_count": 82 }, - "commit": "139d0fd15ce995757d6d75fb45fb417d0b1b4e7d", + "commit": "1710a5a3e0177313954758908735c147796e8981", "missing_artifacts": [], "limitations": [ "The git commit and dirty flags are generation-time context; release lock is blocked by source changes, while generated evidence artifacts are tracked separately.", @@ -1152,7 +1152,7 @@ "interactive_script_count": 0, "package_hash_scope": "source-contract-without-generated-reports", "package_hash_file_count": 247, - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" }, "skill_atlas": { "skill_count": 11, @@ -1176,7 +1176,7 @@ "schema_version": "2.0", "name": "yao-meta-skill", "version": "1.1.0", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "targets": [ "openai", "claude", @@ -1190,7 +1190,7 @@ "trust_level": "local", "license": "MIT", "checksums": { - "package_sha256": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383", + "package_sha256": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3", "archive_sha256": "c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7" }, "compatibility": { @@ -1227,7 +1227,7 @@ "install_simulated": true, "install_simulation": "reports/install_simulation.json" }, - "generated_at": "2026-06-21" + "generated_at": "2026-06-13" }, "failures": [], "warnings": [] @@ -1251,7 +1251,7 @@ "ok": true, "summary": { "archive_present": true, - "archive_entry_count": 580, + "archive_entry_count": 598, "nested_skill_entry_count": 0, "archive_extracted": true, "entrypoint_loaded": true, @@ -1312,7 +1312,7 @@ { "field": "description", "from": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes.", - "to": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "to": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." } ], "checksum_changes": [ @@ -1324,7 +1324,7 @@ { "field": "package_sha256", "from": "0000000000000000000000000000000000000000000000000000000000000000", - "to": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "to": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" } ] }, @@ -2218,7 +2218,7 @@ }, "synthesis_highlights": [ "Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.", - "Borrow a review checkpoint wherever trust matters more than raw speed.", + "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", "Borrow the discipline of defining what the skill should not own before growing the package." ], "artifact_design": { diff --git a/reports/skill_atlas.json b/reports/skill_atlas.json index 05ac9ca8..86449371 100644 --- a/reports/skill_atlas.json +++ b/reports/skill_atlas.json @@ -49,7 +49,7 @@ { "name": "yao-meta-skill", "path": ".", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "owner": "Yao Team", "version": "1.1.0", "status": "active", @@ -262,22 +262,28 @@ "templates/complex_skill.md.j2" ], "token_set": [ - "creation", - "distribution", + "checks", "docs", "evals", "evaluate", - "improvement", - "notes", - "or", + "exclude", + "improve", + "install", + "migration", + "no-skill", "package", - "packaging", "prompts", - "refactor", - "team-ready", - "transcripts", - "workflow", - "workflows" + "release", + "routing", + "scripts", + "sops", + "summary", + "tests", + "translation", + "trigger", + "workflows", + "优化已有", + "评测" ], "atlas_scope": "release", "actionable": true, diff --git a/reports/skill_os2_audit.json b/reports/skill_os2_audit.json index 9210550f..f320b193 100644 --- a/reports/skill_os2_audit.json +++ b/reports/skill_os2_audit.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-06-13", "skill_dir": ".", "summary": { "item_count": 15, diff --git a/reports/skill_os2_audit.md b/reports/skill_os2_audit.md index 72d20b74..ca3c1a30 100644 --- a/reports/skill_os2_audit.md +++ b/reports/skill_os2_audit.md @@ -1,6 +1,6 @@ # Skill OS 2.0 Audit -Generated at: `2026-06-21` +Generated at: `2026-06-13` ## Summary diff --git a/reports/skill_os2_coverage.json b/reports/skill_os2_coverage.json index 4541299a..3c8c43e0 100644 --- a/reports/skill_os2_coverage.json +++ b/reports/skill_os2_coverage.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "item_count": 21, @@ -749,11 +749,11 @@ "exists": true }, { - "path": "reports/skillops/daily/2026-06-21.json", + "path": "reports/skillops/daily/2026-07-02.json", "exists": true }, { - "path": "reports/skillops/daily/2026-06-21.md", + "path": "reports/skillops/daily/2026-07-02.md", "exists": true } ], @@ -776,11 +776,11 @@ "exists": true }, { - "path": "reports/skillops/weekly/2026-W25.json", + "path": "reports/skillops/weekly/2026-W27.json", "exists": true }, { - "path": "reports/skillops/weekly/2026-W25.md", + "path": "reports/skillops/weekly/2026-W27.md", "exists": true } ], diff --git a/reports/skill_os2_coverage.md b/reports/skill_os2_coverage.md index 0369dfb6..fd2a5132 100644 --- a/reports/skill_os2_coverage.md +++ b/reports/skill_os2_coverage.md @@ -1,6 +1,6 @@ # Skill OS 2.0 Blueprint Coverage -Generated at: `2026-06-21` +Generated at: `2026-07-02` ## Summary @@ -235,12 +235,12 @@ These extension tracks come from the user-supplied 2.0 reference plan. They are - objective: Daily operations layer summarizes explicit-source conversation patterns, proposal-only adaptation work, approval state, release locks, and world-class evidence gaps. - status: `covered` -- existing evidence: `scripts/render_daily_skillops_report.py`, `tests/verify_daily_skillops.py`, `reports/skillops/daily/2026-06-21.json`, `reports/skillops/daily/2026-06-21.md` +- existing evidence: `scripts/render_daily_skillops_report.py`, `tests/verify_daily_skillops.py`, `reports/skillops/daily/2026-07-02.json`, `reports/skillops/daily/2026-07-02.md` - next action: Keep Daily SkillOps report aligned with proposal, approval, coverage, and world-class ledger contracts as the operations layer evolves. ### Weekly Curator Report - objective: Weekly curator layer aggregates Daily SkillOps opportunities, Skill Atlas portfolio signals, release locks, and world-class evidence gaps into a maintenance queue. - status: `covered` -- existing evidence: `scripts/render_weekly_curator_report.py`, `tests/verify_weekly_curator.py`, `reports/skillops/weekly/2026-W25.json`, `reports/skillops/weekly/2026-W25.md` +- existing evidence: `scripts/render_weekly_curator_report.py`, `tests/verify_weekly_curator.py`, `reports/skillops/weekly/2026-W27.json`, `reports/skillops/weekly/2026-W27.md` - next action: Use weekly curator output as the Skill Librarian maintenance queue before approving any durable skill-library changes. diff --git a/reports/skillops/daily/2026-07-02.json b/reports/skillops/daily/2026-07-02.json new file mode 100644 index 00000000..82e729ad --- /dev/null +++ b/reports/skillops/daily/2026-07-02.json @@ -0,0 +1,513 @@ +{ + "schema_version": "1.0", + "ok": true, + "generated_at": "2026-07-02T03:41:56Z", + "skill_dir": ".", + "decision": "proposal-review", + "source_supplied": false, + "pattern_count": 5, + "proposal_count": 5, + "approval_count": 0, + "pending_review_count": 0, + "applied_count": 0, + "rollback_count": 0, + "local_blueprint_ready": true, + "public_world_class_ready": false, + "world_class_pending_count": 4, + "release_lock_ready": false, + "evidence_consistency_ok": true, + "writes_source_files": false, + "auto_patch_enabled": false, + "failure_count": 0, + "summary": { + "decision": "proposal-review", + "source_supplied": false, + "pattern_count": 5, + "proposal_count": 5, + "approval_count": 0, + "pending_review_count": 0, + "applied_count": 0, + "rollback_count": 0, + "local_blueprint_ready": true, + "public_world_class_ready": false, + "world_class_pending_count": 4, + "release_lock_ready": false, + "evidence_consistency_ok": true, + "writes_source_files": false, + "auto_patch_enabled": false, + "failure_count": 0 + }, + "operations_contract": { + "schema_version": "1.0", + "contract": "daily-skillops-report", + "explicit_source_required_for_scan": true, + "implicit_private_log_scan": false, + "raw_content_stored": false, + "redacted_excerpts_only": true, + "proposal_only": true, + "approval_required_for_writes": true, + "writes_source_files": false, + "auto_patch_enabled": false, + "daily_report_counts_as_world_class_evidence": false + }, + "report_contract": { + "schema_version": "1.0", + "contract": "daily-skillops-report", + "top_level_mirrors_summary": true, + "summary_fields": [ + "decision", + "source_supplied", + "pattern_count", + "proposal_count", + "approval_count", + "pending_review_count", + "applied_count", + "rollback_count", + "local_blueprint_ready", + "public_world_class_ready", + "world_class_pending_count", + "release_lock_ready", + "evidence_consistency_ok", + "writes_source_files", + "auto_patch_enabled", + "failure_count" + ], + "source_of_truth": [ + "summary", + "operations_contract" + ] + }, + "source": { + "path": "evals/adaptation/user_signals.example.jsonl", + "fingerprint_sha256": "48f94a4f3b82f90db29921c13d09c765ec10aabfde1adce65710c137cc6c5abf", + "record_count": 8, + "explicit_source": true, + "raw_content_stored": false, + "redacted_excerpts_only": true + }, + "patterns": [ + { + "pattern_id": "language_default", + "label": "Default language preference", + "signal_type": "report-language", + "support_count": 2, + "confidence": 0.79, + "reason": "2 redacted records matched repeated report-language signals.", + "recommended_action": "Keep generated reports Chinese-first with an English switch where user-facing.", + "evidence": [ + { + "record_id": "line-1", + "excerpt": "新生成的 Skill 报告默认使用中文简体，并在右上角提供英文切换。" + }, + { + "record_id": "line-2", + "excerpt": "HTML 报告需要双语能力，但默认内容应该保持中文简体。" + } + ] + }, + { + "pattern_id": "report_ui", + "label": "Report UI and visualization preference", + "signal_type": "artifact-design", + "support_count": 5, + "confidence": 0.95, + "reason": "5 redacted records matched repeated artifact-design signals.", + "recommended_action": "Prioritize white-background Kami-style reports with readable charts and stable navigation.", + "evidence": [ + { + "record_id": "line-1", + "excerpt": "新生成的 Skill 报告默认使用中文简体，并在右上角提供英文切换。" + }, + { + "record_id": "line-2", + "excerpt": "HTML 报告需要双语能力，但默认内容应该保持中文简体。" + }, + { + "record_id": "line-3", + "excerpt": "报告排版采用白底 Kami 风格，图表、模块和导航都要清晰。" + } + ] + }, + { + "pattern_id": "approval_safety", + "label": "Approval and privacy boundary", + "signal_type": "governance", + "support_count": 2, + "confidence": 0.79, + "reason": "2 redacted records matched repeated governance signals.", + "recommended_action": "Keep adaptive work proposal-only until a reviewer approves an allowlisted patch path.", + "evidence": [ + { + "record_id": "line-5", + "excerpt": "自适应升级必须先生成提案，不能直接自动修改源文件。" + }, + { + "record_id": "line-6", + "excerpt": "用户偏好扫描必须由用户提供明确路径，不要默认扫描私人日志。" + } + ] + }, + { + "pattern_id": "delivery_format", + "label": "Delivery format preference", + "signal_type": "artifact-format", + "support_count": 2, + "confidence": 0.79, + "reason": "2 redacted records matched repeated artifact-format signals.", + "recommended_action": "Surface stable artifact paths and formats in CLI output and generated summaries.", + "evidence": [ + { + "record_id": "line-2", + "excerpt": "HTML 报告需要双语能力，但默认内容应该保持中文简体。" + }, + { + "record_id": "line-6", + "excerpt": "用户偏好扫描必须由用户提供明确路径，不要默认扫描私人日志。" + } + ] + }, + { + "pattern_id": "evidence_testing", + "label": "Evidence and testing preference", + "signal_type": "quality-gate", + "support_count": 2, + "confidence": 0.79, + "reason": "2 redacted records matched repeated quality-gate signals.", + "recommended_action": "Attach focused tests and refreshed evidence reports to every non-trivial skill upgrade.", + "evidence": [ + { + "record_id": "line-7", + "excerpt": "每次升级都需要测试、覆盖报告和可审计证据，推送前要跑 CI。" + }, + { + "record_id": "line-8", + "excerpt": "涉及 GitHub 推送时，要保留证据链，避免把计划当作完成证明。" + } + ] + } + ], + "proposals": [ + { + "proposal_id": "adapt-18c7517f3d", + "pattern_id": "language_default", + "title": "Keep reports Chinese-first with optional English", + "risk_level": "low", + "status": "proposal-only", + "requires_approval": true, + "target_files": [ + "scripts/render_skill_overview.py", + "references/artifact-design-doctrine.md" + ], + "verification_commands": [ + "python3 tests/verify_skill_overview.py" + ] + }, + { + "proposal_id": "adapt-fbfe921ba5", + "pattern_id": "report_ui", + "title": "Improve report layout, visual hierarchy, and chart readability", + "risk_level": "medium", + "status": "proposal-only", + "requires_approval": true, + "target_files": [ + "scripts/render_skill_overview.py", + "references/artifact-design-doctrine.md", + "tests/verify_skill_overview.py" + ], + "verification_commands": [ + "python3 tests/verify_skill_overview.py", + "python3 tests/verify_skill_report_charts.py" + ] + }, + { + "proposal_id": "adapt-59d219a1fb", + "pattern_id": "approval_safety", + "title": "Keep adaptive iteration approval-gated", + "risk_level": "low", + "status": "proposal-only", + "requires_approval": true, + "target_files": [ + "references/user-memory-policy.md", + "references/autonomous-adaptation.md", + "schemas/adaptation-proposal.schema.json" + ], + "verification_commands": [ + "python3 tests/verify_adaptation_safety.py" + ] + }, + { + "proposal_id": "adapt-457baca160", + "pattern_id": "delivery_format", + "title": "Make generated artifact paths explicit in CLI output", + "risk_level": "low", + "status": "proposal-only", + "requires_approval": true, + "target_files": [ + "scripts/yao.py", + "README.md" + ], + "verification_commands": [ + "python3 tests/verify_yao_cli.py" + ] + }, + { + "proposal_id": "adapt-abfee25d3a", + "pattern_id": "evidence_testing", + "title": "Attach tests and evidence refresh to each upgrade", + "risk_level": "medium", + "status": "proposal-only", + "requires_approval": true, + "target_files": [ + "tests/verify_adaptation_safety.py", + "scripts/render_skill_os2_coverage.py", + "reports/skill_os2_coverage.json" + ], + "verification_commands": [ + "python3 tests/verify_adaptation_safety.py", + "python3 tests/verify_skill_os2_coverage.py" + ] + } + ], + "opportunity_summary": { + "opportunity_count": 5, + "top_score": 88, + "ready_for_approval_review_count": 1, + "proposal_review_count": 4, + "observe_more_evidence_count": 0, + "report_only_count": 0, + "action_type_counts": { + "patch_existing_skill": 3, + "agents_update": 1, + "add_eval": 1 + }, + "decision_counts": { + "ready_for_approval_review": 1, + "proposal_review": 4 + } + }, + "opportunities": [ + { + "opportunity_id": "skillops-4ad91c18dd", + "proposal_id": "adapt-fbfe921ba5", + "pattern_id": "report_ui", + "title": "Improve report layout, visual hierarchy, and chart readability", + "opportunity_type": "artifact-quality", + "action_type": "patch_existing_skill", + "decision": "ready_for_approval_review", + "priority": "medium", + "score": 88, + "score_reasons": [ + "support_count=5", + "confidence=0.95", + "risk=medium", + "target_files_present", + "verification_present", + "approval_required" + ], + "policy_reason": "Repeated report layout feedback maps to existing renderer and design doctrine changes.", + "risk_level": "medium", + "requires_approval": true, + "write_allowed_without_approval": false, + "evidence_count": 5, + "target_files": [ + "scripts/render_skill_overview.py", + "references/artifact-design-doctrine.md", + "tests/verify_skill_overview.py" + ], + "verification_commands": [ + "python3 tests/verify_skill_overview.py", + "python3 tests/verify_skill_report_charts.py" + ] + }, + { + "opportunity_id": "skillops-531e13c1c6", + "proposal_id": "adapt-59d219a1fb", + "pattern_id": "approval_safety", + "title": "Keep adaptive iteration approval-gated", + "opportunity_type": "governance", + "action_type": "agents_update", + "decision": "proposal_review", + "priority": "medium", + "score": 82, + "score_reasons": [ + "support_count=2", + "confidence=0.79", + "risk=low", + "target_files_present", + "verification_present", + "approval_required" + ], + "policy_reason": "Repeated privacy and approval signals should tighten durable operating guidance.", + "risk_level": "low", + "requires_approval": true, + "write_allowed_without_approval": false, + "evidence_count": 2, + "target_files": [ + "references/user-memory-policy.md", + "references/autonomous-adaptation.md", + "schemas/adaptation-proposal.schema.json" + ], + "verification_commands": [ + "python3 tests/verify_adaptation_safety.py" + ] + }, + { + "opportunity_id": "skillops-709ef884f3", + "proposal_id": "adapt-18c7517f3d", + "pattern_id": "language_default", + "title": "Keep reports Chinese-first with optional English", + "opportunity_type": "report-experience", + "action_type": "patch_existing_skill", + "decision": "proposal_review", + "priority": "medium", + "score": 82, + "score_reasons": [ + "support_count=2", + "confidence=0.79", + "risk=low", + "target_files_present", + "verification_present", + "approval_required" + ], + "policy_reason": "Repeated language preferences can improve existing report templates after approval.", + "risk_level": "low", + "requires_approval": true, + "write_allowed_without_approval": false, + "evidence_count": 2, + "target_files": [ + "scripts/render_skill_overview.py", + "references/artifact-design-doctrine.md" + ], + "verification_commands": [ + "python3 tests/verify_skill_overview.py" + ] + }, + { + "opportunity_id": "skillops-f9e43237aa", + "proposal_id": "adapt-457baca160", + "pattern_id": "delivery_format", + "title": "Make generated artifact paths explicit in CLI output", + "opportunity_type": "artifact-discoverability", + "action_type": "patch_existing_skill", + "decision": "proposal_review", + "priority": "medium", + "score": 82, + "score_reasons": [ + "support_count=2", + "confidence=0.79", + "risk=low", + "target_files_present", + "verification_present", + "approval_required" + ], + "policy_reason": "Repeated delivery-path requests can improve CLI and README artifact discoverability.", + "risk_level": "low", + "requires_approval": true, + "write_allowed_without_approval": false, + "evidence_count": 2, + "target_files": [ + "scripts/yao.py", + "README.md" + ], + "verification_commands": [ + "python3 tests/verify_yao_cli.py" + ] + }, + { + "opportunity_id": "skillops-0a3a2de7c3", + "proposal_id": "adapt-abfee25d3a", + "pattern_id": "evidence_testing", + "title": "Attach tests and evidence refresh to each upgrade", + "opportunity_type": "quality-gate", + "action_type": "add_eval", + "decision": "proposal_review", + "priority": "medium", + "score": 74, + "score_reasons": [ + "support_count=2", + "confidence=0.79", + "risk=medium", + "target_files_present", + "verification_present", + "approval_required" + ], + "policy_reason": "Repeated verification and evidence requests should become focused checks.", + "risk_level": "medium", + "requires_approval": true, + "write_allowed_without_approval": false, + "evidence_count": 2, + "target_files": [ + "tests/verify_adaptation_safety.py", + "scripts/render_skill_os2_coverage.py", + "reports/skill_os2_coverage.json" + ], + "verification_commands": [ + "python3 tests/verify_adaptation_safety.py", + "python3 tests/verify_skill_os2_coverage.py" + ] + } + ], + "decision_policy": { + "schema_version": "1.0", + "contract": "skillops-opportunity-scoring", + "score_range": [ + 0, + 100 + ], + "score_bands": { + "85-100": "ready_for_approval_review", + "70-84": "proposal_review", + "50-69": "observe_more_evidence", + "0-49": "report_only_or_no_action" + }, + "action_types": [ + "add_eval", + "agents_update", + "patch_existing_skill", + "report_only" + ], + "writes_source_files": false, + "auto_patch_enabled": false, + "approval_required_for_writes": true + }, + "approval": { + "approval_count": 0, + "pending_review_count": 0, + "applied_count": 0, + "rollback_count": 0 + }, + "release_state": { + "local_blueprint_ready": true, + "public_world_class_ready": false, + "world_class_pending_count": 4, + "release_lock_ready": false, + "evidence_consistency_ok": true + }, + "actions": [ + { + "key": "review-adaptation-proposals", + "priority": "high", + "action": "Review proposal-only adaptation items before preparing any approval ledger entry." + }, + { + "key": "close-world-class-evidence", + "priority": "high", + "action": "Collect accepted external or human evidence for the pending world-class ledger entries." + } + ], + "failures": [], + "source_reports": { + "patterns": "reports/user_patterns.json", + "proposals": "reports/adaptation_proposals.json", + "approval_ledger": "reports/adaptation_approval_ledger.json", + "regression": "reports/adaptation_regression_report.json", + "skill_os2_coverage": "reports/skill_os2_coverage.json", + "world_class_ledger": "reports/world_class_evidence_ledger.json", + "evidence_consistency": "reports/evidence_consistency.json", + "benchmark_reproducibility": "reports/benchmark_reproducibility.json" + }, + "artifacts": { + "json": "reports/skillops/daily/2026-07-02.json", + "markdown": "reports/skillops/daily/2026-07-02.md" + } +} diff --git a/reports/skillops/daily/2026-07-02.md b/reports/skillops/daily/2026-07-02.md new file mode 100644 index 00000000..a3c85a2f --- /dev/null +++ b/reports/skillops/daily/2026-07-02.md @@ -0,0 +1,174 @@ +# Daily SkillOps Report + +Generated at: `2026-07-02T03:41:56Z` + +## Summary + +- decision: `proposal-review` +- source supplied: `false` +- patterns: `5` +- proposals: `5` +- pending approvals: `0` +- applied patches: `0` +- rollbacks: `0` +- local blueprint ready: `true` +- public world-class ready: `false` +- world-class pending: `4` +- release lock ready: `false` +- evidence consistency ok: `true` + +This report is an operations cockpit for explicit-source SkillOps. It does not scan private logs, write source files, apply patches, or count as world-class external or human evidence. + +## Privacy Boundary + +- explicit_source_required_for_scan: `true` +- implicit_private_log_scan: `false` +- raw_content_stored: `false` +- redacted_excerpts_only: `true` +- proposal_only: `true` +- approval_required_for_writes: `true` +- writes_source_files: `false` +- auto_patch_enabled: `false` +- daily_report_counts_as_world_class_evidence: `false` + +## Actions + +- `high` Review proposal-only adaptation items before preparing any approval ledger entry. +- `high` Collect accepted external or human evidence for the pending world-class ledger entries. + +## Opportunities + +- count: `5` +- top score: `88` +- ready for approval review: `1` +### Improve report layout, visual hierarchy, and chart readability + +- ID: `skillops-4ad91c18dd` +- Action: `patch_existing_skill` +- Decision: `ready_for_approval_review` +- Score: `88` +- Risk: `medium` +- Policy: Repeated report layout feedback maps to existing renderer and design doctrine changes. +### Keep adaptive iteration approval-gated + +- ID: `skillops-531e13c1c6` +- Action: `agents_update` +- Decision: `proposal_review` +- Score: `82` +- Risk: `low` +- Policy: Repeated privacy and approval signals should tighten durable operating guidance. +### Keep reports Chinese-first with optional English + +- ID: `skillops-709ef884f3` +- Action: `patch_existing_skill` +- Decision: `proposal_review` +- Score: `82` +- Risk: `low` +- Policy: Repeated language preferences can improve existing report templates after approval. +### Make generated artifact paths explicit in CLI output + +- ID: `skillops-f9e43237aa` +- Action: `patch_existing_skill` +- Decision: `proposal_review` +- Score: `82` +- Risk: `low` +- Policy: Repeated delivery-path requests can improve CLI and README artifact discoverability. +### Attach tests and evidence refresh to each upgrade + +- ID: `skillops-0a3a2de7c3` +- Action: `add_eval` +- Decision: `proposal_review` +- Score: `74` +- Risk: `medium` +- Policy: Repeated verification and evidence requests should become focused checks. + +## Patterns + +- `language_default`: Default language preference (support `2`, confidence `0.79`) +- `report_ui`: Report UI and visualization preference (support `5`, confidence `0.95`) +- `approval_safety`: Approval and privacy boundary (support `2`, confidence `0.79`) +- `delivery_format`: Delivery format preference (support `2`, confidence `0.79`) +- `evidence_testing`: Evidence and testing preference (support `2`, confidence `0.79`) + +## Proposals + +### Keep reports Chinese-first with optional English + +- ID: `adapt-18c7517f3d` +- Pattern: `language_default` +- Status: `proposal-only` +- Risk: `low` +- Requires approval: `true` +- Target files: + - `scripts/render_skill_overview.py` + - `references/artifact-design-doctrine.md` +- Verification: + - `python3 tests/verify_skill_overview.py` + +### Improve report layout, visual hierarchy, and chart readability + +- ID: `adapt-fbfe921ba5` +- Pattern: `report_ui` +- Status: `proposal-only` +- Risk: `medium` +- Requires approval: `true` +- Target files: + - `scripts/render_skill_overview.py` + - `references/artifact-design-doctrine.md` + - `tests/verify_skill_overview.py` +- Verification: + - `python3 tests/verify_skill_overview.py` + - `python3 tests/verify_skill_report_charts.py` + +### Keep adaptive iteration approval-gated + +- ID: `adapt-59d219a1fb` +- Pattern: `approval_safety` +- Status: `proposal-only` +- Risk: `low` +- Requires approval: `true` +- Target files: + - `references/user-memory-policy.md` + - `references/autonomous-adaptation.md` + - `schemas/adaptation-proposal.schema.json` +- Verification: + - `python3 tests/verify_adaptation_safety.py` + +### Make generated artifact paths explicit in CLI output + +- ID: `adapt-457baca160` +- Pattern: `delivery_format` +- Status: `proposal-only` +- Risk: `low` +- Requires approval: `true` +- Target files: + - `scripts/yao.py` + - `README.md` +- Verification: + - `python3 tests/verify_yao_cli.py` + +### Attach tests and evidence refresh to each upgrade + +- ID: `adapt-abfee25d3a` +- Pattern: `evidence_testing` +- Status: `proposal-only` +- Risk: `medium` +- Requires approval: `true` +- Target files: + - `tests/verify_adaptation_safety.py` + - `scripts/render_skill_os2_coverage.py` + - `reports/skill_os2_coverage.json` +- Verification: + - `python3 tests/verify_adaptation_safety.py` + - `python3 tests/verify_skill_os2_coverage.py` + +## Evidence + +- patterns: `reports/user_patterns.json` +- proposals: `reports/adaptation_proposals.json` +- approval_ledger: `reports/adaptation_approval_ledger.json` +- regression: `reports/adaptation_regression_report.json` +- skill_os2_coverage: `reports/skill_os2_coverage.json` +- world_class_ledger: `reports/world_class_evidence_ledger.json` +- evidence_consistency: `reports/evidence_consistency.json` +- benchmark_reproducibility: `reports/benchmark_reproducibility.json` diff --git a/reports/skillops/weekly/2026-W27.json b/reports/skillops/weekly/2026-W27.json new file mode 100644 index 00000000..8830accc --- /dev/null +++ b/reports/skillops/weekly/2026-W27.json @@ -0,0 +1,323 @@ +{ + "schema_version": "1.0", + "ok": true, + "generated_at": "2026-07-02T03:41:56Z", + "skill_dir": ".", + "decision": "curator-review", + "week_id": "2026-W27", + "daily_report_count": 5, + "opportunity_count": 25, + "unique_opportunity_count": 5, + "ready_for_approval_review_count": 1, + "proposal_review_count": 4, + "observe_more_evidence_count": 0, + "report_only_count": 0, + "top_score": 88, + "skill_count": 11, + "actionable_portfolio_issue_count": 0, + "release_lock_ready": false, + "evidence_consistency_ok": true, + "public_world_class_ready": false, + "world_class_pending_count": 4, + "writes_source_files": false, + "auto_patch_enabled": false, + "failure_count": 0, + "summary": { + "decision": "curator-review", + "week_id": "2026-W27", + "daily_report_count": 5, + "opportunity_count": 25, + "unique_opportunity_count": 5, + "ready_for_approval_review_count": 1, + "proposal_review_count": 4, + "observe_more_evidence_count": 0, + "report_only_count": 0, + "top_score": 88, + "skill_count": 11, + "actionable_portfolio_issue_count": 0, + "release_lock_ready": false, + "evidence_consistency_ok": true, + "public_world_class_ready": false, + "world_class_pending_count": 4, + "writes_source_files": false, + "auto_patch_enabled": false, + "failure_count": 0 + }, + "curator_contract": { + "schema_version": "1.0", + "contract": "weekly-skillops-curator-report", + "cadence": "weekly", + "source_of_truth": [ + "reports/skillops/daily/*.json", + "reports/skill_atlas.json", + "reports/benchmark_reproducibility.json", + "reports/evidence_consistency.json", + "reports/world_class_evidence_ledger.json" + ], + "raw_content_stored": false, + "redacted_or_generated_evidence_only": true, + "proposal_only": true, + "writes_source_files": false, + "auto_patch_enabled": false, + "approval_required_for_writes": true, + "counts_as_world_class_evidence": false + }, + "report_contract": { + "schema_version": "1.0", + "contract": "weekly-skillops-curator-report", + "top_level_mirrors_summary": true, + "summary_fields": [ + "decision", + "week_id", + "daily_report_count", + "opportunity_count", + "unique_opportunity_count", + "ready_for_approval_review_count", + "proposal_review_count", + "observe_more_evidence_count", + "report_only_count", + "top_score", + "skill_count", + "actionable_portfolio_issue_count", + "release_lock_ready", + "evidence_consistency_ok", + "public_world_class_ready", + "world_class_pending_count", + "writes_source_files", + "auto_patch_enabled", + "failure_count" + ] + }, + "daily_reports": [ + "reports/skillops/daily/2026-06-16.json", + "reports/skillops/daily/2026-06-17.json", + "reports/skillops/daily/2026-06-20.json", + "reports/skillops/daily/2026-06-21.json", + "reports/skillops/daily/2026-07-02.json" + ], + "opportunity_summary": { + "action_type_counts": { + "add_eval": 1, + "agents_update": 1, + "patch_existing_skill": 3 + }, + "decision_counts": { + "proposal_review": 4, + "ready_for_approval_review": 1 + }, + "risk_level_counts": { + "low": 3, + "medium": 2 + } + }, + "curator_queue": [ + { + "opportunity_id": "skillops-4ad91c18dd", + "proposal_id": "adapt-fbfe921ba5", + "pattern_id": "report_ui", + "title": "Improve report layout, visual hierarchy, and chart readability", + "opportunity_type": "artifact-quality", + "action_type": "patch_existing_skill", + "decision": "ready_for_approval_review", + "priority": "medium", + "score": 88, + "score_reasons": [ + "support_count=5", + "confidence=0.95", + "risk=medium", + "target_files_present", + "verification_present", + "approval_required" + ], + "policy_reason": "Repeated report layout feedback maps to existing renderer and design doctrine changes.", + "risk_level": "medium", + "requires_approval": true, + "write_allowed_without_approval": false, + "evidence_count": 5, + "target_files": [ + "scripts/render_skill_overview.py", + "references/artifact-design-doctrine.md", + "tests/verify_skill_overview.py" + ], + "verification_commands": [ + "python3 tests/verify_skill_overview.py", + "python3 tests/verify_skill_report_charts.py" + ], + "daily_report_date": "2026-06-16" + }, + { + "opportunity_id": "skillops-531e13c1c6", + "proposal_id": "adapt-59d219a1fb", + "pattern_id": "approval_safety", + "title": "Keep adaptive iteration approval-gated", + "opportunity_type": "governance", + "action_type": "agents_update", + "decision": "proposal_review", + "priority": "medium", + "score": 82, + "score_reasons": [ + "support_count=2", + "confidence=0.79", + "risk=low", + "target_files_present", + "verification_present", + "approval_required" + ], + "policy_reason": "Repeated privacy and approval signals should tighten durable operating guidance.", + "risk_level": "low", + "requires_approval": true, + "write_allowed_without_approval": false, + "evidence_count": 2, + "target_files": [ + "references/user-memory-policy.md", + "references/autonomous-adaptation.md", + "schemas/adaptation-proposal.schema.json" + ], + "verification_commands": [ + "python3 tests/verify_adaptation_safety.py" + ], + "daily_report_date": "2026-06-16" + }, + { + "opportunity_id": "skillops-709ef884f3", + "proposal_id": "adapt-18c7517f3d", + "pattern_id": "language_default", + "title": "Keep reports Chinese-first with optional English", + "opportunity_type": "report-experience", + "action_type": "patch_existing_skill", + "decision": "proposal_review", + "priority": "medium", + "score": 82, + "score_reasons": [ + "support_count=2", + "confidence=0.79", + "risk=low", + "target_files_present", + "verification_present", + "approval_required" + ], + "policy_reason": "Repeated language preferences can improve existing report templates after approval.", + "risk_level": "low", + "requires_approval": true, + "write_allowed_without_approval": false, + "evidence_count": 2, + "target_files": [ + "scripts/render_skill_overview.py", + "references/artifact-design-doctrine.md" + ], + "verification_commands": [ + "python3 tests/verify_skill_overview.py" + ], + "daily_report_date": "2026-06-16" + }, + { + "opportunity_id": "skillops-f9e43237aa", + "proposal_id": "adapt-457baca160", + "pattern_id": "delivery_format", + "title": "Make generated artifact paths explicit in CLI output", + "opportunity_type": "artifact-discoverability", + "action_type": "patch_existing_skill", + "decision": "proposal_review", + "priority": "medium", + "score": 82, + "score_reasons": [ + "support_count=2", + "confidence=0.79", + "risk=low", + "target_files_present", + "verification_present", + "approval_required" + ], + "policy_reason": "Repeated delivery-path requests can improve CLI and README artifact discoverability.", + "risk_level": "low", + "requires_approval": true, + "write_allowed_without_approval": false, + "evidence_count": 2, + "target_files": [ + "scripts/yao.py", + "README.md" + ], + "verification_commands": [ + "python3 tests/verify_yao_cli.py" + ], + "daily_report_date": "2026-06-16" + }, + { + "opportunity_id": "skillops-0a3a2de7c3", + "proposal_id": "adapt-abfee25d3a", + "pattern_id": "evidence_testing", + "title": "Attach tests and evidence refresh to each upgrade", + "opportunity_type": "quality-gate", + "action_type": "add_eval", + "decision": "proposal_review", + "priority": "medium", + "score": 74, + "score_reasons": [ + "support_count=2", + "confidence=0.79", + "risk=medium", + "target_files_present", + "verification_present", + "approval_required" + ], + "policy_reason": "Repeated verification and evidence requests should become focused checks.", + "risk_level": "medium", + "requires_approval": true, + "write_allowed_without_approval": false, + "evidence_count": 2, + "target_files": [ + "tests/verify_adaptation_safety.py", + "scripts/render_skill_os2_coverage.py", + "reports/skill_os2_coverage.json" + ], + "verification_commands": [ + "python3 tests/verify_adaptation_safety.py", + "python3 tests/verify_skill_os2_coverage.py" + ], + "daily_report_date": "2026-06-16" + } + ], + "portfolio": { + "skill_count": 11, + "actionable_issue_count": 0, + "route_collision_count": 5, + "actionable_route_collision_count": 0, + "owner_gap_count": 8, + "actionable_owner_gap_count": 0, + "stale_count": 9, + "actionable_stale_count": 0, + "drift_signal_count": 0, + "actionable_drift_signal_count": 0, + "no_route_opportunity_count": 0 + }, + "release_state": { + "release_lock_ready": false, + "evidence_consistency_ok": true, + "public_world_class_ready": false, + "world_class_pending_count": 4 + }, + "actions": [ + { + "key": "review-ready-opportunities", + "priority": "high", + "action": "Review the top ready-for-approval SkillOps opportunities before preparing any approval ledger entry." + }, + { + "key": "close-world-class-evidence", + "priority": "high", + "action": "Collect accepted external or human evidence for pending world-class ledger entries before public claims." + } + ], + "failures": [], + "source_reports": { + "daily_dir": "reports/skillops/daily", + "skill_atlas": "reports/skill_atlas.json", + "benchmark_reproducibility": "reports/benchmark_reproducibility.json", + "evidence_consistency": "reports/evidence_consistency.json", + "world_class_ledger": "reports/world_class_evidence_ledger.json" + }, + "artifacts": { + "json": "reports/skillops/weekly/2026-W27.json", + "markdown": "reports/skillops/weekly/2026-W27.md" + } +} diff --git a/reports/skillops/weekly/2026-W27.md b/reports/skillops/weekly/2026-W27.md new file mode 100644 index 00000000..13f65fba --- /dev/null +++ b/reports/skillops/weekly/2026-W27.md @@ -0,0 +1,110 @@ +# Weekly SkillOps Curator Report + +Generated at: `2026-07-02T03:41:56Z` +Week: `2026-W27` + +## Summary + +- decision: `curator-review` +- daily_report_count: `5` +- unique_opportunity_count: `5` +- ready_for_approval_review_count: `1` +- proposal_review_count: `4` +- top_score: `88` +- skill_count: `11` +- actionable_portfolio_issue_count: `0` +- release_lock_ready: `False` +- evidence_consistency_ok: `True` +- public_world_class_ready: `False` +- world_class_pending_count: `4` + +This report is a weekly curator cockpit for generated SkillOps evidence. It does not scan private logs, write source files, apply patches, or count as world-class evidence. + +## Curator Boundary + +- raw_content_stored: `false` +- redacted_or_generated_evidence_only: `true` +- proposal_only: `true` +- writes_source_files: `false` +- auto_patch_enabled: `false` +- approval_required_for_writes: `true` +- counts_as_world_class_evidence: `false` + +## Actions + +- `high` Review the top ready-for-approval SkillOps opportunities before preparing any approval ledger entry. +- `high` Collect accepted external or human evidence for pending world-class ledger entries before public claims. + +## Curator Queue + +### Improve report layout, visual hierarchy, and chart readability + +- ID: `skillops-4ad91c18dd` +- Action: `patch_existing_skill` +- Decision: `ready_for_approval_review` +- Score: `88` +- Risk: `medium` +- Source day: `2026-06-16` +- Policy: Repeated report layout feedback maps to existing renderer and design doctrine changes. + +### Keep adaptive iteration approval-gated + +- ID: `skillops-531e13c1c6` +- Action: `agents_update` +- Decision: `proposal_review` +- Score: `82` +- Risk: `low` +- Source day: `2026-06-16` +- Policy: Repeated privacy and approval signals should tighten durable operating guidance. + +### Keep reports Chinese-first with optional English + +- ID: `skillops-709ef884f3` +- Action: `patch_existing_skill` +- Decision: `proposal_review` +- Score: `82` +- Risk: `low` +- Source day: `2026-06-16` +- Policy: Repeated language preferences can improve existing report templates after approval. + +### Make generated artifact paths explicit in CLI output + +- ID: `skillops-f9e43237aa` +- Action: `patch_existing_skill` +- Decision: `proposal_review` +- Score: `82` +- Risk: `low` +- Source day: `2026-06-16` +- Policy: Repeated delivery-path requests can improve CLI and README artifact discoverability. + +### Attach tests and evidence refresh to each upgrade + +- ID: `skillops-0a3a2de7c3` +- Action: `add_eval` +- Decision: `proposal_review` +- Score: `74` +- Risk: `medium` +- Source day: `2026-06-16` +- Policy: Repeated verification and evidence requests should become focused checks. + +## Portfolio Signals + +- skill_count: `11` +- actionable_issue_count: `0` +- route_collision_count: `5` +- actionable_route_collision_count: `0` +- owner_gap_count: `8` +- actionable_owner_gap_count: `0` +- stale_count: `9` +- actionable_stale_count: `0` +- drift_signal_count: `0` +- actionable_drift_signal_count: `0` +- no_route_opportunity_count: `0` + +## Evidence + +- daily_dir: `reports/skillops/daily` +- skill_atlas: `reports/skill_atlas.json` +- benchmark_reproducibility: `reports/benchmark_reproducibility.json` +- evidence_consistency: `reports/evidence_consistency.json` +- world_class_ledger: `reports/world_class_evidence_ledger.json` diff --git a/reports/system-model.json b/reports/system-model.json index 0385a0ec..a34ae0c7 100644 --- a/reports/system-model.json +++ b/reports/system-model.json @@ -1,6 +1,6 @@ { "skill_name": "yao-meta-skill", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "systems_doctrine": "Structure drives behavior: improve the boundary, feedback loops, drift watch, and leverage points before adding weight.", "stability": { "score": 100, @@ -56,7 +56,7 @@ "evidence": "reports/reference-synthesis.md", "current_patterns": [ "Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts.", - "Borrow a review checkpoint wherever trust matters more than raw speed.", + "Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed.", "Borrow the discipline of defining what the skill should not own before growing the package.", "Borrow the way it turns a messy workflow into a repeatable operating path.", "Borrow the clear execution entrypoints and command structure." diff --git a/reports/system-model.md b/reports/system-model.md index 4be3a0fb..17f31f82 100644 --- a/reports/system-model.md +++ b/reports/system-model.md @@ -53,7 +53,7 @@ Skill: `yao-meta-skill` - Evidence: reports/reference-synthesis.md - Current patterns: - Borrow progressive disclosure: keep the entrypoint lean and move depth into references or scripts. - - Borrow a review checkpoint wherever trust matters more than raw speed. + - Borrow a small hypothesis-test-learn loop so the first revision is evidence-backed. - Borrow the discipline of defining what the skill should not own before growing the package. - Borrow the way it turns a messy workflow into a repeatable operating path. - Borrow the clear execution entrypoints and command structure. diff --git a/reports/upgrade_check.json b/reports/upgrade_check.json index d955bb57..e824cd2c 100644 --- a/reports/upgrade_check.json +++ b/reports/upgrade_check.json @@ -1,7 +1,7 @@ { "ok": true, "schema_version": "2.0", - "generated_at": "2026-06-21", + "generated_at": "2026-06-13", "previous": { "name": "yao-meta-skill", "version": "1.0.0", @@ -63,7 +63,7 @@ { "field": "description", "from": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes.", - "to": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution." + "to": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs." } ], "checksum_changes": [ @@ -75,7 +75,7 @@ { "field": "package_sha256", "from": "0000000000000000000000000000000000000000000000000000000000000000", - "to": "eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383" + "to": "e0863533e0a0f0b5069bace6a24d7adfb43d615453f08cff7578076a3cb1a2a3" } ] }, @@ -90,7 +90,7 @@ "artifacts": { "previous_package": "registry/examples/yao-meta-skill-1.0.0.json", "current_package": "reports/registry_audit.json", - "json": "reports/upgrade_check.json", - "markdown": "reports/upgrade_check.md" + "json": "tests/tmp_review_studio/upgrade_check.json", + "markdown": "tests/tmp_review_studio/upgrade_check.md" } } diff --git a/reports/world_class_claim_guard.json b/reports/world_class_claim_guard.json index 045e8898..482c64c4 100644 --- a/reports/world_class_claim_guard.json +++ b/reports/world_class_claim_guard.json @@ -1,15 +1,15 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "ledger_ready_to_claim_world_class": false, "ledger_pending_count": 4, - "claim_surface_count": 184, - "json_claim_surface_count": 91, - "metadata_claim_surface_count": 92, - "package_claim_surface_count": 17, + "claim_surface_count": 194, + "json_claim_surface_count": 92, + "metadata_claim_surface_count": 93, + "package_claim_surface_count": 10, "violation_count": 0, "overclaim_guard_active": true, "decision": "claim-guard-pass-evidence-pending" @@ -54,34 +54,6 @@ "path": "agents/interface.yaml", "violation_count": 0 }, - { - "path": "dist/manifest.json", - "violation_count": 0 - }, - { - "path": "dist/targets/claude/README.md", - "violation_count": 0 - }, - { - "path": "dist/targets/claude/adapter.json", - "violation_count": 0 - }, - { - "path": "dist/targets/generic/adapter.json", - "violation_count": 0 - }, - { - "path": "dist/targets/openai/adapter.json", - "violation_count": 0 - }, - { - "path": "dist/targets/vscode/README.md", - "violation_count": 0 - }, - { - "path": "dist/targets/vscode/adapter.json", - "violation_count": 0 - }, { "path": "docs/README.fr-FR.md", "violation_count": 0 @@ -214,6 +186,58 @@ "path": "reports/benchmark_reproducibility.md", "violation_count": 0 }, + { + "path": "reports/blind-human-review-2026-06-29/DO_NOT_OPEN_answer_key.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/DO_NOT_OPEN_answer_key.json", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/README.md", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/adjudication.md", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/blind-pack.json", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/case-01-support-triage.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/case-02-monthly-revenue-reconciliation.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/case-03-webinar-repurposing.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/case-04-incident-postmortem.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/case-05-pr-review-followup.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/index.html", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/review-decisions-template.json", + "violation_count": 0 + }, + { + "path": "reports/blind-human-review-2026-06-29/review-decisions.recorded.json", + "violation_count": 0 + }, { "path": "reports/candidate_registry.json", "violation_count": 0 @@ -638,6 +662,14 @@ "path": "reports/skillops/daily/2026-06-21.md", "violation_count": 0 }, + { + "path": "reports/skillops/daily/2026-07-02.json", + "violation_count": 0 + }, + { + "path": "reports/skillops/daily/2026-07-02.md", + "violation_count": 0 + }, { "path": "reports/skillops/weekly/2026-W25.json", "violation_count": 0 @@ -646,6 +678,14 @@ "path": "reports/skillops/weekly/2026-W25.md", "violation_count": 0 }, + { + "path": "reports/skillops/weekly/2026-W27.json", + "violation_count": 0 + }, + { + "path": "reports/skillops/weekly/2026-W27.md", + "violation_count": 0 + }, { "path": "reports/system-model.json", "violation_count": 0 diff --git a/reports/world_class_claim_guard.md b/reports/world_class_claim_guard.md index 1a7c03c2..0e254e62 100644 --- a/reports/world_class_claim_guard.md +++ b/reports/world_class_claim_guard.md @@ -1,16 +1,16 @@ # World-Class Claim Guard -Generated at: `2026-06-21` +Generated at: `2026-07-02` ## Summary - decision: `claim-guard-pass-evidence-pending` - ledger ready to claim world-class: `false` - ledger pending evidence: `4` -- claim surfaces scanned: `184` -- JSON claim surfaces scanned: `91` -- metadata claim surfaces scanned: `92` -- package/runtime claim surfaces scanned: `17` +- claim surfaces scanned: `194` +- JSON claim surfaces scanned: `92` +- metadata claim surfaces scanned: `93` +- package/runtime claim surfaces scanned: `10` - violations: `0` - overclaim guard active: `true` diff --git a/reports/world_class_evidence_intake.json b/reports/world_class_evidence_intake.json index 9225577d..cf996b42 100644 --- a/reports/world_class_evidence_intake.json +++ b/reports/world_class_evidence_intake.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-20", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "schema_present": true, diff --git a/reports/world_class_evidence_intake.md b/reports/world_class_evidence_intake.md index 90b89f0a..ef76c4e2 100644 --- a/reports/world_class_evidence_intake.md +++ b/reports/world_class_evidence_intake.md @@ -1,6 +1,6 @@ # World-Class Evidence Intake -Generated at: `2026-06-20` +Generated at: `2026-07-02` ## Summary diff --git a/reports/world_class_evidence_ledger.json b/reports/world_class_evidence_ledger.json index c83a59df..c3de04ea 100644 --- a/reports/world_class_evidence_ledger.json +++ b/reports/world_class_evidence_ledger.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-20", + "generated_at": "2026-07-02", "skill_dir": ".", "decision": "evidence-pending", "ready_to_claim_world_class": false, diff --git a/reports/world_class_evidence_ledger.md b/reports/world_class_evidence_ledger.md index 636b3009..0dbc802f 100644 --- a/reports/world_class_evidence_ledger.md +++ b/reports/world_class_evidence_ledger.md @@ -1,6 +1,6 @@ # World-Class Evidence Ledger -Generated at: `2026-06-20` +Generated at: `2026-07-02` ## Summary diff --git a/reports/world_class_evidence_plan.json b/reports/world_class_evidence_plan.json index 370a8448..1219996c 100644 --- a/reports/world_class_evidence_plan.json +++ b/reports/world_class_evidence_plan.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-20", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "audit_decision": "continue-iteration", diff --git a/reports/world_class_evidence_plan.md b/reports/world_class_evidence_plan.md index 8bef1881..b2f12752 100644 --- a/reports/world_class_evidence_plan.md +++ b/reports/world_class_evidence_plan.md @@ -1,6 +1,6 @@ # World-Class Evidence Plan -Generated at: `2026-06-20` +Generated at: `2026-07-02` ## Summary diff --git a/reports/world_class_evidence_preflight.html b/reports/world_class_evidence_preflight.html index 3f03cb52..71a5afaa 100644 --- a/reports/world_class_evidence_preflight.html +++ b/reports/world_class_evidence_preflight.html @@ -92,7 +92,7 @@
Supporting evidence

supporting-evidence

- 31/31 ready + 27/28 ready

Supporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.

copy to artifact_refs: false
@@ -667,7 +667,7 @@
Supporting evidence

supporting-evidence

- 11/11 ready + 7/8 ready

Supporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.

copy to artifact_refs: false
diff --git a/reports/world_class_evidence_preflight.json b/reports/world_class_evidence_preflight.json index 7ae6ff45..31fc038a 100644 --- a/reports/world_class_evidence_preflight.json +++ b/reports/world_class_evidence_preflight.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-21", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "evidence_item_count": 4, @@ -802,8 +802,8 @@ "artifact_prefill_counts_as_evidence": false, "submission_ref_total_count": 2, "submission_ref_ready_count": 2, - "supporting_evidence_total_count": 11, - "supporting_evidence_ready_count": 11, + "supporting_evidence_total_count": 8, + "supporting_evidence_ready_count": 7, "roles": [ { "role": "submission-ref", @@ -1946,8 +1946,8 @@ "artifact_prefill_counts_as_evidence": false, "submission_ref_total_count": 7, "submission_ref_ready_count": 7, - "supporting_evidence_total_count": 31, - "supporting_evidence_ready_count": 31, + "supporting_evidence_total_count": 28, + "supporting_evidence_ready_count": 27, "roles": [ { "role": "submission-ref", diff --git a/reports/world_class_evidence_preflight.md b/reports/world_class_evidence_preflight.md index 67bc6ad0..3fa9cf17 100644 --- a/reports/world_class_evidence_preflight.md +++ b/reports/world_class_evidence_preflight.md @@ -1,6 +1,6 @@ # World-Class Evidence Preflight -Generated at: `2026-06-21` +Generated at: `2026-07-02` ## Summary @@ -33,14 +33,14 @@ This preflight report checks whether an operator can start collecting the remain - drafts count as evidence: `false` - artifact prefill counts as evidence: `false` - submission refs ready: `7` / `7` -- supporting evidence ready: `31` / `31` +- supporting evidence ready: `27` / `28` Generate the submission kit after the real provider, human, native-permission, or native-client work exists. The generated JSON drafts remain `template_only: true` until an operator edits them with real aggregate artifact references and matching SHA-256 digests. The prefill command only inserts local artifact SHA-256 digests; it does not make a draft count as evidence. | Role | Copy to artifact_refs | Ready | Meaning | | --- | --- | --- | --- | | `submission-ref` | `true` | `7 / 7` | Rows marked submission-ref are the aggregate paths expected in artifact_refs. | -| `supporting-evidence` | `false` | `31 / 31` | Supporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs. | +| `supporting-evidence` | `false` | `27 / 28` | Supporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs. | `submission-ref` rows are the only checklist rows expected in `artifact_refs`; `supporting-evidence` rows stay available for audit context and reviewer traceability. @@ -149,7 +149,7 @@ Repair rows convert preflight and source blockers into a prioritized operator qu - prepare draft: `python3 scripts/yao.py world-class-submission-kit . --evidence-key native-permission-enforcement --output-dir evidence/world_class/submissions` - prepare draft with artifact SHA prefill: `python3 scripts/yao.py world-class-submission-kit . --evidence-key native-permission-enforcement --output-dir evidence/world_class/submissions --prefill-artifacts` - submission refs ready: `2` / `2` -- supporting evidence ready: `11` / `11` +- supporting evidence ready: `7` / `8` ### Prechecks diff --git a/reports/world_class_operator_runbook.json b/reports/world_class_operator_runbook.json index ef09b15a..2e7da174 100644 --- a/reports/world_class_operator_runbook.json +++ b/reports/world_class_operator_runbook.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-20", + "generated_at": "2026-07-02", "skill_dir": ".", "summary": { "evidence_item_count": 4, diff --git a/reports/world_class_operator_runbook.md b/reports/world_class_operator_runbook.md index c2dd03a1..a5270e18 100644 --- a/reports/world_class_operator_runbook.md +++ b/reports/world_class_operator_runbook.md @@ -1,6 +1,6 @@ # World-Class Operator Runbook -Generated at: `2026-06-20` +Generated at: `2026-07-02` ## Summary diff --git a/reports/world_class_submission_review.json b/reports/world_class_submission_review.json index f7a6afb8..a4b5ae82 100644 --- a/reports/world_class_submission_review.json +++ b/reports/world_class_submission_review.json @@ -1,7 +1,7 @@ { "schema_version": "1.0", "ok": true, - "generated_at": "2026-06-20", + "generated_at": "2026-07-02", "skill_dir": ".", "decision": "awaiting-submissions", "ready_to_claim_world_class": false, diff --git a/reports/world_class_submission_review.md b/reports/world_class_submission_review.md index fa3259e8..c1a708f7 100644 --- a/reports/world_class_submission_review.md +++ b/reports/world_class_submission_review.md @@ -1,6 +1,6 @@ # World-Class Submission Review -Generated at: `2026-06-20` +Generated at: `2026-07-02` ## Summary diff --git a/skill-ir/examples/yao-meta-skill.json b/skill-ir/examples/yao-meta-skill.json index d9361cc3..70f51367 100644 --- a/skill-ir/examples/yao-meta-skill.json +++ b/skill-ir/examples/yao-meta-skill.json @@ -4,7 +4,7 @@ "title": "Yao Meta Skill", "job_to_be_done": "Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse.", "trigger_surface": { - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "should_trigger": [ "Create a skill from this repeated workflow.", "Turn this runbook into a reusable agent skill.", diff --git a/skill_atlas/catalog.json b/skill_atlas/catalog.json index 9cfe5503..a6731514 100644 --- a/skill_atlas/catalog.json +++ b/skill_atlas/catalog.json @@ -5,7 +5,7 @@ { "name": "yao-meta-skill", "path": ".", - "description": "Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.", + "description": "Create/improve/evaluate agent skills from workflows, prompts, SOPs, scripts. Use for migration/release/package, routing, evals/tests, install checks, 优化已有 skill, 补 trigger 评测. Exclude no-skill summary/translation/docs.", "owner": "Yao Team", "version": "1.1.0", "status": "active", @@ -218,22 +218,28 @@ "templates/complex_skill.md.j2" ], "token_set": [ - "creation", - "distribution", + "checks", "docs", "evals", "evaluate", - "improvement", - "notes", - "or", + "exclude", + "improve", + "install", + "migration", + "no-skill", "package", - "packaging", "prompts", - "refactor", - "team-ready", - "transcripts", - "workflow", - "workflows" + "release", + "routing", + "scripts", + "sops", + "summary", + "tests", + "translation", + "trigger", + "workflows", + "优化已有", + "评测" ], "atlas_scope": "release", "actionable": true, diff --git a/skill_atlas/route_overlap_matrix.csv b/skill_atlas/route_overlap_matrix.csv index f4bc7521..577f808e 100644 --- a/skill_atlas/route_overlap_matrix.csv +++ b/skill_atlas/route_overlap_matrix.csv @@ -1,11 +1,11 @@ skill_a,skill_b,path_a,path_b,score,status,actionable,scope_a,scope_b -yao-meta-skill,release-orchestrator,.,examples/complex-release-orchestrator/generated-skill,0.064,clear,False,release,example -yao-meta-skill,frontend-review,.,examples/evolution-frontend-review/final,0.028,clear,False,release,example +yao-meta-skill,release-orchestrator,.,examples/complex-release-orchestrator/generated-skill,0.037,clear,False,release,example +yao-meta-skill,frontend-review,.,examples/evolution-frontend-review/final,0.0,clear,False,release,example yao-meta-skill,frontend-review,.,examples/evolution-frontend-review/v0,0.0,clear,False,release,example -yao-meta-skill,frontend-review,.,examples/evolution-frontend-review/v1,0.029,clear,False,release,example -yao-meta-skill,incident-command-governor,.,examples/governed-incident-command/generated-skill,0.034,clear,False,release,example -yao-meta-skill,note-cleanup,.,examples/simple-note-cleanup/generated-skill,0.061,clear,False,release,example -yao-meta-skill,frontend-review,.,examples/team-frontend-review/generated-skill,0.029,clear,False,release,example +yao-meta-skill,frontend-review,.,examples/evolution-frontend-review/v1,0.0,clear,False,release,example +yao-meta-skill,incident-command-governor,.,examples/governed-incident-command/generated-skill,0.0,clear,False,release,example +yao-meta-skill,note-cleanup,.,examples/simple-note-cleanup/generated-skill,0.0,clear,False,release,example +yao-meta-skill,frontend-review,.,examples/team-frontend-review/generated-skill,0.0,clear,False,release,example yao-meta-skill,invalid-governance-skill,.,tests/fixtures/governance_invalid_manifest,0.0,clear,False,release,test-fixture yao-meta-skill,broken-yaml-skill,.,tests/fixtures/package_invalid_yaml,0.0,clear,False,release,test-fixture yao-meta-skill,broken-skill,.,tests/fixtures/package_missing_interface_field,0.0,clear,False,release,test-fixture
路径Path作用Role类型Type
SKILL.mdSkill 入口文件Skill entrypoint文件file
README.md人类可读使用说明Human-readable usage guide文件file
agents/interface.yaml跨平台接口元数据Neutral interface metadata文件file
manifest.json生命周期与打包元数据Lifecycle and portability metadata文件file
references扩展指导与复用资料Extended guidance and reusable notes目录folder
scripts确定性脚本或本地工具Deterministic helpers or local tooling目录folder
evals触发与质量检查Trigger and quality checks目录folder
reports生成的证据与总结报告Generated evidence and overview artifacts目录folder