Review Studio 2.0

Yao Meta Skill

Create, refactor, evaluate, and package agent skills from workflows, prompts, transcripts, docs, or notes. Use for skill creation, reusable workflow packaging, skill improvement, evals, and team-ready distribution.

审查结论 review Score 91/100

核心指标

Skill IR2.0.0

5 targets in platform-neutral contract

Compiler5/5

target contracts compiled from Skill IR

Output Delta100.0

5 cases; 1 file-backed

Exec Runs10

command 10; model 10; recorded 0

Blind A/B5

review pairs hide baseline vs with-skill labels

Review Kit0/5

pending 5; answer key hidden

Review A/B0/5

adjudication decisions; pending 5

Public Claimblocked

3 blockers; local reproducible true

Blueprint21/21

2.0 coverage; extensions partial 0, planned 0; evidence pending 4

Runtime5/5

target conformance pass rate

Perm Probe4/4

0 native; 4 installer-enforced

Trust0

155 scripts scanned; secrets found

Py Compat0

234 files scanned for Python 3.11

Arch Debt0

719 largest lines; 0 watchlist; 6 early; 71 CLI handlers; 18 in entrypoint

Atlas5

11 scanned skills; route collisions

Driftlow

1 metadata events; 0 missed triggers

Daily Ops5

proposal-review; approval 0; release lock true

Weekly Queue5

curator-review; ready 1; top score 88

Waivers0

0 gates covered; human risk decisions

Intake4/4

0 valid submissions; 0 invalid

Claim Guard0

184 public surfaces scanned

Notes0/0

0 open blocker annotations

Registry1.1.0

5 targets; MIT license

Archivepass

580 zip entries; package verification

Installpass

4 adapters; 12 permissions enforced; 0 permission failures

Upgrademinor

declared minor; 0 breaking changes

审查闸门

通过

意图画布

intent confidence 100/100; Intent is clear enough to package the first routeable version.

通过

触发实验

13 trigger cases; 0 misroutes; 0 ambiguous

关注

输出实验

5/5 cases; with-skill 100.0; baseline 0.0; file-backed 1; near-neighbor 1; blind A/B 5; exec 10; command 10; model 10; recorded 0; reviewed 0/5; review pending 5

通过

上下文

initial load 998/1000; deferred 530871/120000; top deferred scripts 469568; resource governance governed; quality density 140.3

通过

运行矩阵

5 / 5 targets pass

通过

信任报告

0 secrets; 155 scripts; 3 network-capable scripts; 0 help smoke failures

通过

Python 兼容

Python 3.11; 234 files; 0 compatibility issues; 0 syntax; 0 f-string 3.11 hazards

通过

架构维护

232 Python files; 0 hotspots; 0 watchlist files; 6 early watch files; 0 blockers; largest 719 lines; 71 CLI handlers; 18 in entrypoint

通过

权限批准

3/3 permissions approved; gaps 0; required file_write, network, subprocess

通过

权限探针

4/4 targets probed; native 0; metadata fallback 4; installer 4; residual risks 4

通过

组合治理

11 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 0 actionable drift; 22 scoped non-actionable issues

通过

运营回路

1 metadata events; adoption 100.0; missed 0; bad-output 0; risk low; daily proposals 5; daily decision proposal-review; daily release lock true; weekly queue 5 unique; weekly ready 1; weekly top 88; weekly release lock true

关注

人工批准

0 active waivers; 1 warning gates still need reviewer decision

关注

世界证据

4 pending world-class evidence entries; 1 human pending; 3 external pending; source checks 12/19 pass; 7 blocked; overclaim guard true

通过

注册审计

yao-meta-skill 1.1.0; 6/6 compatibility entries pass; install pass with 4 adapters; installer permissions 12 enforced / 0 failures

通过

发布路线

0 promote; 3 keep current; 0 blocked; upgrade minor declared / minor recommended

闸门契约

Ok: 是
期望 Gate: 16
实际 Gate: 16
缺失 Gate: 无
未知 Gate: 无
重复 Gate: 无
未加权 Gate: 无

阻断事项

无。

关注事项

输出实验5/5 cases; with-skill 100.0; baseline 0.0; file-backed 1; near-neighbor 1; blind A/B 5; exec 10; command 10; model 10; recorded 0; reviewed 0/5; review pending 5
人工批准0 active waivers; 1 warning gates still need reviewer decision
世界证据4 pending world-class evidence entries; 1 human pending; 3 external pending; source checks 12/19 pass; 7 blocked; overclaim guard true

修复动作

关注

输出实验

补足 output eval 覆盖、execution evidence、blind A/B 和 reviewer adjudication。同步补足盲审声明。

没有输出质量和人工盲评证据时，Skill 只能证明会触发，不能证明输出真的更好且经得起审查。

修复位置: evals/output/cases.jsonl + reports/output_quality_scorecard.md + reports/output_review_kit.html + reports/output_review_adjudication.md + reports/output_review_decisions.json
验证命令: python3 scripts/adjudicate_output_review.py --write-template && python3 scripts/yao.py output-review

evals/output/cases.jsonl:1output eval cases · eval · pattern: "id"
{"id":"skill-package-contract","prompt":"Turn this repeated workflow into a reusable team skill package.","baseline_output":"I can write a prompt for that workflow and include a s…
reports/output_quality_scorecard.md:1output scorecard · report · pattern: # Output
# Output Quality Scorecard
reports/output_execution_runs.md:1output execution runs · report · pattern: # Output Execution
# Output Execution Runs
reports/output_blind_review_pack.md:1blind A/B review pack · report · pattern: # Output Blind
# Output Blind A/B Review Pack
reports/output_review_kit.html:6reviewer cockpit · report · pattern: Output Review Kit
<title>Output Review Kit</title>
reports/output_review_adjudication.md:1review adjudication · report · pattern: # Output Review
# Output Review Adjudication

关注

人工批准

对保留的 warning 写入 reviewer、理由、范围和到期时间，或修掉 warning。

warning 可以被接受，但必须可审计、会过期，并且不能掩盖 blocker。

修复位置: reports/review_waivers.md
验证命令: python3 scripts/render_review_waivers.py .

reports/review_waivers.md:1waiver ledger · report · pattern: # Review
# Review Waivers
references/review-waiver-method.md:1waiver method · method · pattern: # Review
# Review Waiver Method

关注

世界证据

补齐 provider、真人盲评、原生权限执行和真实客户端遥测证据，或明确本次发布不声明 world-class 完成。

世界级结论必须来自已接受的外部/人工证据；计划、metadata fallback、待评审和本地命令都不能替代完成证据。

修复位置: reports/world_class_operator_runbook.html + reports/world_class_evidence_ledger.md + reports/world_class_evidence_intake.md + reports/world_class_submission_review.md
验证命令: python3 scripts/yao.py world-class-runbook . --submissions-dir evidence/world_class/submissions && python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions && python3 scripts/yao.py review-studio .

证据采集

以下条目仍需真实外部或人工证据；提交文件、校验命令和阻断检查必须同时闭环。

pending · external

Provider Holdout

model-executed 10; token-observed 10

提交: evidence/world_class/submissions/provider-holdout.json
模板: evidence/world_class/templates/provider-holdout.intake.json
阻断: 0 blocked / 3 pass
修复: 1 repair rows; counts as completion: false
阶段: 1 blocked phases; counts as completion: false
下一步: Run provider-backed holdout cases with real credentials and commit only aggregate evidence.

阻断检查

暂无阻断检查。

阶段队列

#20 · unblock-accessUnblock access1 / 1 blockedowners: operator with provider credentialsevidence: provider-holdoutSet one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions

修复清单

#20 · unblock-access · precheckprovider-api-keyoperator with provider credentialsRequired env_any precheck is missing.Set one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions

操作命令

准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key provider-holdout --output-dir evidence/world_class/submissions
校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions

首要步骤

Set one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.
For OpenAI Responses: python3 scripts/yao.py output-exec --provider-runner openai --provider-model ${YAO_OUTPUT_EVAL_MODEL:-gpt-4.1-mini} --timeout-seconds 60
For DeepSeek Chat Completions: python3 scripts/yao.py output-exec --provider-runner deepseek --provider-model deepseek-v4-flash --provider-api-format chat-completions --provider-thinking disabled --api-key-env DEEPSEEK_API_KEY --timeout-seconds 120

采集契约

来源要求

provider-backed model run
observed timing
observed token metadata

通过条件

reports/output_execution_runs.json summary.model_executed_count > 0
reports/output_execution_runs.json summary.timing_observed_count > 0
reports/output_execution_runs.json summary.token_observed_count > 0
reports/skill_os2_audit.json item provider-holdout status becomes pass

证据资产

reports/output_execution_runs.json
reports/output_execution_runs.md
reports/skill_os2_audit.json
evidence/world_class/intake.schema.json
evidence/world_class/templates/provider-holdout.intake.json

资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

submission-ref1 / 1 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
supporting-evidence6 / 6 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.

隐私边界

Do not commit provider credentials or environment dumps.
The output execution report records output hashes and aggregate run metadata, not raw provider prompts.

pending · human

Human Adjudication

0/5 decisions; pending 5

提交: evidence/world_class/submissions/human-adjudication.json
模板: evidence/world_class/templates/human-adjudication.intake.json
阻断: 5 blocked / 5 pass
修复: 6 repair rows; counts as completion: false
阶段: 2 blocked phases; counts as completion: false
下一步: Record real A/B choices, reviewer metadata, and blind-review attestation, then regenerate adjudication.

阻断检查

No pending decisionspending_count: 5 / ==0Record a reviewer choice and reason for every pair.
Judgments completejudgment_count: 0 / ==pair_countEvery pair needs one valid human judgment.
Reviewer metadatareviewer_metadata_present: False / trueRecord reviewer and reviewed_at before adjudication can count.
Blind review attestedblind_review_attested: False / trueSet reviewer_attestation only after choices are completed before opening the answer key.
Human evidence readyready_for_human_evidence: False / trueComplete all reviewer decisions with metadata and rationale, plus blind-review attestation and integrity fingerprints.

阶段队列

#20 · unblock-accessUnblock access1 / 1 blockedowners: human reviewerevidence: human-adjudicationAssign a real reviewer identity before claiming human adjudication.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
#40 · collect-sourceCollect source5 / 5 blockedowners: human reviewerevidence: human-adjudicationSet reviewer_attestation only after choices are completed before opening the answer key.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions

修复清单

#20 · unblock-access · precheckhuman-reviewerhuman reviewerRequired human precheck is human-required.Assign a real reviewer identity before claiming human adjudication.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
#40 · collect-source · source-checkblind_review_attestedhuman reviewerCurrent value False does not satisfy 'true'.Set reviewer_attestation only after choices are completed before opening the answer key.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
#40 · collect-source · source-checkjudgment_counthuman reviewerCurrent value 0 does not satisfy '==pair_count'.Every pair needs one valid human judgment.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
#40 · collect-source · source-checkpending_counthuman reviewerCurrent value 5 does not satisfy '==0'.Record a reviewer choice and reason for every pair.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
#40 · collect-source · source-checkready_for_human_evidencehuman reviewerCurrent value False does not satisfy 'true'.Complete all reviewer decisions with metadata and rationale, plus blind-review attestation and integrity fingerprints.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
#40 · collect-source · source-checkreviewer_metadata_presenthuman reviewerCurrent value False does not satisfy 'true'.Record reviewer and reviewed_at before adjudication can count.python3 scripts/yao.py output-review && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions

操作命令

准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key human-adjudication --output-dir evidence/world_class/submissions
校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions

首要步骤

python3 scripts/yao.py output-review-kit --write-template
Open reports/output_review_kit.md and choose A or B for each pair without opening the answer key.
python3 scripts/adjudicate_output_review.py --write-template

采集契约

来源要求

real reviewer identity
blind A/B decisions
answer key unopened until decisions exist

通过条件

reports/output_review_adjudication.json summary.pending_count == 0
reports/output_review_adjudication.json summary.judgment_count == summary.pair_count
reports/output_review_adjudication.json summary.invalid_decision_count == 0
reports/output_review_adjudication.json summary.reviewer_metadata_present is true
reports/output_review_adjudication.json summary.blind_review_attested is true
reports/output_review_adjudication.json review_integrity.blind_pack_sha256 exists and matches reports/output_review_decisions.json
reports/output_review_adjudication.json pairs and reviewer_checklist store prompt_sha256, not raw prompt text
reports/output_review_adjudication.json summary.ready_for_human_evidence is true
reports/skill_os2_audit.json item human-adjudication status becomes pass

证据资产

reports/output_blind_review_pack.md
reports/output_review_kit.md
reports/output_review_decisions.json
reports/output_review_adjudication.json
reports/output_review_adjudication.md

资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
supporting-evidence8 / 8 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.

隐私边界

Reviewer decisions should not include raw user data or private customer detail.
Reviewer reasons must be rubric-based and must not include raw user data or private customer detail.
The decision importer rejects raw prompt, output, transcript, message, and answer-key fields.
The adjudication evidence stores prompt_sha256 instead of raw prompt text.

pending · external

Native Permission Enforcement

native-enforced targets 0; installer-enforced targets 4

提交: evidence/world_class/submissions/native-permission-enforcement.json
模板: evidence/world_class/templates/native-permission-enforcement.intake.json
阻断: 1 blocked / 2 pass
修复: 2 repair rows; counts as completion: false
阶段: 2 blocked phases; counts as completion: false
下一步: Integrate a real target-client or external installer runtime guard before claiming native permission enforcement.

阻断检查

Native enforcementnative_enforcement_count: 0 / >0Collect real target-client or external runtime guard proof.

阶段队列

#20 · unblock-accessUnblock access1 / 1 blockedowners: target client or installer integratorevidence: native-permission-enforcementAttach a real target-client or external installer runtime guard; metadata fallback is not enough.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
#40 · collect-sourceCollect source1 / 1 blockedowners: target client or installer integratorevidence: native-permission-enforcementCollect real target-client or external runtime guard proof.python3 scripts/yao.py runtime-permissions . --package-dir dist && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions

修复清单

#20 · unblock-access · prechecknative-guardtarget client or installer integratorRequired external precheck is external-required.Attach a real target-client or external installer runtime guard; metadata fallback is not enough.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
#40 · collect-source · source-checknative_enforcement_counttarget client or installer integratorCurrent value 0 does not satisfy '>0'.Collect real target-client or external runtime guard proof.python3 scripts/yao.py runtime-permissions . --package-dir dist && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions

操作命令

准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key native-permission-enforcement --output-dir evidence/world_class/submissions
校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions

首要步骤

Implement or connect a real target client or external installer runtime guard that blocks undeclared network, file_write, or subprocess capabilities.
Update the generated target adapter only when the guard is actually enforced by that target.
python3 scripts/yao.py package . --platform openai --platform claude --platform generic --platform vscode --output-dir dist --zip

采集契约

来源要求

real target client or external installer runtime guard
native enforcement flag or externally accepted guard proof
residual risk retained for fallback targets

通过条件

reports/runtime_permission_probes.json summary.native_enforcement_count > 0
reports/runtime_permission_probes.json summary.failure_count == 0
reports/runtime_permission_probes.json summary.installer_enforcement_pass_count records local installer enforcement but does not replace native evidence
reports/skill_os2_audit.json item native-permission-enforcement status becomes pass

证据资产

dist/targets/*/adapter.json
reports/runtime_permission_probes.json
reports/runtime_permission_probes.md
reports/install_simulation.json
reports/install_simulation.md

资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
supporting-evidence11 / 11 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.

隐私边界

Do not mark native_enforcement true for metadata-only fallbacks.
Keep residual risks visible for targets that still rely on operator enforcement.

pending · external

Native Client Telemetry

external source events 0; adoption samples 1

提交: evidence/world_class/submissions/native-client-telemetry.json
模板: evidence/world_class/templates/native-client-telemetry.intake.json
阻断: 1 blocked / 2 pass
修复: 2 repair rows; counts as completion: false
阶段: 2 blocked phases; counts as completion: false
下一步: Install a real client against the native host and import production metadata-only events.

阻断检查

External eventsexternal_source_events: 0 / >0Import at least one metadata-only event from a real client.

阶段队列

#20 · unblock-accessUnblock access1 / 1 blockedowners: Browser/Chrome/IDE/provider client integratorevidence: native-client-telemetryInstall a real Browser, Chrome, IDE, or provider client that emits metadata-only events.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
#40 · collect-sourceCollect source1 / 1 blockedowners: Browser/Chrome/IDE/provider client integratorevidence: native-client-telemetryImport at least one metadata-only event from a real client.python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions

修复清单

#20 · unblock-access · precheckexternal-clientBrowser/Chrome/IDE/provider client integratorRequired external precheck is external-required.Install a real Browser, Chrome, IDE, or provider client that emits metadata-only events.python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions
#40 · collect-source · source-checkexternal_source_eventsBrowser/Chrome/IDE/provider client integratorCurrent value 0 does not satisfy '>0'.Import at least one metadata-only event from a real client.python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl && python3 scripts/yao.py world-class-preflight . --submissions-dir evidence/world_class/submissions

操作命令

准备提交python3 scripts/yao.py world-class-submission-kit . --evidence-key native-client-telemetry --output-dir evidence/world_class/submissions
校验入口python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
审查提交python3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
刷新台账python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions

首要步骤

python3 scripts/telemetry_native_host.py . --write-launcher /tmp/yao-telemetry-host.sh --write-manifest /tmp/yao-telemetry-host.json --allowed-origin chrome-extension://<extension-id>/
Install the generated native messaging manifest for the real client and send at least one accepted skill_activation or skill_output event.
python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl

采集契约

来源要求

real external client source
metadata-only event
local-first import path

通过条件

reports/adoption_drift_report.json summary.source_types.external > 0
reports/adoption_drift_report.json summary.adoption_sample_count > 0
reports/skill_os2_audit.json item native-client-telemetry status becomes pass

证据资产

reports/adoption_drift_report.json
reports/adoption_drift_report.md
reports/telemetry_hook_recipes.json
scripts/telemetry_native_host.py
evidence/world_class/intake.schema.json

资产角色

source: world-class-submission-kit; counts as evidence: false; prefill counts as evidence: false

submission-ref2 / 2 readyartifact_refs: trueRows marked submission-ref are the aggregate paths expected in artifact_refs.
supporting-evidence6 / 6 readyartifact_refs: falseSupporting-evidence rows help reviewers audit the packet but do not all need to be copied into artifact_refs.

隐私边界

Telemetry must remain metadata-only and local-first.
Do not package reports/telemetry_events.jsonl or any raw prompt, output, transcript, note, or message field.

reports/world_class_evidence_ledger.md:1world-class evidence ledger · report · pattern: # World-Class Evidence Ledger
# World-Class Evidence Ledger
reports/world_class_evidence_plan.md:1world-class evidence plan · report · pattern: # World-Class Evidence Plan
# World-Class Evidence Plan
reports/world_class_evidence_intake.md:1world-class evidence intake · report · pattern: # World-Class Evidence Intake
# World-Class Evidence Intake
reports/world_class_evidence_preflight.md:1world-class evidence preflight · report · pattern: # World-Class Evidence Preflight
# World-Class Evidence Preflight
reports/world_class_evidence_preflight.html:6world-class preflight HTML · report · pattern: World-Class Evidence Preflight
<title>World-Class Evidence Preflight</title>
reports/world_class_submission_review.md:1world-class submission review · report · pattern: # World-Class Submission Review
# World-Class Submission Review
reports/world_class_claim_guard.md:1world-class claim guard · report · pattern: # World-Class Claim Guard
# World-Class Claim Guard
evidence/world_class/intake.schema.json:4evidence intake schema · schema · pattern: Yao World-Class Evidence Intake
"title": "Yao World-Class Evidence Intake",
evidence/world_class/templates/provider-holdout.intake.json:3provider intake template · template · pattern: provider-holdout
"evidence_key": "provider-holdout",
evidence/world_class/templates/human-adjudication.intake.json:3human intake template · template · pattern: human-adjudication
"evidence_key": "human-adjudication",
evidence/world_class/templates/native-permission-enforcement.intake.json:3permission intake template · template · pattern: native-permission-enforcement
"evidence_key": "native-permission-enforcement",
evidence/world_class/templates/native-client-telemetry.intake.json:3telemetry intake template · template · pattern: native-client-telemetry
"evidence_key": "native-client-telemetry",
reports/skill_os2_audit.md:1Skill OS 2.0 audit · report · pattern: # Skill OS
# Skill OS 2.0 Audit
reports/output_review_decisions.json:30human review decisions · report · pattern: winner_variant
"winner_variant": "Use A or B after reading the blind review pack. Leave blank when pending.",
reports/runtime_permission_probes.md:1runtime permission probes · report · pattern: # Runtime
# Runtime Permission Probes
reports/adoption_drift_report.md:1adoption drift · report · pattern: # Adoption
# Adoption And Drift Report

审查批注

当前批注：0 条批注；开放 0；阻断 0

当前没有 reviewer 批注。

意图画布

Turn repeated workflows, prompts, transcripts, runbooks, documents, or existing skill packages into routeable, evaluable, packageable, and governable agent skills for personal, team, library, or governed reuse. Primary output: A working skill package with lean SKILL.md, aligned agents/interface.yaml, justified references, scripts only when useful, eval evidence, reports, packaging metadata, and clear next iteration recommendations.. Exclusions: one-off writing, translation, explanation, or brainstorming requests that do not need a reusable skill, general code review or debugging unless the user is packaging that workflow as a skill, raw private material that was not intentionally supplied as skill evidence, platform-specific plugin creation when the request is not about an agent skill package.

证据路径

skill_overviewreports/skill-overview.html
review_viewerreports/review-viewer.html
output_evalreports/output_quality_scorecard.md
output_executionreports/output_execution_runs.md
output_blind_reviewreports/output_blind_review_pack.md
output_review_kitreports/output_review_kit.md
output_review_kit_htmlreports/output_review_kit.html
output_review_decisionsreports/output_review_decisions.json
output_review_adjudicationreports/output_review_adjudication.md
benchmark_reproducibilityreports/benchmark_reproducibility.md
skill_os2_coveragereports/skill_os2_coverage.md
runtime_conformancereports/conformance_matrix.md
trust_reportreports/security_trust_report.md
python_compatibilityreports/python_compatibility.md
architecture_maintainabilityreports/architecture_maintainability.md
permission_policysecurity/permission_policy.md
runtime_permissionsreports/runtime_permission_probes.md
skill_atlasreports/skill_atlas.html
compiled_targetsreports/compiled_targets.md
adoption_driftreports/adoption_drift_report.md
review_waiversreports/review_waivers.md
review_annotationsreports/review_annotations.md
adaptation_proposalsreports/adaptation_proposals.md
adaptation_approval_ledgerreports/adaptation_approval_ledger.json
adaptation_regressionreports/adaptation_regression_report.md
world_class_evidence_planreports/world_class_evidence_plan.md
world_class_evidence_ledgerreports/world_class_evidence_ledger.md
world_class_evidence_intakereports/world_class_evidence_intake.md
world_class_evidence_preflightreports/world_class_evidence_preflight.md
world_class_evidence_preflight_htmlreports/world_class_evidence_preflight.html
world_class_submission_reviewreports/world_class_submission_review.md
world_class_operator_runbookreports/world_class_operator_runbook.md
world_class_operator_runbook_htmlreports/world_class_operator_runbook.html
world_class_claim_guardreports/world_class_claim_guard.md
registry_auditreports/registry_audit.md
package_verificationreports/package_verification.md
install_simulationreports/install_simulation.md
upgrade_checkreports/upgrade_check.md
migrationdocs/migration-v2.md
skill_irskill-ir/examples/yao-meta-skill.json
daily_skillopsreports/skillops/daily/2026-06-21.md
weekly_curatorreports/skillops/weekly/2026-W25.md

触发实验

13 trigger cases; 0 misroutes; 0 ambiguous

组合治理

Skill 数: 11
纳入审查: 1
待处理冲突: 0
待处理 owner: 0
待处理过期: 0
Telemetry Report Count: 1
Actionable Drift Signal Count: 0
Drift Signal Count: 0
非行动项: 22

输出实验

案例数: 5
With Skill: 100
Baseline: 0
增益: 100
Gate: 是

执行证据

运行数: 10
命令执行: 10
模型执行: 10
记录样本: 0
计时样本: 10
估算 Token: 0

盲评包

Pair Count: 5
Answer Key Separate: 是
With Skill Hidden Count: 5

审定报告

Pair Count: 5
Judgment Count: 0
待审: 5
Agreement Count: 0
Disagreement Count: 0
Invalid Decision Count: 0
答案揭示: 0
答案隐藏: 5
Reviewer Checklist Count: 5
Reviewer Checklist Pending Count: 5

评审清单

先打开 reports/output_review_kit.md；每张卡片对应一个 blind A/B case，说明当前是否待判、答案是否仍隐藏、应填写的决策文件、有效字段和复跑命令。

awaiting-decision · 答案隐藏

skill-package-contract

Reviewer has not selected A or B yet; answer key remains hidden.

盲评包: reports/output_blind_review_pack.json
决策表: reports/output_review_decisions.json
提示哈希: 95405d805bd516d0998b69d31ece2c1a465b0aa99f04d603628a3939028133b5

操作命令

prepare_review_kitpython3 scripts/yao.py output-review-kit
write_templatepython3 scripts/adjudicate_output_review.py --write-template
import_decisionspython3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication
adjudicatepython3 scripts/yao.py output-review
refresh_review_studiopython3 scripts/yao.py review-studio .

字段要求

winner_variantA or B after reading only the blind review pack.
confidenceOptional number from 0 to 1.
reasonRequired rationale; do not reveal baseline or with-skill labels before adjudication.
reviewerHuman reviewer name or review group at the decision-file top level.
reviewed_atReview date or timestamp at the decision-file top level.
reviewer_attestation.blind_review_completed_before_answer_keyTrue only after the reviewer has completed choices before opening the answer key.
reviewer_attestation.answer_key_not_opened_before_decisionsTrue only when the answer key was not opened before decisions were recorded.

隐私边界

Do not paste raw private user data into the decision reason.
Do not open the answer key before reviewer choices are recorded.
Leave winner_variant blank when the reviewer is not ready to decide.

awaiting-decision · 答案隐藏

output-eval-expectation

Reviewer has not selected A or B yet; answer key remains hidden.

盲评包: reports/output_blind_review_pack.json
决策表: reports/output_review_decisions.json
提示哈希: 3829e97814ca269ad8e400ec984c56ddcd54b79448d3846c15e7f5ff44c2edc4

操作命令

prepare_review_kitpython3 scripts/yao.py output-review-kit
write_templatepython3 scripts/adjudicate_output_review.py --write-template
import_decisionspython3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication
adjudicatepython3 scripts/yao.py output-review
refresh_review_studiopython3 scripts/yao.py review-studio .

字段要求

winner_variantA or B after reading only the blind review pack.
confidenceOptional number from 0 to 1.
reasonRequired rationale; do not reveal baseline or with-skill labels before adjudication.
reviewerHuman reviewer name or review group at the decision-file top level.
reviewed_atReview date or timestamp at the decision-file top level.
reviewer_attestation.blind_review_completed_before_answer_keyTrue only after the reviewer has completed choices before opening the answer key.
reviewer_attestation.answer_key_not_opened_before_decisionsTrue only when the answer key was not opened before decisions were recorded.

隐私边界

Do not paste raw private user data into the decision reason.
Do not open the answer key before reviewer choices are recorded.
Leave winner_variant blank when the reviewer is not ready to decide.

awaiting-decision · 答案隐藏

ir-before-packaging

Reviewer has not selected A or B yet; answer key remains hidden.

盲评包: reports/output_blind_review_pack.json
决策表: reports/output_review_decisions.json
提示哈希: d00333fa41f2e28dc7505f25f259aefb82e65b2df1abda2822686c9a6f07bb80

操作命令

prepare_review_kitpython3 scripts/yao.py output-review-kit
write_templatepython3 scripts/adjudicate_output_review.py --write-template
import_decisionspython3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication
adjudicatepython3 scripts/yao.py output-review
refresh_review_studiopython3 scripts/yao.py review-studio .

字段要求

winner_variantA or B after reading only the blind review pack.
confidenceOptional number from 0 to 1.
reasonRequired rationale; do not reveal baseline or with-skill labels before adjudication.
reviewerHuman reviewer name or review group at the decision-file top level.
reviewed_atReview date or timestamp at the decision-file top level.
reviewer_attestation.blind_review_completed_before_answer_keyTrue only after the reviewer has completed choices before opening the answer key.
reviewer_attestation.answer_key_not_opened_before_decisionsTrue only when the answer key was not opened before decisions were recorded.

隐私边界

Do not paste raw private user data into the decision reason.
Do not open the answer key before reviewer choices are recorded.
Leave winner_variant blank when the reviewer is not ready to decide.

awaiting-decision · 答案隐藏

near-neighbor-boundary

Reviewer has not selected A or B yet; answer key remains hidden.

盲评包: reports/output_blind_review_pack.json
决策表: reports/output_review_decisions.json
提示哈希: 2d0585e9fa6d03e28786f5e8f0c090781eb209afd344d25e2f91203c49a656d0

操作命令

prepare_review_kitpython3 scripts/yao.py output-review-kit
write_templatepython3 scripts/adjudicate_output_review.py --write-template
import_decisionspython3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication
adjudicatepython3 scripts/yao.py output-review
refresh_review_studiopython3 scripts/yao.py review-studio .

字段要求

winner_variantA or B after reading only the blind review pack.
confidenceOptional number from 0 to 1.
reasonRequired rationale; do not reveal baseline or with-skill labels before adjudication.
reviewerHuman reviewer name or review group at the decision-file top level.
reviewed_atReview date or timestamp at the decision-file top level.
reviewer_attestation.blind_review_completed_before_answer_keyTrue only after the reviewer has completed choices before opening the answer key.
reviewer_attestation.answer_key_not_opened_before_decisionsTrue only when the answer key was not opened before decisions were recorded.

隐私边界

Do not paste raw private user data into the decision reason.
Do not open the answer key before reviewer choices are recorded.
Leave winner_variant blank when the reviewer is not ready to decide.

awaiting-decision · 答案隐藏

file-backed-governed-package

Reviewer has not selected A or B yet; answer key remains hidden.

盲评包: reports/output_blind_review_pack.json
决策表: reports/output_review_decisions.json
提示哈希: c79a554b573517682da48879c1ef54399841d9ebf570088a2174dc38de39cc7d

操作命令

prepare_review_kitpython3 scripts/yao.py output-review-kit
write_templatepython3 scripts/adjudicate_output_review.py --write-template
import_decisionspython3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication
adjudicatepython3 scripts/yao.py output-review
refresh_review_studiopython3 scripts/yao.py review-studio .

字段要求

winner_variantA or B after reading only the blind review pack.
confidenceOptional number from 0 to 1.
reasonRequired rationale; do not reveal baseline or with-skill labels before adjudication.
reviewerHuman reviewer name or review group at the decision-file top level.
reviewed_atReview date or timestamp at the decision-file top level.
reviewer_attestation.blind_review_completed_before_answer_keyTrue only after the reviewer has completed choices before opening the answer key.
reviewer_attestation.answer_key_not_opened_before_decisionsTrue only when the answer key was not opened before decisions were recorded.

隐私边界

Do not paste raw private user data into the decision reason.
Do not open the answer key before reviewer choices are recorded.
Leave winner_variant blank when the reviewer is not ready to decide.

发布标准

Governed 和 Library 至少需要 5 个 output eval cases，并覆盖 file-backed、near-neighbor、boundary case、execution evidence 和 blind A/B review pack。

人工结论

没有 reviewer 决策时只显示 pending；只有真实决策文件会进入一致率和分歧统计。

评审方式

先打开 reports/output_review_kit.html 做盲评，填入 reports/output_review_decisions.json，再用 reports/output_review_adjudication.md 核对答案 key。

运行方式

reports/output_execution_runs.md 会标明 recorded fixture、command run 或 model run；只有 provider runner 返回 model metadata 时才算 model-executed。

运行矩阵

目标数: 5
通过数: 5
Fail Count: 0

目标编译

目标数: 5
通过数: 5
Warn Count: 0
Block Count: 0
失败数: 0

上下文

initial load 998/1000; deferred 530871/120000; top deferred scripts 469568; resource governance governed; quality density 140.3

编译证据

Review reports/compiled_targets.md before packaging to inspect target adapter modes, generated files, preserved semantics, warnings, and unsupported features.

信任报告

Secret: 0
脚本数: 155
网络脚本: 3
Help 失败: 0
包体哈希: eddeea80a657e9c97cee2e1242d97577b886b33e2ffa6e319629ef5fcb029383

安全边界

高风险 secret、远程 inline execution、缺失依赖策略或无法解释的脚本接口应阻断 governed release。

Python 兼容

目标 Python: 3.11
文件数: 234
问题数: 0
语法错误: 0
F-string 3.11: 0

解释器边界

CI 和发布审查以 Python 3.11 兼容为底线；本地更高版本允许的新语法不能绕过兼容门禁。

权限批准

3/3 permissions approved; gaps 0; required file_write, network, subprocess

批准策略

高权限能力需要 reviewer、scope、reason、expires_at 和 openai/claude/generic 目标端 enforcement 说明。

权限探针

4/4 targets probed; native 0; metadata fallback 4; installer 4; residual risks 4

运行边界

目标数: 4
通过数: 4
原生执行: 0
Metadata fallback: 4
安装拦截: 4
安装权限失败: 0
残余风险: 4
失败数: 0

组合治理

11 skills, 1 actionable; 0 actionable route collisions; 0 actionable owner gaps; 0 actionable stale; 0 actionable drift; 22 scoped non-actionable issues

下一动作

优先处理真实 portfolio 中的 duplicate names、stale skills、owner gaps，再用运营回路判断真实影响。

运营回路

漂移信号

事件数: 1
采用率: 100
漏触发: 0
Bad Output Count: 0
风险带: low

日常运维

决策: proposal-review
提案数: 5
Approval Count: 0
Pending Review Count: 0
发布锁: 是
世界级: 否
Writes Source Files: 否
Auto Patch Enabled: 否

周度队列

决策: curator-review
Week Id: 2026-W25
日报数: 4
唯一机会: 5
待批准复核: 1
提案复核: 4
Top Score: 88
发布锁: 是
Writes Source Files: 否
Auto Patch Enabled: 否

人工批准

warning 可以被有边界地接受，但必须写入 reviewer、理由、范围和到期时间；blocker 与 world-class 完成证据不能通过 waiver 变成通过。

批准概况

0 active waivers; 1 warning gates still need reviewer decision

批准台账

Waiver Count: 0
Active Count: 0
Expired Count: 0
Invalid Count: 0
覆盖 Gate: 0

批准候选

可批准 · needs-reviewer-decision

Output Lab

review pending 5; model-executed 10; output failures 0

Gate: output-lab
证据: reports/output_review_adjudication.md
选项: accepted-risk, false-positive, temporary-exception
边界: Does not count as provider, human, or public world-class completion evidence.

审查条件

Reviewer confirms this release does not claim provider-backed or human-adjudicated output superiority.
Reviewer names the release scope and expiry date.
Reviewer links output_review_adjudication or output_execution evidence.

建议命令

python3 scripts/yao.py review-waivers . --add-waiver --gate-key output-lab --reviewer "<reviewer>" --reason "Output Lab has pending human/provider evidence; accepted only for this bounded review scope." --expires-at 2027-06-13 --evidence reports/output_review_adjudication.md

不可批准 · cannot-waive

World-Class Evidence

4 pending evidence entries; 1 human pending; 3 external pending

Gate: world-class-evidence
证据: reports/world_class_evidence_ledger.md
选项: 无
边界: Non-waivable completion boundary.

审查条件

Do not use a waiver to claim public world-class readiness.
Either submit accepted ledger evidence or state that this release does not claim world-class completion.
Keep claim guard active until ledger summary.ready_to_claim_world_class is true.

建议命令

python3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions && python3 scripts/yao.py world-class-claim-guard .

世界证据

这里列出每个 world-class 证据项的当前状态、完成定义、证据来源、隐私约束和下一步；计划、metadata fallback、待评审和本地命令不会被当成完成证据。

待补证 · external

Provider Holdout

Collect at least one provider-backed output-eval holdout run with model, timing, and token metadata.

负责人: operator with provider credentials
当前状态: model-executed 10; token-observed 10
下一步: Run provider-backed holdout cases with real credentials and commit only aggregate evidence.
观测值: model_executed_count: 10; timing_observed_count: 10; token_observed_count: 10; accepted: True
提交态: status: missing; path: evidence/world_class/submissions/provider-holdout.json; attested_real_evidence: False; privacy_contract_satisfied: False

执行步骤

Set one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.
For OpenAI Responses: python3 scripts/yao.py output-exec --provider-runner openai --provider-model ${YAO_OUTPUT_EVAL_MODEL:-gpt-4.1-mini} --timeout-seconds 60
For DeepSeek Chat Completions: python3 scripts/yao.py output-exec --provider-runner deepseek --provider-model deepseek-v4-flash --provider-api-format chat-completions --provider-thinking disabled --api-key-env DEEPSEEK_API_KEY --timeout-seconds 120
python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD>
Copy evidence/world_class/templates/provider-holdout.intake.json to evidence/world_class/submissions/provider-holdout.json and fill only real evidence fields.
python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions

完成定义

reports/output_execution_runs.json summary.model_executed_count > 0
reports/output_execution_runs.json summary.timing_observed_count > 0
reports/output_execution_runs.json summary.token_observed_count > 0
reports/skill_os2_audit.json item provider-holdout status becomes pass

证据来源

reports/output_execution_runs.json
reports/output_execution_runs.md
reports/skill_os2_audit.json
evidence/world_class/intake.schema.json
evidence/world_class/templates/provider-holdout.intake.json
reports/world_class_evidence_intake.json
reports/world_class_evidence_intake.md

隐私约束

Do not commit provider credentials or environment dumps.
The output execution report records output hashes and aggregate run metadata, not raw provider prompts.

源证据检查

Provider model runmodel_executed_count: 10 / >0Run provider-backed output-exec with real credentials.
Timing observedtiming_observed_count: 10 / >0Provider execution should record timing metadata.
Token usage observedtoken_observed_count: 10 / >0Provider execution should return non-estimated token usage.

待补证 · human

Human Adjudication

Record real blind A/B reviewer decisions before claiming human output review completion.

负责人: human reviewer
当前状态: 0/5 decisions; pending 5
下一步: Record real A/B choices, reviewer metadata, and blind-review attestation, then regenerate adjudication.
观测值: pair_count: 5; judgment_count: 0; pending_count: 5; invalid_decision_count: 0; answer_revealed_count: 0; reviewer_metadata_present: False; reason_required: True; blind_review_attested: False; raw_content_excluded_attested: True; reviewer_reason_required_attested: True; ready_for_human_evidence: False; raw_content_allowed: False; raw_content_path_count: 0; accepted: False
提交态: status: missing; path: evidence/world_class/submissions/human-adjudication.json; attested_real_evidence: False; privacy_contract_satisfied: False

执行步骤

python3 scripts/yao.py output-review-kit --write-template
Open reports/output_review_kit.md and choose A or B for each pair without opening the answer key.
python3 scripts/adjudicate_output_review.py --write-template
Record reviewer choices in a separate JSON, JSONL, or CSV decision source with reviewer, reviewed_at, case_id, winner_variant, confidence, required reason, and truthful reviewer_attestation only.
python3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication
python3 scripts/yao.py output-review
python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD>
Copy evidence/world_class/templates/human-adjudication.intake.json to evidence/world_class/submissions/human-adjudication.json and fill only real evidence fields.
python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions

完成定义

reports/output_review_adjudication.json summary.pending_count == 0
reports/output_review_adjudication.json summary.judgment_count == summary.pair_count
reports/output_review_adjudication.json summary.invalid_decision_count == 0
reports/output_review_adjudication.json summary.reviewer_metadata_present is true
reports/output_review_adjudication.json summary.blind_review_attested is true
reports/output_review_adjudication.json review_integrity.blind_pack_sha256 exists and matches reports/output_review_decisions.json
reports/output_review_adjudication.json pairs and reviewer_checklist store prompt_sha256, not raw prompt text
reports/output_review_adjudication.json summary.ready_for_human_evidence is true
reports/skill_os2_audit.json item human-adjudication status becomes pass

证据来源

reports/output_blind_review_pack.md
reports/output_review_kit.md
reports/output_review_decisions.json
reports/output_review_adjudication.json
reports/output_review_adjudication.md
scripts/import_output_review_decisions.py
evidence/world_class/intake.schema.json
evidence/world_class/templates/human-adjudication.intake.json
reports/world_class_evidence_intake.json
reports/world_class_evidence_intake.md

隐私约束

Reviewer decisions should not include raw user data or private customer detail.
Reviewer reasons must be rubric-based and must not include raw user data or private customer detail.
The decision importer rejects raw prompt, output, transcript, message, and answer-key fields.
The adjudication evidence stores prompt_sha256 instead of raw prompt text.
The decision and adjudication artifacts preserve blind_pack_sha256 so reviewers can audit exactly which pack was judged.
Keep the answer key separate until after decisions are recorded.

源证据检查

Review pairs existpair_count: 5 / >0Generate the blind A/B review pack.
No pending decisionspending_count: 5 / ==0Record a reviewer choice and reason for every pair.
Judgments completejudgment_count: 0 / ==pair_countEvery pair needs one valid human judgment.
No invalid decisionsinvalid_decision_count: 0 / ==0Fix malformed winner/confidence entries.
Reviewer metadatareviewer_metadata_present: False / trueRecord reviewer and reviewed_at before adjudication can count.
Reason requiredreason_required: True / trueKeep reason mandatory for every imported or direct reviewer decision.
Blind review attestedblind_review_attested: False / trueSet reviewer_attestation only after choices are completed before opening the answer key.
Raw content attestedraw_content_excluded_attested: True / trueAttest that reviewer decisions exclude raw prompts, outputs, transcripts, messages, and private user content.
Raw content blockedraw_content_allowed: False / falseAdjudication evidence should store prompt hashes and reviewer metadata, not raw prompts, outputs, transcripts, or messages.
Human evidence readyready_for_human_evidence: False / trueComplete all reviewer decisions with metadata and rationale, plus blind-review attestation and integrity fingerprints.

待补证 · external

Native Permission Enforcement

Prove at least one real target client or external installer runtime guard enforces approved high-permission capabilities.

负责人: target client or installer integrator
当前状态: native-enforced targets 0; installer-enforced targets 4
下一步: Integrate a real target-client or external installer runtime guard before claiming native permission enforcement.
观测值: native_enforcement_count: 0; metadata_fallback_count: 4; installer_enforcement_pass_count: 4; installer_permission_failure_count: 0; installer_enforcement_ready: True; residual_risk_count: 4; failure_count: 0; accepted: False
提交态: status: missing; path: evidence/world_class/submissions/native-permission-enforcement.json; attested_real_evidence: False; privacy_contract_satisfied: False

执行步骤

Implement or connect a real target client or external installer runtime guard that blocks undeclared network, file_write, or subprocess capabilities.
Update the generated target adapter only when the guard is actually enforced by that target.
python3 scripts/yao.py package . --platform openai --platform claude --platform generic --platform vscode --output-dir dist --zip
python3 scripts/yao.py install-simulate . --package-dir dist --install-root dist/install-simulation
python3 scripts/yao.py runtime-permissions . --package-dir dist
python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD>
Copy evidence/world_class/templates/native-permission-enforcement.intake.json to evidence/world_class/submissions/native-permission-enforcement.json and fill only real evidence fields.
python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions

完成定义

reports/runtime_permission_probes.json summary.native_enforcement_count > 0
reports/runtime_permission_probes.json summary.failure_count == 0
reports/runtime_permission_probes.json summary.installer_enforcement_pass_count records local installer enforcement but does not replace native evidence
reports/skill_os2_audit.json item native-permission-enforcement status becomes pass

证据来源

dist/targets/*/adapter.json
reports/runtime_permission_probes.json
reports/runtime_permission_probes.md
reports/install_simulation.json
reports/install_simulation.md
security/permission_policy.json
evidence/world_class/intake.schema.json
evidence/world_class/templates/native-permission-enforcement.intake.json
reports/world_class_evidence_intake.json
reports/world_class_evidence_intake.md

隐私约束

Do not mark native_enforcement true for metadata-only fallbacks.
Keep residual risks visible for targets that still rely on operator enforcement.

源证据检查

Native enforcementnative_enforcement_count: 0 / >0Collect real target-client or external runtime guard proof.
Probe failuresfailure_count: 0 / ==0Runtime permission probes must stay clean.
Installer supportinstaller_enforcement_ready: True / trueInstaller enforcement is supporting evidence, not native proof.

待补证 · external

Native Client Telemetry

Import production metadata-only events from a real external client into the local drift loop.

负责人: Browser/Chrome/IDE/provider client integrator
当前状态: external source events 0; adoption samples 1
下一步: Install a real client against the native host and import production metadata-only events.
观测值: external_source_events: 0; adoption_sample_count: 1; raw_content_allowed: False; risk_band: low; accepted: False
提交态: status: missing; path: evidence/world_class/submissions/native-client-telemetry.json; attested_real_evidence: False; privacy_contract_satisfied: False

执行步骤

python3 scripts/telemetry_native_host.py . --write-launcher /tmp/yao-telemetry-host.sh --write-manifest /tmp/yao-telemetry-host.json --allowed-origin chrome-extension://<extension-id>/
Install the generated native messaging manifest for the real client and send at least one accepted skill_activation or skill_output event.
python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl
python3 scripts/yao.py skill-atlas --workspace-root .
python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD>
Copy evidence/world_class/templates/native-client-telemetry.intake.json to evidence/world_class/submissions/native-client-telemetry.json and fill only real evidence fields.
python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions

完成定义

reports/adoption_drift_report.json summary.source_types.external > 0
reports/adoption_drift_report.json summary.adoption_sample_count > 0
reports/skill_os2_audit.json item native-client-telemetry status becomes pass

证据来源

reports/adoption_drift_report.json
reports/adoption_drift_report.md
reports/telemetry_hook_recipes.json
scripts/telemetry_native_host.py
evidence/world_class/intake.schema.json
evidence/world_class/templates/native-client-telemetry.intake.json
reports/world_class_evidence_intake.json
reports/world_class_evidence_intake.md

隐私约束

Telemetry must remain metadata-only and local-first.
Do not package reports/telemetry_events.jsonl or any raw prompt, output, transcript, note, or message field.

源证据检查

External eventsexternal_source_events: 0 / >0Import at least one metadata-only event from a real client.
Adoption sampleadoption_sample_count: 1 / >0Telemetry must include adoption outcome evidence.
Raw content blockedraw_content_allowed: False / falseTelemetry must stay metadata-only.

提交清单

每张卡片给出模板、提交文件、准备命令、校验命令、收集要求、通过条件和隐私边界；只有真实 provider、真人、原生权限或真实客户端证据通过 intake 后，才进入 ledger review。

awaiting-submission · external

Provider Holdout

No real evidence submission has been provided yet.

负责人: operator with provider credentials
模板: evidence/world_class/templates/provider-holdout.intake.json
提交: evidence/world_class/submissions/provider-holdout.json
下一步: Run provider-backed holdout cases with real credentials and commit only aggregate evidence.

操作命令

prepare_submissionpython3 scripts/yao.py world-class-submission-kit . --evidence-key provider-holdout --output-dir evidence/world_class/submissions
validate_intakepython3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
submission_reviewpython3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
refresh_ledgerpython3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
guard_claimpython3 scripts/yao.py world-class-claim-guard .

收集要求

provider-backed model run
observed timing
observed token metadata

执行步骤

Set one provider API key in the operator shell, such as OPENAI_API_KEY or DEEPSEEK_API_KEY; never commit or print the value.
For OpenAI Responses: python3 scripts/yao.py output-exec --provider-runner openai --provider-model ${YAO_OUTPUT_EVAL_MODEL:-gpt-4.1-mini} --timeout-seconds 60
For DeepSeek Chat Completions: python3 scripts/yao.py output-exec --provider-runner deepseek --provider-model deepseek-v4-flash --provider-api-format chat-completions --provider-thinking disabled --api-key-env DEEPSEEK_API_KEY --timeout-seconds 120
python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD>
Copy evidence/world_class/templates/provider-holdout.intake.json to evidence/world_class/submissions/provider-holdout.json and fill only real evidence fields.
python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions

通过条件

reports/output_execution_runs.json summary.model_executed_count > 0
reports/output_execution_runs.json summary.timing_observed_count > 0
reports/output_execution_runs.json summary.token_observed_count > 0
reports/skill_os2_audit.json item provider-holdout status becomes pass

隐私边界

Do not commit provider credentials or environment dumps.
The output execution report records output hashes and aggregate run metadata, not raw provider prompts.

awaiting-submission · human

Human Adjudication

No real evidence submission has been provided yet.

负责人: human reviewer
模板: evidence/world_class/templates/human-adjudication.intake.json
提交: evidence/world_class/submissions/human-adjudication.json
下一步: Record real A/B choices, reviewer metadata, and blind-review attestation, then regenerate adjudication.

操作命令

prepare_submissionpython3 scripts/yao.py world-class-submission-kit . --evidence-key human-adjudication --output-dir evidence/world_class/submissions
validate_intakepython3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
submission_reviewpython3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
refresh_ledgerpython3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
guard_claimpython3 scripts/yao.py world-class-claim-guard .

收集要求

real reviewer identity
blind A/B decisions
answer key unopened until decisions exist

执行步骤

python3 scripts/yao.py output-review-kit --write-template
Open reports/output_review_kit.md and choose A or B for each pair without opening the answer key.
python3 scripts/adjudicate_output_review.py --write-template
Record reviewer choices in a separate JSON, JSONL, or CSV decision source with reviewer, reviewed_at, case_id, winner_variant, confidence, required reason, and truthful reviewer_attestation only.
python3 scripts/yao.py output-review-import --input <reviewer-decisions.json> --blind-review-attested --run-adjudication
python3 scripts/yao.py output-review
python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD>
Copy evidence/world_class/templates/human-adjudication.intake.json to evidence/world_class/submissions/human-adjudication.json and fill only real evidence fields.
python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions

通过条件

reports/output_review_adjudication.json summary.pending_count == 0
reports/output_review_adjudication.json summary.judgment_count == summary.pair_count
reports/output_review_adjudication.json summary.invalid_decision_count == 0
reports/output_review_adjudication.json summary.reviewer_metadata_present is true
reports/output_review_adjudication.json summary.blind_review_attested is true
reports/output_review_adjudication.json review_integrity.blind_pack_sha256 exists and matches reports/output_review_decisions.json
reports/output_review_adjudication.json pairs and reviewer_checklist store prompt_sha256, not raw prompt text
reports/output_review_adjudication.json summary.ready_for_human_evidence is true
reports/skill_os2_audit.json item human-adjudication status becomes pass

隐私边界

Reviewer decisions should not include raw user data or private customer detail.
Reviewer reasons must be rubric-based and must not include raw user data or private customer detail.
The decision importer rejects raw prompt, output, transcript, message, and answer-key fields.
The adjudication evidence stores prompt_sha256 instead of raw prompt text.
The decision and adjudication artifacts preserve blind_pack_sha256 so reviewers can audit exactly which pack was judged.
Keep the answer key separate until after decisions are recorded.

awaiting-submission · external

Native Permission Enforcement

No real evidence submission has been provided yet.

负责人: target client or installer integrator
模板: evidence/world_class/templates/native-permission-enforcement.intake.json
提交: evidence/world_class/submissions/native-permission-enforcement.json
下一步: Integrate a real target-client or external installer runtime guard before claiming native permission enforcement.

操作命令

prepare_submissionpython3 scripts/yao.py world-class-submission-kit . --evidence-key native-permission-enforcement --output-dir evidence/world_class/submissions
validate_intakepython3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
submission_reviewpython3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
refresh_ledgerpython3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
guard_claimpython3 scripts/yao.py world-class-claim-guard .

收集要求

real target client or external installer runtime guard
native enforcement flag or externally accepted guard proof
residual risk retained for fallback targets

执行步骤

Implement or connect a real target client or external installer runtime guard that blocks undeclared network, file_write, or subprocess capabilities.
Update the generated target adapter only when the guard is actually enforced by that target.
python3 scripts/yao.py package . --platform openai --platform claude --platform generic --platform vscode --output-dir dist --zip
python3 scripts/yao.py install-simulate . --package-dir dist --install-root dist/install-simulation
python3 scripts/yao.py runtime-permissions . --package-dir dist
python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD>
Copy evidence/world_class/templates/native-permission-enforcement.intake.json to evidence/world_class/submissions/native-permission-enforcement.json and fill only real evidence fields.
python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions

通过条件

reports/runtime_permission_probes.json summary.native_enforcement_count > 0
reports/runtime_permission_probes.json summary.failure_count == 0
reports/runtime_permission_probes.json summary.installer_enforcement_pass_count records local installer enforcement but does not replace native evidence
reports/skill_os2_audit.json item native-permission-enforcement status becomes pass

隐私边界

Do not mark native_enforcement true for metadata-only fallbacks.
Keep residual risks visible for targets that still rely on operator enforcement.

awaiting-submission · external

Native Client Telemetry

No real evidence submission has been provided yet.

负责人: Browser/Chrome/IDE/provider client integrator
模板: evidence/world_class/templates/native-client-telemetry.intake.json
提交: evidence/world_class/submissions/native-client-telemetry.json
下一步: Install a real client against the native host and import production metadata-only events.

操作命令

prepare_submissionpython3 scripts/yao.py world-class-submission-kit . --evidence-key native-client-telemetry --output-dir evidence/world_class/submissions
validate_intakepython3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions
submission_reviewpython3 scripts/yao.py world-class-submission-review . --submissions-dir evidence/world_class/submissions
refresh_ledgerpython3 scripts/yao.py world-class-ledger . --submissions-dir evidence/world_class/submissions
guard_claimpython3 scripts/yao.py world-class-claim-guard .

收集要求

real external client source
metadata-only event
local-first import path

执行步骤

python3 scripts/telemetry_native_host.py . --write-launcher /tmp/yao-telemetry-host.sh --write-manifest /tmp/yao-telemetry-host.json --allowed-origin chrome-extension://<extension-id>/
Install the generated native messaging manifest for the real client and send at least one accepted skill_activation or skill_output event.
python3 scripts/yao.py telemetry-import . --input-jsonl .yao/telemetry_spool/external_events.jsonl
python3 scripts/yao.py skill-atlas --workspace-root .
python3 scripts/yao.py skill-os2-audit . --generated-at <YYYY-MM-DD>
Copy evidence/world_class/templates/native-client-telemetry.intake.json to evidence/world_class/submissions/native-client-telemetry.json and fill only real evidence fields.
python3 scripts/yao.py world-class-intake . --submissions-dir evidence/world_class/submissions

通过条件

reports/adoption_drift_report.json summary.source_types.external > 0
reports/adoption_drift_report.json summary.adoption_sample_count > 0
reports/skill_os2_audit.json item native-client-telemetry status becomes pass

隐私边界

Telemetry must remain metadata-only and local-first.
Do not package reports/telemetry_events.jsonl or any raw prompt, output, transcript, note, or message field.

蓝图覆盖

项目数: 21
模块数: 8
建议 PR: 13
通过数: 21
Warn Count: 0
缺失数: 0
Extension Track Count: 4
Extension Covered Count: 4
Extension Partial Count: 0
Extension Planned Count: 0
Adaptive Extension Ready: 是
本地蓝图: 是
世界级: 否
待补证据: 4

覆盖边界

蓝图覆盖只证明 2.0 模块、建议 PR、脚本、报告和测试在本地闭环；public world-class 仍以 world-class evidence ledger 的真人和外部证据为准。

公开声明

本地复现: 是
发布锁: 是
可公开声明: 否
声明阻断: 3
Provider 证据: 是
人审完成: 否
世界级就绪: 否

声明阻断

阻断human blind-review adjudication is incomplete
阻断world-class evidence is not accepted yet (3 open gaps, 4 ledger pending)
阻断world-class source checks are not all accepted (12/19 pass, 7 blocked)

世界证据

4 pending world-class evidence entries; 1 human pending; 3 external pending; source checks 12/19 pass; 7 blocked; overclaim guard true

证据台账

Ledger Entry Count: 4
Accepted Count: 0
待审: 4
Human Pending Count: 1
External Pending Count: 3
Overclaim Guard Active: 是
Ready To Claim World Class: 否

证据入口

Schema: 是
模板数: 4
模板通过: 4
提交数: 0
有效提交: 0
无效提交: 0
可收证据: 是
可审台账: 否
Ready To Claim World Class: 否

入口边界

intake 只校验证据包格式、来源、隐私和反过度声明；只有 ledger 看到真实 provider、真人、原生权限或真实客户端结果后，world-class 才能进入完成审计。

声明守卫

台账可声明: 否
台账待补: 4
声明面: 184
违规数: 0
Overclaim Guard Active: 是

声明边界

claim guard 扫描 README、docs 和 reports 中的完成态表述；ledger 未 ready 时，任何英文完成断言、true 状态声明或中文完成态都会阻断发布审查。

注册审计

yao-meta-skill 1.1.0; 6/6 compatibility entries pass; install pass with 4 adapters; installer permissions 12 enforced / 0 failures

包体元数据

名称: yao-meta-skill
版本: 1.1.0
Maturity: governed
Owner: Yao Team
License: MIT
信任级别: local
目标平台: openai, claude, generic, agent-skills-compatible, vscode
兼容通过: 6/6
归档哈希: c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7

发布路线

0 promote; 3 keep current; 0 blocked; upgrade minor declared / minor recommended

包体验证

目标数: 4
Adapter: 4
归档存在: 是
Zip 条目: 580
失败数: 0
警告数: 0
归档哈希: c56a8519244c0bc8ec17c92d4641c832629cc059f4b51fe177a2c039515253a7

安装模拟

Archive Extracted: 是
Entrypoint Loaded: 是
Manifest Loaded: 是
Interface Loaded: 是
Adapter: 4
安装权限: 12
安装权限失败: 0
权限目标: 4
权限能力: 3
失败数: 0
警告数: 0

权限覆盖

安装模拟会读取解压后的 target adapter，并按 declared_capabilities 核对 security/permission_policy.json 中的有效批准、证据、过期时间和目标端 enforcement 说明。