2026-03-31 19:59:29 +08:00
2026-03-31 19:59:29 +08:00
2026-03-31 19:59:29 +08:00
2026-03-31 19:59:29 +08:00

Yao Meta Skill

English 中文 日本語 Français Русский

yao-meta-skill is a meta-skill for building other agent skills.

It turns rough workflows, transcripts, prompts, notes, and runbooks into reusable skill packages with:

  • a clear trigger surface
  • a lean SKILL.md
  • optional references, scripts, and evals
  • neutral source metadata plus client-specific adapters

Quick Start

  1. Describe the workflow, prompt set, or repeated task you want to turn into a skill.
  2. Use yao-meta-skill to generate or improve the package in scaffold, production, or library mode.
  3. Run context_sizer.py, trigger_eval.py, and cross_packager.py as needed to validate and export the result.

5-Minute Workflow

  1. Start from a raw workflow note.
  2. Turn it into a skill package with SKILL.md, agents/interface.yaml, and only the folders the workflow actually needs.
  3. Validate the trigger description with evals/trigger_cases.json.
  4. Export compatibility artifacts for the clients you care about.
  5. Compare the result against the examples in examples/.

Minimum commands:

python3 scripts/trigger_eval.py --description-file evals/improved_description.txt --cases evals/trigger_cases.json
python3 scripts/context_sizer.py .
python3 scripts/cross_packager.py . --platform openai --platform claude --platform generic --expectations evals/packaging_expectations.json --zip
python3 tests/verify_packager_failures.py

Or run everything together:

make test

Results

The homepage panel below is generated from the current eval suite so the family-level outcome is visible without opening raw JSON.

  • regression corpus: 66 prompts across 21 families
  • aggregate result: 0 false positives, 0 false negatives, average precision 1.0, average recall 1.0
  • suite status:
Suite Cases FP FN Precision Recall
train 31 0 0 1.0 1.0
dev 22 0 0 1.0 1.0
holdout 13 0 0 1.0 1.0
Family Cases Pass Rate
brainstorm_only 2 1.0
brainstorm_vs_build 1 1.0
complex_multi_asset 3 1.0
document_export_vs_agent_skill 4 1.0
document_only 3 1.0
explain_not_package 1 1.0
explain_only 5 1.0
future_outline_vs_build 4 1.0
iterate_existing_skill 5 1.0
long_context_document_only 3 1.0
long_context_near_neighbor 3 1.0
long_context_summary_only 2 1.0
long_context_trigger 4 1.0
meta_skill_creation 1 1.0
one_off_vs_reusable 2 1.0
package_for_team 2 1.0
paraphrase_trigger 5 1.0
partial_scaffold_not_full_skill 4 1.0
summary_only 3 1.0
translate_only 4 1.0
workflow_to_skill 5 1.0

Full reports: reports/eval_suite.json and reports/family_summary.md

  • packaging validation: openai, claude, and generic targets pass contract checks
  • packaging failure fixtures: invalid metadata, invalid YAML, and unsupported targets fail as expected
  • failure library regressions: anti-pattern families pass automated checks

What It Does

This project helps you create, refactor, evaluate, and package skills as durable capability bundles rather than one-off prompts.

The design logic is simple:

  1. Capture the real recurring job behind the user's request.
  2. Set a clean skill boundary so one package does one coherent job.
  3. Optimize the trigger description before over-writing the body.
  4. Keep the main skill file small and move details into references or scripts.
  5. Add quality gates only when they pay for themselves.
  6. Export compatibility artifacts only for the clients you actually need.

Why It Exists

Most teams keep valuable operating knowledge scattered across chats, personal prompts, oral habits, and undocumented workflows. This project converts that hidden process knowledge into:

  • discoverable skill packages
  • repeatable execution flows
  • lower-context instructions
  • reusable team assets
  • compatibility-ready distributions

Repository Structure

yao-meta-skill/
├── SKILL.md
├── README.md
├── LICENSE
├── .gitignore
├── agents/
│   └── interface.yaml
├── evals/
├── examples/
├── references/
├── scripts/
└── templates/

Core Components

SKILL.md

The main skill entrypoint. It defines the trigger surface, operating modes, compact workflow, and output contract.

agents/interface.yaml

The neutral metadata source of truth. It stores display and compatibility metadata without locking the source tree to one vendor-specific path.

references/

Long-form material that should not bloat the main skill file. This includes design rules, evaluation guidance, compatibility strategy, and quality rubrics.

scripts/

Utility scripts that make the meta-skill operational:

  • trigger_eval.py: evaluates trigger descriptions with semantic intent concepts, explicit exclusions, and near-neighbor prompts
  • run_eval_suite.py: runs train/dev/holdout trigger suites, reports family-level regressions, and fails if aggregate regressions appear
  • context_sizer.py: estimates context weight and warns when the initial load gets too large
  • cross_packager.py: builds client-specific export artifacts with explicit platform contracts and validation
  • init_skill.py, lint_skill.py, validate_skill.py, diff_eval.py: minimal authoring toolchain

evals/

Reusable trigger and packaging checks, including baseline and improved descriptions for comparison.

examples/

End-to-end examples showing raw workflow input, design summary, and final generated skill shape, including one thicker complex benchmark and one evolution chain.

.github/workflows/test.yml

Continuous integration entrypoint that runs the full local regression suite on push and pull request.

Validation Notes

  • Trigger evaluation now uses a local semantic-intent model with explicit positive concepts, exclusion concepts, and boundary-case reporting.
  • The sample trigger report now covers a larger positive, negative, and near-neighbor set rather than a tiny demo set.
  • Train/dev/holdout trigger suites now separate iterative tuning from final verification.
  • Packaging validation now uses explicit contracts and YAML parsing, but it is still a lightweight local validation layer rather than a full platform integration suite.
  • evals/failure-cases.md captures known weak spots that should remain part of regression checks.
  • failures/ captures reusable anti-pattern writeups and machine-runnable failure cases for routing, packaging, and authoring failures.
  • tests/verify_packager_failures.py checks that invalid metadata, invalid YAML, and unsupported targets fail clearly.

templates/

Starter templates for simple and more advanced skill packages.

How To Use

1. Use the skill directly

Invoke yao-meta-skill when you want to:

  • create a new skill
  • improve an existing skill
  • add evals to a skill
  • convert a workflow into a reusable package
  • prepare a skill for wider team adoption

2. Generate a new skill package

The typical flow is:

  1. describe the workflow or capability
  2. identify trigger phrases and outputs
  3. choose scaffold, production, or library mode
  4. generate the package
  5. run the sizing and trigger checks if needed
  6. export target-specific compatibility artifacts

3. Export compatibility artifacts

Examples:

python3 scripts/cross_packager.py ./yao-meta-skill --platform openai --platform claude --expectations evals/packaging_expectations.json --zip
python3 scripts/context_sizer.py ./yao-meta-skill
python3 scripts/trigger_eval.py --description-file evals/improved_description.txt --cases evals/trigger_cases.json --baseline-description-file evals/baseline_description.txt

Advantages

  • Neutral by default: source files stay vendor-neutral, while adapters are generated only when needed.
  • Context efficient: the project explicitly pushes detail out of the main skill file.
  • Evaluation-aware: trigger and sizing checks are built into the workflow.
  • Reusable: the output is a package, not just a paragraph of prompt text.
  • Portable: compatibility is handled through packaging rather than duplicating source files for every client.

Best Fit

This project is best for:

  • agent builders
  • internal tooling teams
  • prompt engineers moving toward structured skills
  • organizations building reusable skill libraries

Documentation

Language Entry
English README.md
中文 docs/README.zh-CN.md
日本語 docs/README.ja-JP.md
Français docs/README.fr-FR.md
Русский docs/README.ru-RU.md

Examples And Evals

License

MIT. See LICENSE.

S
Description
创建、评估、打包和治理可复用 Agent Skill 的工程化框架
Readme MIT 18 MiB
Languages
Python 77.6%
HTML 20.7%
CSS 1.2%
Makefile 0.4%