commit d4eccba69e8e96dbfd3217313f37d422db056894 Author: yaojingang Date: Tue Mar 31 19:59:29 2026 +0800 feat: open source yao-meta-skill diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..e82e7deb --- /dev/null +++ b/.gitignore @@ -0,0 +1,4 @@ +.DS_Store +dist/ +*.zip +__pycache__/ diff --git a/LICENSE b/LICENSE new file mode 100644 index 00000000..766e4f14 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Yao Team + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 00000000..64add83d --- /dev/null +++ b/README.md @@ -0,0 +1,133 @@ +# Yao Meta Skill + +`yao-meta-skill` is a meta-skill for building other agent skills. + +It turns rough workflows, transcripts, prompts, notes, and runbooks into reusable skill packages with: + +- a clear trigger surface +- a lean `SKILL.md` +- optional references, scripts, and evals +- neutral source metadata plus client-specific adapters + +## What It Does + +This project helps you create, refactor, evaluate, and package skills as durable capability bundles rather than one-off prompts. + +The design logic is simple: + +1. Capture the real recurring job behind the user's request. +2. Set a clean skill boundary so one package does one coherent job. +3. Optimize the trigger description before over-writing the body. +4. Keep the main skill file small and move details into references or scripts. +5. Add quality gates only when they pay for themselves. +6. Export compatibility artifacts only for the clients you actually need. + +## Why It Exists + +Most teams keep valuable operating knowledge scattered across chats, personal prompts, oral habits, and undocumented workflows. This project converts that hidden process knowledge into: + +- discoverable skill packages +- repeatable execution flows +- lower-context instructions +- reusable team assets +- compatibility-ready distributions + +## Repository Structure + +```text +yao-meta-skill/ +├── SKILL.md +├── README.md +├── LICENSE +├── .gitignore +├── agents/ +│ └── interface.yaml +├── references/ +├── scripts/ +└── templates/ +``` + +## Core Components + +### `SKILL.md` + +The main skill entrypoint. It defines the trigger surface, operating modes, compact workflow, and output contract. + +### `agents/interface.yaml` + +The neutral metadata source of truth. It stores display and compatibility metadata without locking the source tree to one vendor-specific path. + +### `references/` + +Long-form material that should not bloat the main skill file. This includes design rules, evaluation guidance, compatibility strategy, and quality rubrics. + +### `scripts/` + +Utility scripts that make the meta-skill operational: + +- `trigger_eval.py`: checks whether a trigger description is too broad or too weak +- `context_sizer.py`: estimates context weight and warns when the initial load gets too large +- `cross_packager.py`: builds client-specific export artifacts from the neutral source package + +### `templates/` + +Starter templates for simple and more advanced skill packages. + +## How To Use + +### 1. Use the skill directly + +Invoke `yao-meta-skill` when you want to: + +- create a new skill +- improve an existing skill +- add evals to a skill +- convert a workflow into a reusable package +- prepare a skill for wider team adoption + +### 2. Generate a new skill package + +The typical flow is: + +1. describe the workflow or capability +2. identify trigger phrases and outputs +3. choose scaffold, production, or library mode +4. generate the package +5. run the sizing and trigger checks if needed +6. export target-specific compatibility artifacts + +### 3. Export compatibility artifacts + +Examples: + +```bash +python3 scripts/cross_packager.py ./yao-meta-skill --platform openai --platform claude --zip +python3 scripts/context_sizer.py ./yao-meta-skill +python3 scripts/trigger_eval.py --description "Create and improve agent skills..." --cases ./cases.json +``` + +## Advantages + +- **Neutral by default**: source files stay vendor-neutral, while adapters are generated only when needed. +- **Context efficient**: the project explicitly pushes detail out of the main skill file. +- **Evaluation-aware**: trigger and sizing checks are built into the workflow. +- **Reusable**: the output is a package, not just a paragraph of prompt text. +- **Portable**: compatibility is handled through packaging rather than duplicating source files for every client. + +## Best Fit + +This project is best for: + +- agent builders +- internal tooling teams +- prompt engineers moving toward structured skills +- organizations building reusable skill libraries + +## Documentation + +- Chinese introduction: [docs/README.zh-CN.md](docs/README.zh-CN.md) +- Japanese introduction: [docs/README.ja-JP.md](docs/README.ja-JP.md) + +## License + +MIT. See [LICENSE](LICENSE). diff --git a/SKILL.md b/SKILL.md new file mode 100644 index 00000000..105fc32f --- /dev/null +++ b/SKILL.md @@ -0,0 +1,177 @@ +--- +name: yao-meta-skill +description: Create, refactor, evaluate, and package agent skills from workflows, transcripts, prompts, docs, or rough notes. Use when asked to create a skill, turn a repeated process into a reusable skill, improve an existing skill, optimize skill triggering, add evals, or prepare a skill for team reuse. Relevant for meta-skills, skill factories, skill templates, skill QA, and cross-platform skill packaging. +metadata: + author: Yao Team + philosophy: "structured design, evaluation loop, template ergonomics, operational packaging" +--- + +# Yao Meta Skill + +Build skills as reusable products, not long prompts. + +## Core Rules + +- Treat a skill as a maintained capability package. +- Write the frontmatter `description` early; it is the main trigger surface. +- Keep `SKILL.md` lean. Put detail in `references/`, deterministic logic in `scripts/`, and output artifacts in `assets/`. +- Use the lightest process that still protects quality. +- Package for reuse only when the user actually needs reuse. + +## Use Cases + +Use this skill to: + +- create a new skill +- turn a workflow, runbook, transcript, or prompt set into a skill +- improve a skill's boundary, description, evals, or packaging +- design a team skill template or skill-library standard +- migrate a skill toward the Agent Skills open format + +## Modes + +Choose the lightest mode that fits. + +### Scaffold + +Use for exploratory or personal skills. + +Deliver: + +- `SKILL.md` +- `agents/interface.yaml` +- `references/` only if clearly needed + +### Production + +Use for reusable team skills. + +Deliver: + +- concise package structure +- focused `references/` +- `scripts/` when prose would be brittle or repetitive +- `evals/` when output quality can be checked + +### Library + +Use for important organizational skills or meta-skills. + +Add: + +- trigger positives, negatives, and near neighbors +- revision rubric +- packaging guidance +- maintenance metadata when useful + +## Factory Components + +Use these when they materially improve quality: + +- `templates/basic_skill.md.j2` +- `templates/complex_skill.md.j2` +- `scripts/trigger_eval.py` +- `scripts/context_sizer.py` +- `scripts/cross_packager.py` + +## Workflow + +### 1. Capture the real job + +Infer: + +- the recurring task or decision +- likely trigger phrases and contexts +- expected outputs +- what must be deterministic +- whether the skill is personal, team, or cross-platform + +Keep discovery lean. Default to no more than two clarification rounds unless guessing is risky. + +### 2. Set the boundary + +One skill should usually have: + +- one capability family +- one trigger surface +- one coherent workflow + +Split oversized skills. Move variants into `references/` or sibling skills. + +### 3. Design the trigger + +The `description` should say: + +- what the skill does +- when to use it +- phrases, artifacts, or file types that should trigger it +- adjacent cases that are easy to miss + +For important skills, create `should_trigger`, `should_not_trigger`, and near-neighbor prompts. Use `scripts/trigger_eval.py` when helpful. + +### 4. Write the package + +Default structure: + +```text +skill-name/ +├── SKILL.md +├── agents/interface.yaml +├── references/ +├── scripts/ +├── assets/ +└── evals/ +``` + +Only create folders that earn their keep. Start from the basic template unless the skill clearly needs the complex one. + +### 5. Add quality gates + +Use the minimum useful QA: + +- basic: structure and naming check +- standard: realistic prompts and expected outcomes +- advanced: trigger evals, benchmark comparisons, revision loop + +For production or library-grade skills, run `scripts/context_sizer.py` before finalizing. + +### 6. Package for reuse + +If team reuse matters, include: + +- stable folder name +- aligned `agents/interface.yaml` +- minimal tool assumptions +- version or maintenance metadata when useful +- target-specific packaging only for requested platforms + +Use `scripts/cross_packager.py` when packaging artifacts are needed. + +### 7. Report the result + +Summarize: + +- what was packaged +- what trigger surface was chosen +- what was excluded +- what quality gates exist +- what should be improved next + +## Output Contract + +Unless the user asks otherwise, produce: + +1. a working skill directory +2. a trigger-aware `SKILL.md` +3. aligned `agents/interface.yaml` +4. references only where they reduce context bloat +5. optional scripts, evals, and `manifest.json` when justified + +## Reference Map + +- [Skill Design Guidelines](references/skill_design_guidelines.md) +- [Client Compatibility](references/client-compatibility.md) +- [Comparative Analysis](references/comparative-analysis.md) +- [Meta-Skill Rubric](references/design-rubric.md) +- [Skill Template](references/skill-template.md) +- [Trigger And Eval Playbook](references/eval-playbook.md) diff --git a/agents/interface.yaml b/agents/interface.yaml new file mode 100644 index 00000000..1838f458 --- /dev/null +++ b/agents/interface.yaml @@ -0,0 +1,10 @@ +interface: + display_name: "Yao Meta Skill" + short_description: "Create robust, trigger-aware agent skills" + default_prompt: "Use $yao-meta-skill to turn my workflow, prompt set, or rough notes into a production-ready skill with strong triggering, lean structure, and an eval plan." +compatibility: + canonical_format: "agent-skills" + adapter_targets: + - "openai" + - "claude" + - "generic" diff --git a/docs/README.ja-JP.md b/docs/README.ja-JP.md new file mode 100644 index 00000000..d6d1f814 --- /dev/null +++ b/docs/README.ja-JP.md @@ -0,0 +1,57 @@ +# Yao Meta Skill 日本語紹介 + +`yao-meta-skill` は「スキルを作るためのスキル」です。 + +目的は、長いプロンプトを書くことではなく、散在したワークフロー、会話ログ、運用ノウハウ、手順書を、再利用可能で評価可能かつ配布可能なスキルパッケージに変換することです。 + +## 基本ロジック + +処理の流れは次の 6 段階です。 + +1. ユーザーが本当に繰り返している仕事を特定する +2. スキルの境界を明確にし、1 つのパッケージで役割を混在させない +3. 本文を長くする前に、トリガー用 description を最適化する +4. `SKILL.md` を小さく保ち、詳細は `references/` や `scripts/` に分離する +5. 必要な場合にだけ評価や品質ゲートを追加する +6. 必要なクライアント向けにだけ互換アダプタを生成する + +## 使い方 + +次のようなケースに向いています。 + +- 新しい skill を作る +- 既存 skill を改善する +- workflow を skill 化する +- skill に eval を追加する +- チーム向けに skill を共有または公開する + +一般的な流れ: + +1. 対象となる workflow や能力を説明する +2. トリガー条件と期待出力を整理する +3. `Scaffold`、`Production`、`Library` モードを選ぶ +4. skill パッケージを生成する +5. トリガー評価とコンテキストサイズチェックを実行する +6. 必要なクライアント向け互換パッケージを出力する + +## 強み + +- **中立的な設計**: ソースツリーは単一ベンダーのパスに依存しない +- **コンテキスト効率**: メインファイルを小さく保つ前提で設計されている +- **評価可能**: トリガー評価とサイズ計測の仕組みを含む +- **再利用しやすい**: 出力が単発プロンプトではなく skill パッケージになる +- **互換性重視**: 複数クライアント対応をソース複製ではなくパッケージ生成で実現する + +## リポジトリ構成 + +- `SKILL.md`: メインエントリ +- `agents/interface.yaml`: 中立的なメタデータ源 +- `references/`: 長い説明や設計ルール +- `scripts/`: 評価、サイズ計測、互換パッケージ生成 +- `templates/`: 基本テンプレートと複雑テンプレート + +## 想定ユーザー + +- agent 用の能力ライブラリを構築する人 +- 内部自動化や知識資産化を進めるチーム +- prompt engineering から skill engineering に進みたい個人や組織 diff --git a/docs/README.zh-CN.md b/docs/README.zh-CN.md new file mode 100644 index 00000000..d4db0087 --- /dev/null +++ b/docs/README.zh-CN.md @@ -0,0 +1,57 @@ +# Yao Meta Skill 中文介绍 + +`yao-meta-skill` 是一个“生成技能的技能”。 + +它的目标不是再写一段更长的提示词，而是把零散的工作流、聊天记录、经验规则和运行手册，沉淀成可复用、可评估、可分发的技能包。 + +## 核心逻辑 + +它的工作逻辑分成六步： + +1. 识别用户真正反复要做的事情 +2. 给技能划定清晰边界，避免一个技能包做太多事 +3. 优先优化触发描述，而不是先把正文写得很长 +4. 把主 `SKILL.md` 保持精简，细节放进 `references/` 或 `scripts/` +5. 只在必要时加入评测和质量门槛 +6. 只为真正需要的客户端生成兼容适配文件 + +## 怎么使用 + +适合这些场景： + +- 创建新 skill +- 优化已有 skill +- 把 workflow 转成 skill +- 给 skill 增加 eval +- 为团队开源或共享 skill + +常见使用流程： + +1. 描述你的目标流程或能力 +2. 明确触发语境和产出结果 +3. 选择 `Scaffold`、`Production` 或 `Library` 模式 +4. 生成 skill 包 +5. 运行触发检查和上下文体积检查 +6. 按目标客户端导出兼容产物 + +## 优势与特点 + +- **默认中性**：源码结构不绑定单一厂商路径 +- **上下文节省**：强调主入口精简和按需加载 +- **可评测**：自带触发检查与体积检查工具 +- **可复用**：产物是完整 skill 包，不是一段一次性 prompt +- **可兼容**：通过打包阶段生成适配层，而不是在源码中堆叠多套品牌文件 + +## 仓库结构 + +- `SKILL.md`：主入口 +- `agents/interface.yaml`：中性元数据源 +- `references/`：长文档和规则说明 +- `scripts/`：评测、体积分析、兼容打包脚本 +- `templates/`：基础模板与复杂模板 + +## 适合谁 + +- 构建 agent 能力库的人 +- 做内部自动化和知识沉淀的团队 +- 想把 prompt 工程升级成 skill 工程的个人或组织 diff --git a/manifest.json b/manifest.json new file mode 100644 index 00000000..528e6906 --- /dev/null +++ b/manifest.json @@ -0,0 +1,18 @@ +{ + "name": "yao-meta-skill", + "version": "1.1.0", + "owner": "Yao Team", + "updated_at": "2026-03-31", + "review_cadence": "quarterly", + "target_platforms": [ + "openai", + "claude", + "generic", + "agent-skills-compatible" + ], + "factory_components": [ + "templates", + "references", + "scripts" + ] +} diff --git a/references/client-compatibility.md b/references/client-compatibility.md new file mode 100644 index 00000000..df6a7fe6 --- /dev/null +++ b/references/client-compatibility.md @@ -0,0 +1,34 @@ +# Client Compatibility + +This skill package uses a neutral source-of-truth file: + +- `agents/interface.yaml` + +That file is the canonical metadata source for display name, short description, default prompt, and adapter targets. + +## Compatibility Strategy + +Use a two-layer model: + +1. **Canonical source** + - Keep brand-neutral metadata in `agents/interface.yaml`. + - Keep behavior in `SKILL.md`, `references/`, and `scripts/`. + +2. **Adapter outputs** + - Generate client-specific metadata only when exporting or packaging. + - Do not keep vendor-specific metadata files in the source tree unless a client strictly requires them. + +## Supported Targets + +The current adapter flow is designed for: + +- OpenAI-compatible clients +- Claude-compatible clients +- generic Agent Skills clients + +## Compatibility Rules + +- Keep `SKILL.md` as the primary behavior definition. +- Keep client metadata minimal and presentation-focused. +- Avoid putting client-specific logic in the main workflow. +- Prefer packaging-time conversion over source-tree duplication. diff --git a/references/comparative-analysis.md b/references/comparative-analysis.md new file mode 100644 index 00000000..2df3935e --- /dev/null +++ b/references/comparative-analysis.md @@ -0,0 +1,113 @@ +# Comparative Analysis + +This reference distills four meta-skill archetypes into one design system. + +## Shared Logic + +All four approaches converge on the same model: + +1. A skill is a folderized capability package, not a prompt snippet. +2. Frontmatter description is the trigger surface. +3. Long instructions should be split by progressive disclosure. +4. Skills are most valuable for repeated, multi-step, tool-using workflows. +5. Skills become more valuable when they are portable, maintainable, and shareable. + +## Structure-First Creator + +Primary strengths: + +- clear structure and boundary discipline +- strong context-efficiency mindset +- good guidance on progressive disclosure +- pragmatic skeleton for authoring without overbuilding + +Primary gaps: + +- lighter on trigger benchmarking than the eval-first archetype +- lighter on distribution and registry than the factory archetype +- less opinionated on organizational operations beyond good authoring practice + +Use it for: + +- canonical package structure +- concise writing standard +- deciding what belongs in `SKILL.md` vs `references/` + +## Eval-First Creator + +Primary strengths: + +- eval-first mindset +- explicit trigger optimization +- positive and negative prompt testing +- iterative improvement loop with benchmark thinking + +Primary gaps: + +- heavier process cost +- more suitable for high-value skills than quick one-offs +- some workflow assumptions are tied to a specific runtime style + +Use it for: + +- trigger eval design +- benchmark loops +- systematic improvement of important skills + +## Template-First Scaffold + +Primary strengths: + +- fast onboarding +- clean scaffold +- easy explanation of required fields +- good for normalizing team authoring habits + +Primary gaps: + +- shallow evaluation model +- limited operations guidance after scaffolding +- more a template than a full skill engineering system + +Use it for: + +- starter layout +- contributor onboarding +- low-friction authoring workflow + +## Factory-First Builder + +Primary strengths: + +- strongest productization instinct +- cross-platform packaging and export thinking +- validation, security, staleness, and registry mindset +- closest to a skill factory instead of a skill template + +Primary gaps: + +- heavier system and maintenance cost +- portability requires adaptation layers in practice +- may be excessive for small personal skills + +Use it for: + +- distribution and registry model +- packaging lifecycle +- maintenance and governance thinking + +## Yao Synthesis + +The right synthesis is: + +- **structure-first for clarity** +- **eval-first for reliability** +- **template-first for onboarding** +- **factory-first for operations and scale** + +That combination yields a meta-skill that is: + +- lightweight enough to use often +- rigorous enough for important skills +- structured enough for team adoption +- operational enough for long-term reuse diff --git a/references/design-rubric.md b/references/design-rubric.md new file mode 100644 index 00000000..bce00748 --- /dev/null +++ b/references/design-rubric.md @@ -0,0 +1,68 @@ +# Meta-Skill Rubric + +Score each generated skill from 1 to 5 on each dimension. + +## 1. Trigger Clarity + +Questions: + +- Does the description say what the skill does? +- Does it say when to use it? +- Does it include realistic trigger phrases and adjacent cases? +- Is it likely to reduce under-triggering? + +## 2. Boundary Clarity + +Questions: + +- Is the skill solving one coherent capability family? +- Are unrelated tasks excluded? +- Are variants separated cleanly into references or sibling skills? + +## 3. Context Efficiency + +Questions: + +- Is `SKILL.md` lean? +- Are details pushed into `references/`? +- Are scripts used where prose would be brittle or repetitive? + +## 4. Execution Reliability + +Questions: + +- Are the workflow steps actionable? +- Are critical constraints explicit? +- Are deterministic steps captured in scripts or templates? +- Is there a clear failure-handling path? + +## 5. Evaluation Quality + +Questions: + +- Are there realistic usage prompts? +- Are there trigger positives and negatives for important skills? +- Is there a way to compare revisions? + +## 6. Reuse And Maintenance + +Questions: + +- Can another teammate discover and reuse the skill? +- Is `agents/interface.yaml` aligned with the package? +- Are ownership and future iteration obvious? + +## 7. Portability + +Questions: + +- Does the package stay close to the open format? +- Are tool-specific assumptions minimized or isolated? +- Can the skill be adapted to another compatible client without rewrite? + +## Interpretation + +- `30+`: production-ready +- `24-29`: solid but improve weak dimensions +- `18-23`: usable draft, not yet robust +- `<18`: redesign the boundary or trigger strategy diff --git a/references/eval-playbook.md b/references/eval-playbook.md new file mode 100644 index 00000000..b5fa83d7 --- /dev/null +++ b/references/eval-playbook.md @@ -0,0 +1,69 @@ +# Trigger And Eval Playbook + +Use this playbook for skills that matter enough to test. + +## A. Trigger Evaluation + +Create three prompt buckets: + +### 1. Should Trigger + +Prompts that clearly need the skill. + +Goal: + +- verify recall + +### 2. Should Not Trigger + +Prompts that are clearly outside the skill boundary. + +Goal: + +- verify precision + +### 3. Near Neighbors + +Prompts that look similar but should use another skill or no skill. + +Goal: + +- catch false positives and ambiguous routing + +## B. Execution Evaluation + +For each important use case, create 1 to 3 realistic prompts with: + +- user-like phrasing +- representative inputs or file types +- expected output description +- key checks + +## C. Revision Loop + +When a skill underperforms: + +1. Fix boundary or description problems before adding more body text. +2. Move brittle logic into scripts or templates. +3. Split reference content if `SKILL.md` becomes bloated. +4. Re-run the same eval set before expanding scope. + +## D. Minimum QA By Skill Tier + +### Personal skill + +- 2 realistic prompts +- manual review + +### Team skill + +- 3 to 5 realistic prompts +- trigger positives and negatives +- one revision loop + +### Infrastructure or meta-skill + +- 5+ execution prompts +- trigger positives, negatives, and near neighbors +- benchmark notes across revisions +- ownership and drift review diff --git a/references/skill-template.md b/references/skill-template.md new file mode 100644 index 00000000..de87b67f --- /dev/null +++ b/references/skill-template.md @@ -0,0 +1,74 @@ +# Skill Template + +Use this skeleton when generating a new skill. + +```markdown +--- +name: skill-name +description: Describe what the skill does and when to use it. Include realistic trigger phrases, task types, artifacts, or adjacent scenarios that should activate it. +metadata: + author: Your team +--- + +# Skill Title + +One-sentence summary of the capability. + +## When To Use This Skill + +- Trigger scenario 1 +- Trigger scenario 2 +- Trigger scenario 3 + +## Workflow + +### 1. Understand the request + +- Identify the actual job to be done +- Check assumptions and inputs + +### 2. Choose the path + +- Use path A when ... +- Use path B when ... + +### 3. Execute + +- Step 1 +- Step 2 +- Step 3 + +### 4. Validate + +- Minimum acceptance checks + +## Reference Map + +- Read `[topic-a](references/topic-a.md)` when ... +- Read `[topic-b](references/topic-b.md)` when ... +``` + +## Optional Folders + +Create only when justified: + +- `agents/interface.yaml` +- `references/` +- `scripts/` +- `assets/` +- `evals/` + +## agents/interface.yaml Template + +```yaml +interface: + display_name: "Human Friendly Name" + short_description: "Short capability summary" + default_prompt: "Use $skill-name to ..." +compatibility: + canonical_format: "agent-skills" + adapter_targets: + - "openai" + - "claude" + - "generic" +``` diff --git a/references/skill_design_guidelines.md b/references/skill_design_guidelines.md new file mode 100644 index 00000000..0a4f5d07 --- /dev/null +++ b/references/skill_design_guidelines.md @@ -0,0 +1,67 @@ +# Skill Design Guidelines + +This document captures the operating rules for `yao-meta-skill`. + +## 1. Treat Skill Creation As Product Design + +A generated skill should answer these questions clearly: + +- What exact recurring job does this package improve? +- Who is the intended user or agent? +- What is in scope and out of scope? +- What should be stable across runs? +- What is expected to evolve over time? + +## 2. Separate Three Kinds Of Content + +Put each kind of content in the right place: + +- `SKILL.md`: core workflow, trigger surface, decision rules +- `references/`: detailed domain material, examples, schemas, policy docs +- `scripts/`: deterministic or brittle operations + +If you mix all three into `SKILL.md`, quality and maintainability drop fast. + +## 3. Use A Trigger Matrix + +Every important skill should have a trigger matrix: + +- positive prompts +- clear negatives +- near neighbors + +The goal is not just "can it trigger", but "does it trigger at the right boundary". + +## 4. Keep The Body Small On Purpose + +Do not optimize only for completeness. + +Optimize for: + +- low context cost +- clear branch selection +- discoverable references +- safe defaults + +## 5. Add Lifecycle Metadata Only When It Helps + +For personal or disposable skills, extra metadata can be noise. + +For shared skills, a `manifest.json` is useful for: + +- owner +- version +- updated_at +- target platforms +- review cadence + +## 6. Prefer Progressive Industrialization + +Move through these stages: + +1. Working draft +2. Trigger-hardened draft +3. Reusable team skill +4. Governed skill asset + +Do not start every skill at stage 4. diff --git a/scripts/context_sizer.py b/scripts/context_sizer.py new file mode 100644 index 00000000..89b9b831 --- /dev/null +++ b/scripts/context_sizer.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python3 +import argparse +import json +from pathlib import Path + + +TEXT_EXTS = {".md", ".txt", ".yaml", ".yml", ".json", ".py", ".sh", ".js", ".ts"} + + +def estimate_tokens(text: str) -> int: + # Fast heuristic suitable for local gating. + return max(1, len(text) // 4) + + +def read_text(path: Path) -> str: + try: + return path.read_text(encoding="utf-8") + except UnicodeDecodeError: + return path.read_text(encoding="utf-8", errors="ignore") + + +def classify(path: Path) -> str: + parts = set(path.parts) + if path.name == "SKILL.md": + return "skill_body" + if "references" in parts: + return "reference" + if "scripts" in parts: + return "script" + if "assets" in parts: + return "asset" + if path.suffix in TEXT_EXTS: + return "other_text" + return "binary_or_other" + + +def summarize(skill_dir: Path) -> dict: + files = [] + total_tokens = 0 + initial_tokens = 0 + for path in sorted(skill_dir.rglob("*")): + if not path.is_file(): + continue + kind = classify(path) + if kind in {"binary_or_other", "asset"} and path.suffix not in TEXT_EXTS: + size = path.stat().st_size + files.append({"path": str(path.relative_to(skill_dir)), "kind": kind, "bytes": size}) + continue + text = read_text(path) + tokens = estimate_tokens(text) + record = { + "path": str(path.relative_to(skill_dir)), + "kind": kind, + "chars": len(text), + "estimated_tokens": tokens, + } + files.append(record) + total_tokens += tokens + if kind in {"skill_body", "other_text"}: + initial_tokens += tokens + return { + "skill_dir": str(skill_dir), + "estimated_initial_load_tokens": initial_tokens, + "estimated_total_text_tokens": total_tokens, + "warning": initial_tokens > 2000, + "files": files, + } + + +def main() -> None: + parser = argparse.ArgumentParser(description="Estimate context size for a skill package.") + parser.add_argument("skill_dir", help="Path to the skill directory") + parser.add_argument("--json", action="store_true", help="Emit machine-readable JSON") + args = parser.parse_args() + + report = summarize(Path(args.skill_dir).resolve()) + if args.json: + print(json.dumps(report, ensure_ascii=False, indent=2)) + return + + print(f"Skill: {report['skill_dir']}") + print(f"Estimated initial-load tokens: {report['estimated_initial_load_tokens']}") + print(f"Estimated total text tokens: {report['estimated_total_text_tokens']}") + print(f"Initial-load warning (>2000): {'YES' if report['warning'] else 'NO'}") + print("") + for file in report["files"]: + if "estimated_tokens" in file: + print(f"{file['kind']:12} {file['estimated_tokens']:6}t {file['path']}") + else: + print(f"{file['kind']:12} {file['bytes']:6}b {file['path']}") + + +if __name__ == "__main__": + main() diff --git a/scripts/cross_packager.py b/scripts/cross_packager.py new file mode 100644 index 00000000..af3396fc --- /dev/null +++ b/scripts/cross_packager.py @@ -0,0 +1,171 @@ +#!/usr/bin/env python3 +import argparse +import json +import shutil +import zipfile +from pathlib import Path + + +def read_simple_yaml(path: Path) -> dict: + lines = path.read_text(encoding="utf-8").splitlines() + data: dict = {} + stack: list[tuple[int, dict]] = [(0, data)] + for raw_line in lines: + if not raw_line.strip() or raw_line.lstrip().startswith("#"): + continue + indent = len(raw_line) - len(raw_line.lstrip(" ")) + line = raw_line.strip() + while len(stack) > 1 and indent <= stack[-1][0]: + stack.pop() + parent = stack[-1][1] + if line.startswith("- "): + item = line[2:].strip().strip("'\"") + existing = parent.setdefault("__list__", []) + existing.append(item) + continue + if ":" not in line: + continue + key, value = line.split(":", 1) + key = key.strip() + value = value.strip() + if value == "": + child: dict = {} + parent[key] = child + stack.append((indent, child)) + else: + parent[key] = value.strip("'\"") + return data + + +def read_frontmatter(skill_md: Path) -> dict: + text = skill_md.read_text(encoding="utf-8") + if not text.startswith("---"): + return {} + parts = text.split("---", 2) + if len(parts) < 3: + return {} + data = {} + for line in parts[1].splitlines(): + if ":" not in line: + continue + key, value = line.split(":", 1) + data[key.strip()] = value.strip().strip("'\"") + return data + + +def read_interface(skill_dir: Path) -> dict: + path = skill_dir / "agents" / "interface.yaml" + if not path.exists(): + return {} + raw = read_simple_yaml(path) + compatibility = raw.get("compatibility", {}) + targets = compatibility.get("adapter_targets", {}) + if isinstance(targets, dict) and "__list__" in targets: + compatibility["adapter_targets"] = targets["__list__"] + raw["compatibility"] = compatibility + return raw + + +def build_manifest(skill_dir: Path, platform: str) -> dict: + frontmatter = read_frontmatter(skill_dir / "SKILL.md") + interface = read_interface(skill_dir).get("interface", {}) + compatibility = read_interface(skill_dir).get("compatibility", {}) + return { + "name": frontmatter.get("name", skill_dir.name), + "description": frontmatter.get("description", ""), + "version": frontmatter.get("version", "1.0.0"), + "platform": platform, + "skill_root": skill_dir.name, + "display_name": interface.get("display_name", skill_dir.name), + "short_description": interface.get("short_description", ""), + "default_prompt": interface.get("default_prompt", ""), + "canonical_metadata": "agents/interface.yaml", + "adapter_targets": compatibility.get("adapter_targets", []), + } + + +def write_yaml_like(path: Path, payload: dict) -> None: + interface = payload.get("interface", {}) + lines = ["interface:"] + for key in ("display_name", "short_description", "default_prompt"): + value = interface.get(key, "") + lines.append(f' {key}: "{value}"') + path.write_text("\n".join(lines) + "\n", encoding="utf-8") + + +def write_adapter(skill_dir: Path, out_dir: Path, platform: str) -> Path: + target_dir = out_dir / "targets" / platform + target_dir.mkdir(parents=True, exist_ok=True) + payload = build_manifest(skill_dir, platform) + if platform == "openai": + meta_dir = target_dir / "agents" + meta_dir.mkdir(parents=True, exist_ok=True) + write_yaml_like( + meta_dir / "openai.yaml", + { + "interface": { + "display_name": payload["display_name"], + "short_description": payload["short_description"], + "default_prompt": payload["default_prompt"], + } + }, + ) + payload["install_hint"] = f"Use the packaged skill and include targets/openai/agents/openai.yaml when the client expects OpenAI-style interface metadata." + elif platform == "claude": + notes = target_dir / "README.md" + notes.write_text( + f"# Claude-Compatible Package\n\nUse `{skill_dir.name}` with its neutral source files. This target does not require vendor metadata by default.\n", + encoding="utf-8", + ) + payload["install_hint"] = f"Use the packaged skill directly; this target relies on SKILL.md and optional neutral metadata." + else: + payload["install_hint"] = f"Use {skill_dir.name} as an Agent Skills compatible package." + path = target_dir / "adapter.json" + path.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8") + return path + + +def make_zip(skill_dir: Path, out_dir: Path) -> Path: + zip_path = out_dir / f"{skill_dir.name}.zip" + with zipfile.ZipFile(zip_path, "w", compression=zipfile.ZIP_DEFLATED) as zf: + for path in skill_dir.rglob("*"): + if path.is_file(): + zf.write(path, arcname=str(path.relative_to(skill_dir.parent))) + return zip_path + + +def copy_manifest(skill_dir: Path, out_dir: Path) -> Path: + manifest_path = out_dir / "manifest.json" + manifest_path.write_text( + json.dumps(build_manifest(skill_dir, "generic"), ensure_ascii=False, indent=2), + encoding="utf-8", + ) + return manifest_path + + +def main() -> None: + parser = argparse.ArgumentParser(description="Generate lightweight cross-platform packaging artifacts.") + parser.add_argument("skill_dir", help="Path to the skill directory") + parser.add_argument("--platform", action="append", default=[], help="Target platform: openai, claude, generic") + parser.add_argument("--output-dir", default="dist", help="Output directory") + parser.add_argument("--zip", action="store_true", help="Create a zip package") + args = parser.parse_args() + + skill_dir = Path(args.skill_dir).resolve() + out_dir = Path(args.output_dir).resolve() + if out_dir.exists(): + shutil.rmtree(out_dir) + out_dir.mkdir(parents=True) + + manifest = copy_manifest(skill_dir, out_dir) + generated = [str(manifest)] + for platform in (args.platform or ["generic"]): + generated.append(str(write_adapter(skill_dir, out_dir, platform))) + if args.zip: + generated.append(str(make_zip(skill_dir, out_dir))) + + print(json.dumps({"output_dir": str(out_dir), "generated": generated}, ensure_ascii=False, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/scripts/trigger_eval.py b/scripts/trigger_eval.py new file mode 100644 index 00000000..cf2e0c2c --- /dev/null +++ b/scripts/trigger_eval.py @@ -0,0 +1,94 @@ +#!/usr/bin/env python3 +import argparse +import json +import re +from pathlib import Path + + +WORD_RE = re.compile(r"[a-zA-Z0-9][a-zA-Z0-9_-]*") + + +def words(text: str) -> set[str]: + return {w.lower() for w in WORD_RE.findall(text)} + + +def load_cases(path: Path) -> dict: + return json.loads(path.read_text(encoding="utf-8")) + + +def extract_description(text: str) -> str: + if not text.startswith("---"): + return text + parts = text.split("---", 2) + if len(parts) < 3: + return text + frontmatter = parts[1].splitlines() + for line in frontmatter: + if line.strip().startswith("description:"): + return line.split(":", 1)[1].strip().strip("'\"") + return text + + +def score_prompt(description_words: set[str], prompt: str) -> float: + prompt_words = words(prompt) + if not prompt_words: + return 0.0 + overlap = description_words & prompt_words + return len(overlap) / len(prompt_words) + + +def evaluate(description: str, cases: dict, threshold: float) -> dict: + desc_words = words(description) + results = {"should_trigger": [], "should_not_trigger": []} + fp = 0 + fn = 0 + + for bucket in ("should_trigger", "should_not_trigger"): + for prompt in cases.get(bucket, []): + score = score_prompt(desc_words, prompt) + predicted = score >= threshold + expected = bucket == "should_trigger" + passed = predicted == expected + if not passed and expected: + fn += 1 + if not passed and not expected: + fp += 1 + results[bucket].append( + { + "prompt": prompt, + "score": round(score, 3), + "predicted_trigger": predicted, + "passed": passed, + } + ) + + return { + "threshold": threshold, + "false_positives": fp, + "false_negatives": fn, + "results": results, + } + + +def main() -> None: + parser = argparse.ArgumentParser(description="Heuristic trigger quality evaluator.") + parser.add_argument("--description", help="Description string to evaluate") + parser.add_argument("--description-file", help="Read description text from file") + parser.add_argument("--cases", required=True, help="JSON file with should_trigger and should_not_trigger arrays") + parser.add_argument("--threshold", type=float, default=0.18, help="Token overlap threshold") + args = parser.parse_args() + + description = args.description + if args.description_file: + description = extract_description(Path(args.description_file).read_text(encoding="utf-8")) + if not description: + raise SystemExit("Provide --description or --description-file") + + report = evaluate(description, load_cases(Path(args.cases)), args.threshold) + print(json.dumps(report, ensure_ascii=False, indent=2)) + if report["false_positives"] > 2: + raise SystemExit(2) + + +if __name__ == "__main__": + main() diff --git a/templates/basic_skill.md.j2 b/templates/basic_skill.md.j2 new file mode 100644 index 00000000..1624f4e4 --- /dev/null +++ b/templates/basic_skill.md.j2 @@ -0,0 +1,42 @@ +--- +name: {{ skill_name }} +description: {{ description }} +metadata: + author: {{ author }} +--- + +# {{ title }} + +{{ summary }} + +## When To Use This Skill + +{% for item in when_to_use %} +- {{ item }} +{% endfor %} + +## Workflow + +### 1. Understand the request + +{% for item in understand_steps %} +- {{ item }} +{% endfor %} + +### 2. Execute + +{% for item in execute_steps %} +- {{ item }} +{% endfor %} + +### 3. Validate + +{% for item in validate_steps %} +- {{ item }} +{% endfor %} + +## Reference Map + +{% for item in reference_map %} +- {{ item }} +{% endfor %} diff --git a/templates/complex_skill.md.j2 b/templates/complex_skill.md.j2 new file mode 100644 index 00000000..1f1ab998 --- /dev/null +++ b/templates/complex_skill.md.j2 @@ -0,0 +1,61 @@ +--- +name: {{ skill_name }} +description: {{ description }} +metadata: + author: {{ author }} + version: {{ version }} +--- + +# {{ title }} + +{{ summary }} + +## When To Use This Skill + +{% for item in when_to_use %} +- {{ item }} +{% endfor %} + +## Operating Contract + +- Primary output: {{ primary_output }} +- Main tools or systems: {{ tools_summary }} +- Failure tolerance: {{ failure_tolerance }} + +## Workflow + +### 1. Intake + +{% for item in intake_steps %} +- {{ item }} +{% endfor %} + +### 2. Choose the path + +{% for item in routing_steps %} +- {{ item }} +{% endfor %} + +### 3. Execute + +{% for item in execute_steps %} +- {{ item }} +{% endfor %} + +### 4. Validate + +{% for item in validate_steps %} +- {{ item }} +{% endfor %} + +## Bundled Resources + +- Read `references/` when domain-specific detail or long examples are needed. +- Use `scripts/` when deterministic or repetitive steps should not be re-authored in prose. +- Use `assets/` for templates or output artifacts, not instructions. + +## Reference Map + +{% for item in reference_map %} +- {{ item }} +{% endfor %}