--- title: Frontier-model skill modernization methodology date: 2026-06-10 category: skill-design module: compound-engineering problem_type: design_pattern component: development_workflow severity: medium applies_when: - "Reviewing or modernizing an existing skill against current frontier-model prompting guidance" - "Deciding whether to compress enumerated judgment examples or keep protocol text verbatim in SKILL.md" - "Designing model-tier vocabulary and degradation rules for sub-agent fleets" - "Extracting load-bearing skill content into reference files with inline load stubs" - "Verifying skill prose changes with injected-subagent evals instead of cached plugin dispatch" tags: - skill-design - prompting-guidance - frontier-models - model-tiers - context-window - reference-extraction - load-reliability - subagent-evals --- # Frontier-Model Skill Modernization Methodology ## Context We modernized the `ce-ideate` skill (a ~13-agent ideation orchestrator) against the Claude Fable 5 prompting guide and the Claude prompting best-practices doc, then verified the result with a transcript-graded eval. The review surfaced a repeatable methodology for bringing any orchestration-heavy skill up to frontier-model standards: the skill went from 424 to 372 lines (-12%) while *gaining* capability (model tiering, file-based data flow, ceiling-raising dispatch mechanics), and the eval passed 6/8 mechanical assertions with 0 failures. This doc generalizes that sequence so the next skill review (ce-plan, ce-brainstorm, ce-code-review, ...) starts from the playbook instead of rediscovering it. --- ## Guidance Run the steps in order. Each has a named test or rule — apply the test, don't improvise the judgment. ### 1. Audit: classify every prescriptive block as PROTOCOL or JUDGMENT Read the skill top to bottom and tag each instruction block: - **PROTOCOL** — *what to do*: output-format resolution order, cache file shapes, scratch paths, read budgets, agent counts, checkpoint mechanics. Unambiguous, costs a strong model nothing, and the workflow mechanically breaks without it. **Keep at full prescription.** (`git-workflow-skills-need-explicit-state-machines.md` is the canonical example of protocol content that regressed whenever it was softened to prose.) - **JUDGMENT** — *how to think*: enumerated example lists, multi-row sample classification tables, multi-paragraph elaborations of a single principle. A frontier model already has the capability; the prescription only narrows it. **Candidate for compression.** The test: *would a strong model behave correctly given only the principle?* If yes, it's JUDGMENT. If the skill produces wrong file paths, wrong agent counts, or broken handoffs without it, it's PROTOCOL. This refines (not replaces) the three-level prescription model in the plugin's AGENTS.md Skill Design Principles — hard rules / strong guidance / trust. Protocol maps to hard rules; the protocol-vs-judgment test decides which of the other two levels a block deserves. ### 2. Establish the orchestrator-model floor before cutting anything Pruning JUDGMENT prescription is only safe if the skill realistically runs on frontier models. State the floor argument explicitly in your review notes. For ce-ideate: anyone launching a 13-agent workflow on any platform picks a frontier model, so the floor holds. If a skill plausibly runs on small models (e.g., a lightweight formatting skill), keep more scaffolding. The Fable guide's warning is the anchor: "Skills developed for prior models are often too prescriptive for Claude Fable 5 and can degrade output quality." ### 3. Prune: compress each JUDGMENT block to principle + ONE contrast pair The compression rule: replace the enumeration with the underlying principle and a single minimal contrast pair that makes the boundary unmistakable. Example from ce-ideate: a list of vague-phrase examples became "`browser sniff` is identifiable, `quick wins` is not — vagueness is about referent, not length." One pair carries the distinction; seven rows of table did not carry more. Also deduplicate: triplicated boilerplate becomes one full copy + pointers. This matches the broader principle: prefer principles + a named test over enumerated specifics — specifics drift. (auto memory [claude]) ### 4. Tier: define cost tiers semantically, once, and reference by name Define three tiers in one place in SKILL.md; everywhere else refers to the tier name, never a model name: - **Extraction tier** — cheapest capable model. Scouts, retrieval, quoting. - **Generation tier** — mid-tier model. Evidence-driven generation, mechanical verification. - **Ceiling tier** — *inherit the orchestrator's model by omitting the model parameter*. Never name a model for the ceiling. Rules that travel with the tiers: - Per-platform model hints follow the plugin's existing platform-enumeration pattern (the same shape used for blocking-question tools); never pin other vendors' model names in pass-through skill content — naming drifts faster than release cadence. Note: the converter does propagate `model:` params to all targets (see `best-practices/ce-pipeline-end-to-end-learnings.md`), so tier hints are not Claude-only decoration. - **Degradation rule**: when the platform's subagent primitive lacks per-agent model selection, dispatch everything on the inherited model and keep read budgets/output caps — cost control comes from structure, not tiering. Write this rule into the skill; it fired correctly in our eval (the harness had no nested dispatch). - **Architecture principle**: separate evidence-gathering (cheap extraction scouts producing quote+pointer dossiers) from ceiling reasoning (strong model only at choke points: ceiling framing, cross-cutting synthesis, final arbitration). This is cheaper *and* better grounded than a uniform fleet fed a thin summary. ### 5. Optimize context: extract only conditional/late content, and move bulk data to files Two independent levers: - **Reference extraction pays only for CONDITIONAL or LATE-SEQUENCE content.** Early unconditional content gains nothing — it would be read at start and carried anyway, plus a read round-trip. The test: *how many turns of other work happen before this content executes, and might it never execute?* ce-ideate's Phase 2 (~100 lines, ~22% of the file, runs after 5-8 turns of grounding) qualified; Phase 0 gating did not. - **Data flows usually dominate prose.** Measure both: 5 scouts × 150-line dossiers ≈ 10k tokens carried every subsequent turn if returned inline — more than the entire SKILL.md (~6k). Fix: subagents write outputs to scratch files (`/tmp/compound-engineering///...`), return a 3-5-line gist; downstream agents receive paths and read the files themselves. This extends the established path-passing pattern (`skill-design/pass-paths-not-content-to-subagents.md`) with the gist refinement: the orchestrator keeps just enough orientation to route, never the bulk. ### 6. Load-stub design: make extracted references information-asymmetric A soft pointer ("see references/X.md for details") gets skipped. When extracting load-bearing content, the inline stub must satisfy all five properties: 1. **Load-instruction-only** — no spec, no contract, nothing to improvise from. Converts "should load" into "cannot proceed without loading." 2. **Names exactly what the reference contains** and states those details appear nowhere in the main body. 3. **Names the failure mode of skipping** in the skill's own terms (e.g., "improvising produces unverifiable candidates — the precise failure this skill exists to prevent"). 4. **Closes inline-information leaks** — any number or detail that remains inline for other reasons gets explicitly disclaimed ("the fleet counts in Phase 0.6 are cost transparency, not the dispatch spec"). 5. **Pre-empts rationalizations** ("'Quickly' means smaller volume targets, not skipping the reference"). Defense in depth: anchor downstream phases on *different* reference files (rejection criteria, section contract) so a skipped load fails visibly, not silently. This is the complement of `skill-design/post-menu-routing-belongs-inline.md`: inline the content when it is always-on; use an information-asymmetric stub when it is genuinely conditional or late-sequence but must load when its branch fires. ### 7. Eval verification: fresh subagent, mechanical transcript grading - **Bypass the cache.** Plugin skill/agent definitions cache at session start; typed invocation tests the stale copy. Instead, spawn a fresh subagent told to read the skill source from disk and follow it. - **Grade from the transcript, not the self-report.** Parse the JSONL into a tool-call timeline and assert mechanically: Read-event ordering against generation checkpoints (e.g., the extracted-reference Read landing after scout writes and before the candidates checkpoint); zero orchestrator Reads of bulk data files; filesystem artifacts present with correct names and the full section contract. - **Know the harness limits and record them.** An eval subagent without nested dispatch *cannot* verify dispatch payload shape or fleet tiering — mark those assertions "not testable," don't fudge them. It *does* verify load ordering, file contracts, volume/format overrides, and degradation-rule behavior. Closing the gap requires a main-session run. - Errors encountered while following the skill during the eval are findings about the skill, not noise. (auto memory [claude]) ### 8. Ceiling mechanics: explicitly request above-and-beyond behavior in dispatches Floor-guarding (basis requirements, rejection criteria) prevents bad output; it does not produce ambitious output. From the best-practices doc: - **Ambition charter**, included verbatim in every generation dispatch: intent framing (why the output matters), warm-up framing ("your first few ideas are warm-up; keep only those that earn their place after the non-obvious ones exist"), and an anti-genericness test ("if it would appear in a generic listicle, sharpen or drop"). - **Fresh-context verifiers over self-critique** (per the Fable guide): the orchestrator grading its own synthesis is anchored; a verifier that never saw generation, prompted to *refute*, is not. - **Dispatch payload structure**: XML tags (` `); longform shared material first, task last (documented long-context gain); byte-identical shared prefix across parallel dispatches for prompt-cache reuse; constraint-vs-background made mechanical by tags rather than prose. --- ## Why This Matters Skills written for prior model generations accumulate two opposite debts simultaneously: too much JUDGMENT prescription (which the Fable guide warns actively degrades frontier-model output) and too little PROTOCOL infrastructure for cost, context, and verification. A naive "shorten it" pass cuts the wrong things; a naive "harden it" pass bloats the wrong things. The PROTOCOL/JUDGMENT split plus the ordered sequence resolves the tension: prune where the model is strong, prescribe where the workflow is mechanical. Measured outcomes from the ce-ideate application: - SKILL.md: 424 → 372 lines (-12%) while adding model tiering, file-based dossier flow, the information-asymmetric stub, and the ambition charter. ~16 lines recovered from three judgment-prescription cuts alone, with no behavior loss in the verification run. - Context math: inlined dossiers would have cost ~10k tokens carried every turn — more than the whole SKILL.md (~6k). The file+gist pattern removed that entirely from the orchestrator's window. - Eval: 6/8 mechanical assertions passed, 2 correctly reported untestable (nested-dispatch assertions in a dispatch-less harness), 0 failures. The degradation rule fired as designed. Degraded inline run: 14 minutes, 208k tokens. - Cost architecture: cheap extraction scouts feeding quote+pointer dossiers to ceiling-tier choke points was both cheaper and better grounded than a uniform inherited-model fleet fed a thin summary. --- ## When to Apply Run this sequence when reviewing a skill that matches one or more of: - **Multi-agent orchestration skills** — anything dispatching subagent fleets (the tiering, file-flow, and dispatch-payload steps only matter here). - **Skills over ~300 lines** — large enough that conditional/late-sequence extraction and judgment pruning have measurable payoff. - **Skills written before frontier models** (or before the current Fable guide) — likely over-prescribed on judgment, under-built on protocol. - **Skills inlining bulk data into dispatch prompts or return values** — any place subagent output re-enters the orchestrator's window as content rather than a path. - **Skills with soft "see reference" pointers guarding load-bearing content** — apply step 6 even without the rest. Skip or scale down when: the skill is short and unconditional (extraction won't pay), or it plausibly runs on non-frontier models (the step-2 floor fails — keep the scaffolding). Always pair structural changes with the step-7 eval; never ship on the agent's self-report. --- ## Examples **1. Judgment enumeration → principle + one contrast pair** Before (enumerated list of vague phrases plus a 7-row sample classification table): ``` Vague subjects include: "quick wins", "low-hanging fruit", "improvements", "polish", "cleanup", "things to fix", ... [+ 7-row table classifying sample subjects as vague/identifiable] ``` After: ``` A subject is workable when it names an identifiable referent: `browser sniff` is identifiable, `quick wins` is not — vagueness is about referent, not length. ``` **2. Inline bulk data → file + gist** Before (scout returns its full dossier; orchestrator carries it forever): ``` Return your complete evidence dossier (~150 lines of quotes + pointers) in your final message. ``` After: ``` Write your dossier to /tmp/compound-engineering///evidence-.md. Return only a 3-5 line gist plus the file path. Downstream agents read the file themselves; the orchestrator never does. ``` **3. Soft pointer → information-asymmetric stub** Before: ``` Phase 2: Divergent ideation. See references/divergent-ideation.md for details on the fleet structure. Dispatch the agents and collect candidates. ``` After: ``` Phase 2: Read references/divergent-ideation.md now. It contains the fleet spec, per-agent dispatch contract, and volume targets — none of which appear in this main body. Dispatch prompts cannot be correctly constructed without it; improvising them produces unverifiable candidates — the precise failure this skill exists to prevent. The fleet counts in Phase 0.6 are cost transparency, not the dispatch spec. "Quickly" means smaller volume targets, not skipping the reference. ``` The before version leaves enough inline (phase name, "dispatch the agents") to improvise from; the after version makes proceeding without the read impossible, names the skip-failure, closes the leak, and pre-empts the "we're in a hurry" rationalization. --- ## Related - `docs/solutions/skill-design/pass-paths-not-content-to-subagents.md` — established precedent for path-passing to subagents; step 5 extends it with the gist refinement. - `docs/solutions/skill-design/post-menu-routing-belongs-inline.md` — the complementary lever for the same load-reliability failure: inline always-on content; load-stub conditional content (step 6). - `docs/solutions/skill-design/git-workflow-skills-need-explicit-state-machines.md` — canonical example of PROTOCOL content that must keep full prescription (step 1). - `docs/solutions/skill-design/script-first-skill-architecture.md` — complementary token-optimization pattern (bundled scripts instead of model-context work). - `docs/solutions/skill-design/safe-auto-rubric-calibration.md` — earlier eval-methodology precedent (fixture-based grading, variance awareness) consistent with step 7. - `docs/solutions/skill-design/paired-old-vs-new-injection-skill-evals.md` — sharpens step 7's fresh-subagent grading into a controlled old-vs-new blind A/B that separates demonstrated improvement from no-regression. - `docs/solutions/best-practices/ce-pipeline-end-to-end-learnings.md` — evidence that `model:` params propagate to all conversion targets (step 4). - Plugin `AGENTS.md` → Skill Design Principles — the prescription-calibration framework this methodology refines; and the conditional/late-sequence extraction rule step 5 operationalizes. - GitHub issues #714 and #374 — historical reference-load failures in the same family step 6 addresses.