chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,117 @@
|
||||
---
|
||||
date: 2026-02-14
|
||||
topic: copilot-converter-target
|
||||
---
|
||||
|
||||
# Add GitHub Copilot Converter Target
|
||||
|
||||
## What We're Building
|
||||
|
||||
A new converter target that transforms the compound-engineering Claude Code plugin into GitHub Copilot's native format. This follows the same established pattern as the existing converters (Cursor, Codex, OpenCode, Droid, Pi) and outputs files that Copilot can consume directly from `.github/` (repo-level) or `~/.copilot/` (user-wide).
|
||||
|
||||
Copilot's customization system (as of early 2026) supports: custom agents (`.agent.md`), agent skills (`SKILL.md`), prompt files (`.prompt.md`), custom instructions (`copilot-instructions.md`), and MCP servers (via repo settings).
|
||||
|
||||
## Why This Approach
|
||||
|
||||
The repository already has a robust multi-target converter infrastructure with a consistent `TargetHandler` pattern. Adding Copilot as a new target follows this proven pattern rather than inventing something new. Copilot's format is close enough to Claude Code's that the conversion is straightforward, and the SKILL.md format is already cross-compatible.
|
||||
|
||||
### Approaches Considered
|
||||
|
||||
1. **Full converter target (chosen)** — Follow the existing pattern with types, converter, writer, and target registration. Most consistent with codebase conventions.
|
||||
2. **Minimal agent-only converter** — Only convert agents, skip commands/skills. Too limited; users would lose most of the plugin's value.
|
||||
3. **Documentation-only approach** — Just document how to manually set up Copilot. Doesn't compound — every user would repeat the work.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
### Component Mapping
|
||||
|
||||
| Claude Code Component | Copilot Equivalent | Notes |
|
||||
|----------------------|-------------------|-------|
|
||||
| **Agents** (`.md`) | **Custom Agents** (`.agent.md`) | Full frontmatter mapping: description, tools, target, infer |
|
||||
| **Commands** (`.md`) | **Agent Skills** (`SKILL.md`) | Commands become skills since Copilot has no direct command equivalent. `allowed-tools` dropped silently. |
|
||||
| **Skills** (`SKILL.md`) | **Agent Skills** (`SKILL.md`) | Copy as-is — format is already cross-compatible |
|
||||
| **MCP Servers** | **Repo settings JSON** | Generate a `copilot-mcp-config.json` users paste into GitHub repo settings |
|
||||
| **Hooks** | **Skipped with warning** | Copilot doesn't have a hooks equivalent |
|
||||
|
||||
### Agent Frontmatter Mapping
|
||||
|
||||
| Claude Field | Copilot Field | Mapping |
|
||||
|-------------|--------------|---------|
|
||||
| `name` | `name` | Direct pass-through |
|
||||
| `description` | `description` (required) | Direct pass-through, generate fallback if missing |
|
||||
| `capabilities` | Body text | Fold into body as "## Capabilities" section (like Cursor) |
|
||||
| `model` | `model` | Pass through (works in IDE, may be ignored on github.com) |
|
||||
| — | `tools` | Default to `["*"]` (all tools). Claude agents have unrestricted tool access, so Copilot agents should too. |
|
||||
| — | `target` | Omit (defaults to `both` — IDE + github.com) |
|
||||
| — | `infer` | Set to `true` (auto-selection enabled) |
|
||||
|
||||
### Output Directories
|
||||
|
||||
- **Repository-level (default):** `.github/agents/`, `.github/skills/`
|
||||
- **User-wide (with --personal flag):** `~/.copilot/skills/` (only skills supported at this level)
|
||||
|
||||
### Content Transformation
|
||||
|
||||
Apply transformations similar to Cursor converter:
|
||||
|
||||
1. **Task agent calls:** `Task agent-name(args)` → `Use the agent-name skill to: args`
|
||||
2. **Slash commands:** `/workflows:plan` → `/plan` (flatten namespace)
|
||||
3. **Path rewriting:** `.claude/` → `.github/` (Copilot's repo-level config path)
|
||||
4. **Agent references:** `@agent-name` → `the agent-name agent`
|
||||
|
||||
### MCP Server Handling
|
||||
|
||||
Generate a `copilot-mcp-config.json` file with the structure Copilot expects:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"server-name": {
|
||||
"type": "local",
|
||||
"command": "npx",
|
||||
"args": ["package"],
|
||||
"tools": ["*"],
|
||||
"env": {
|
||||
"KEY": "COPILOT_MCP_KEY"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note: Copilot requires env vars to use the `COPILOT_MCP_` prefix. The converter should transform env var names accordingly and include a comment/note about this.
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
### New Files
|
||||
|
||||
- `src/types/copilot.ts` — Type definitions (CopilotAgent, CopilotSkill, CopilotBundle, etc.)
|
||||
- `src/converters/claude-to-copilot.ts` — Converter with `transformContentForCopilot()`
|
||||
- `src/targets/copilot.ts` — Writer with `writeCopilotBundle()`
|
||||
- `docs/specs/copilot.md` — Format specification document
|
||||
|
||||
### Modified Files
|
||||
|
||||
- `src/targets/index.ts` — Register copilot target handler
|
||||
- `src/commands/sync.ts` — Add "copilot" to valid sync targets
|
||||
|
||||
### Test Files
|
||||
|
||||
- `tests/copilot-converter.test.ts` — Converter tests following existing patterns
|
||||
|
||||
### Character Limit
|
||||
|
||||
Copilot imposes a 30,000 character limit on agent body content. If an agent body exceeds this after folding in capabilities, the converter should truncate with a warning to stderr.
|
||||
|
||||
### Agent File Extension
|
||||
|
||||
Use `.agent.md` (not plain `.md`). This is the canonical Copilot convention and makes agent files immediately identifiable.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should the converter generate a `copilot-setup-steps.yml` workflow file for MCP servers that need special dependencies (e.g., `uv`, `pipx`)?
|
||||
- Should `.github/copilot-instructions.md` be generated with any base instructions from the plugin?
|
||||
|
||||
## Next Steps
|
||||
|
||||
→ `/workflows:plan` for implementation details
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
date: 2026-02-17
|
||||
topic: copilot-skill-naming
|
||||
---
|
||||
|
||||
# Copilot Skill Naming: Preserve Namespace
|
||||
|
||||
## What We're Building
|
||||
|
||||
Change the Copilot converter to preserve command namespaces when converting commands to skills. Currently `workflows:plan` flattens to `plan`, which is too generic and clashes with Copilot's own features in the chat suggestion UI.
|
||||
|
||||
## Why This Approach
|
||||
|
||||
The `flattenCommandName` function strips everything before the last colon, producing names like `plan`, `review`, `work` that are too generic for Copilot's skill discovery UI. Replacing colons with hyphens (`workflows:plan` -> `workflows-plan`) preserves context while staying within valid filename characters.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Replace colons with hyphens** instead of stripping the prefix: `workflows:plan` -> `workflows-plan`
|
||||
- **Copilot only** — other converters (Cursor, Droid, etc.) keep their current flattening behavior
|
||||
- **Content transformation too** — slash command references in body text also use hyphens: `/workflows:plan` -> `/workflows-plan`
|
||||
|
||||
## Changes Required
|
||||
|
||||
1. `src/converters/claude-to-copilot.ts` — change `flattenCommandName` to replace colons with hyphens
|
||||
2. `src/converters/claude-to-copilot.ts` — update `transformContentForCopilot` slash command rewriting
|
||||
3. `tests/copilot-converter.test.ts` — update affected tests
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> Implement directly (small, well-scoped change)
|
||||
@@ -0,0 +1,85 @@
|
||||
---
|
||||
date: 2026-03-14
|
||||
topic: ce-plan-rewrite
|
||||
---
|
||||
|
||||
# Rewrite `ce:plan` to Separate Planning from Implementation
|
||||
|
||||
## Problem Frame
|
||||
|
||||
`ce:plan` sits between `ce:brainstorm` and `ce:work`, but the current skill mixes issue authoring, technical planning, and pseudo-implementation. That makes plans brittle and pushes the planning phase to predict details that are often only discoverable during implementation. PR #246 intensifies this by asking plans to include complete code, exact commands, and micro-step TDD and commit choreography. The rewrite should keep planning strong enough for a capable agent or engineer to execute, while moving code-writing, test-running, and execution-time learning back into `ce:work`.
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. `ce:plan` must accept either a raw feature description or a requirements document produced by `ce:brainstorm` as primary input.
|
||||
- R2. `ce:plan` must preserve compound-engineering's planning strengths: repo pattern scan, institutional learnings, conditional external research, and requirements-gap checks when warranted.
|
||||
- R3. `ce:plan` must produce a durable implementation plan focused on decisions, sequencing, file paths, dependencies, risks, and test scenarios, not implementation code.
|
||||
- R4. `ce:plan` must not instruct the planner to run tests, generate exact implementation snippets, or learn from execution-time results. Those belong to `ce:work`.
|
||||
- R5. Plan tasks and subtasks must be right-sized for implementation handoff, but sized as logical units or atomic commits rather than 2-5 minute copy-paste steps.
|
||||
- R6. Plans must remain shareable and portable as documents or issues without tool-specific executor litter such as TodoWrite instructions, `/ce:work` choreography, or git command recipes in the artifact itself.
|
||||
- R7. `ce:plan` must carry forward product decisions, scope boundaries, success criteria, and deferred questions from `ce:brainstorm` without re-inventing them.
|
||||
- R8. `ce:plan` must explicitly distinguish what gets resolved during planning from what is intentionally deferred to implementation-time discovery.
|
||||
- R9. `ce:plan` must hand off cleanly to `ce:work`, giving enough information for task creation without pre-writing code.
|
||||
- R10. If detail levels remain, they must change depth of analysis and documentation, not the planning philosophy. A small plan can be terse while still staying decision-first.
|
||||
- R11. If an upstream requirements document contains unresolved `Resolve Before Planning` items, `ce:plan` must classify whether they are true product blockers or misfiled technical questions before proceeding.
|
||||
- R12. `ce:plan` must not plan past unresolved product decisions that would change behavior, scope, or success criteria, but it may absorb technical or research questions by reclassifying them into planning-owned investigation.
|
||||
- R13. When true blockers remain, `ce:plan` must pause helpfully: surface the blockers, allow the user to convert them into explicit assumptions or decisions, or route them back to `ce:brainstorm`.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- A fresh implementer can start work from the plan without needing clarifying questions, but the plan does not contain implementation code.
|
||||
- `ce:work` can derive actionable tasks from the plan without relying on micro-step commands or embedded git/test instructions.
|
||||
- Plans stay accurate longer as repo context changes because they capture decisions and boundaries rather than speculative code.
|
||||
- A requirements document from `ce:brainstorm` flows into planning without losing decisions, scope boundaries, or success criteria.
|
||||
- Plans do not proceed past unresolved product blockers unless the user explicitly converts them into assumptions or decisions.
|
||||
- For the same feature, the rewritten `ce:plan` produces output that is materially shorter and less brittle than the current skill or PR #246's proposed format while remaining execution-ready.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Do not redesign `ce:brainstorm`'s product-definition role.
|
||||
- Do not remove decomposition, file paths, verification, or risk analysis from `ce:plan`.
|
||||
- Do not move planning into a vague, under-specified artifact that leaves execution to guess.
|
||||
- Do not change `ce:work` in this phase beyond possible follow-up clarification of what plan structure it should prefer.
|
||||
- Do not require heavyweight PRD ceremony for small or straightforward work.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- Use a hybrid model: keep compound-engineering's research and handoff strengths, but adopt iterative-engineering's "decisions, not code" boundary.
|
||||
- Planning stops before execution: no running tests, no fail/pass learning, no exact implementation snippets, and no commit shell commands in the plan.
|
||||
- Use logical tasks and subtasks sized around atomic changes or commit units rather than 2-5 minute micro-steps.
|
||||
- Keep explicit verification and test scenarios, but express them as expected coverage and validation outcomes rather than commands with predicted output.
|
||||
- Preserve `ce:brainstorm` as the preferred upstream input when available, with clear handling for deferred technical questions.
|
||||
- Treat `Resolve Before Planning` as a classification gate: planning first distinguishes true product blockers from technical questions, then investigates only the latter.
|
||||
|
||||
## High-Level Direction
|
||||
|
||||
- Phase 0: Resume existing plan work when relevant, detect brainstorm input, and assess scope.
|
||||
- Phase 1: Gather context through repo research, institutional learnings, and conditional external research.
|
||||
- Phase 2: Resolve planning-time technical questions and capture implementation-time unknowns separately.
|
||||
- Phase 3: Structure the plan around components, dependencies, files, test targets, risks, and verification.
|
||||
- Phase 4: Write a right-sized plan artifact whose depth varies by scope, but whose boundary stays planning-only.
|
||||
- Phase 5: Review and hand off to refinement, deeper research, issue sharing, or `ce:work`.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
- Keep the current `ce:plan` and only reject PR #246.
|
||||
Rejected because the underlying issue remains: the current skill already drifts toward issue-template output plus pseudo-implementation.
|
||||
- Adopt Superpowers `writing-plans` nearly wholesale.
|
||||
Rejected because it is intentionally execution-script-oriented and collapses planning into detailed code-writing and command choreography.
|
||||
- Adopt iterative-engineering `tech-planning` wholesale.
|
||||
Rejected because it would lose useful compound-engineering behaviors such as brainstorm-origin integration, institutional learnings, and richer post-plan handoff options.
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- `ce:work` can continue creating its own actionable task list from a decision-first plan.
|
||||
- If `ce:work` later benefits from an explicit section such as `## Implementation Units` or `## Work Breakdown`, that should be a separate follow-up designed around execution needs rather than micro-step code generation.
|
||||
|
||||
## Resolved During Planning
|
||||
|
||||
- [Affects R10][Technical] Replaced `MINIMAL` / `MORE` / `A LOT` with `Lightweight` / `Standard` / `Deep` to align `ce:plan` with `ce:brainstorm`'s scope model.
|
||||
- [Affects R9][Technical] Updated `ce:work` to explicitly consume decision-first plan sections such as `Implementation Units`, `Requirements Trace`, `Files`, `Test Scenarios`, and `Verification`.
|
||||
- [Affects R2][Needs research] Kept SpecFlow as a conditional planning aid: use it for `Standard` or `Deep` plans when flow completeness is unclear rather than making it mandatory for every plan.
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> Review, refine, and commit the `ce:plan` and `ce:work` rewrite
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
date: 2026-03-15
|
||||
topic: ce-ideate-skill
|
||||
---
|
||||
|
||||
# ce:ideate — Open-Ended Ideation Skill
|
||||
|
||||
## Problem Frame
|
||||
|
||||
The ce:brainstorm skill is reactive — the user brings an idea, and the skill helps refine it through collaborative dialogue. There is no workflow for the opposite direction: having the AI proactively generate ideas by deeply understanding the project and then filtering them through critical self-evaluation. Users currently achieve this through ad-hoc prompting (e.g., "come up with 100 ideas and give me your best 10"), but that approach has no codebase grounding, no structured output, no durable artifact, and no connection to the ce:* workflow pipeline.
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. ce:ideate is a standalone skill, separate from ce:brainstorm, with its own SKILL.md in `plugins/compound-engineering/skills/ce-ideate/`
|
||||
- R2. Accepts an optional freeform argument that serves as a focus hint — can be a concept ("DX improvements"), a path ("plugins/compound-engineering/skills/"), a constraint ("low-complexity quick wins"), or empty for fully open ideation
|
||||
- R3. Performs a deep codebase scan before generating ideas, grounding ideation in the actual project state rather than abstract speculation
|
||||
- R4. Preserves the user's proven prompt mechanism as the core workflow: generate many ideas first, then systematically and critically reject weak ones, then explain only the surviving ideas in detail
|
||||
- R5. Self-critiques the full list, rejecting weak ideas with explicit reasoning — the adversarial filtering step is the core quality mechanism
|
||||
- R6. Presents the top 5-7 surviving ideas with structured analysis: description, rationale, downsides, confidence score (0-100%), estimated complexity
|
||||
- R7. Includes a brief rejection summary — one-line per rejected idea with the reason — so the user can see what was considered and why it was cut
|
||||
- R8. Writes a durable ideation artifact to `docs/ideation/YYYY-MM-DD-<topic>-ideation.md` (or `YYYY-MM-DD-open-ideation.md` when no focus area). This compounds — rejected ideas prevent re-exploring dead ends, and un-acted-on ideas remain available for future sessions.
|
||||
- R9. The default volume (~30 ideas, top 5-7 presented) can be overridden by the user's argument (e.g., "give me your top 3" or "go deep, 100 ideas")
|
||||
- R10. Handoff options after presenting ideas: brainstorm a selected idea (feeds into ce:brainstorm), refine the ideation (dig deeper, re-evaluate, explore new angles), share to Proof, or end the session
|
||||
- R11. Always routes to ce:brainstorm when the user wants to act on an idea — ideation output is never detailed enough to skip requirements refinement
|
||||
- R12. Session completion: when ending, offer to commit the ideation doc to the current branch. If the user declines, leave the file uncommitted. Do not create branches or push — just the local commit.
|
||||
- R13. Resume behavior: when ce:ideate is invoked, check `docs/ideation/` for ideation docs created within the last 30 days. If a relevant one exists, offer to continue from it (add new ideas, revisit rejected ones, act on un-explored ideas) or start fresh.
|
||||
- R14. Present the surviving candidates to the user before writing the durable ideation artifact, so the user can ask questions or lightly reshape the candidate set before it is archived
|
||||
- R15. The ideation artifact must be written or updated before any downstream handoff, Proof sharing, or session end, even though the initial survivor presentation happens first
|
||||
- R16. Refine routes based on intent: "add more ideas" or "explore new angles" returns to generation (Phase 2), "re-evaluate" or "raise the bar" returns to critique (Phase 3), "dig deeper on idea #N" expands that idea's analysis in place. The ideation doc is updated after each refinement when the refined state is being preserved
|
||||
- R17. Uses agent intelligence to improve ideation quality, but only as support for the core prompt mechanism rather than as a replacement for it
|
||||
- R18. Uses existing research agents for codebase grounding, but ideation and critique sub-agents are prompt-defined roles with distinct perspectives rather than forced reuse of existing named review agents
|
||||
- R19. When sub-agents are used for ideation, each one receives the same grounding summary, the user focus hint, and the current volume target
|
||||
- R20. Focus hints influence both candidate generation and final filtering; they are not only an evaluation-time bias
|
||||
- R21. Ideation sub-agents return ideas in a standardized structured format so the orchestrator can merge, dedupe, and reason over them consistently
|
||||
- R22. The orchestrator owns final scoring, ranking, and survivor decisions across the merged idea set; sub-agents may emit lightweight local signals, but they do not authoritatively rank their own ideas
|
||||
- R23. Distinct ideation perspectives should be created through prompt framing methods that encourage creative spread without over-constraining the workflow; examples include friction, unmet need, inversion, assumption-breaking, leverage, and extreme-case prompts
|
||||
- R24. The skill does not hardcode a fixed number of sub-agents for all runs; it should use the smallest useful set that preserves diversity without overwhelming the orchestrator's context window
|
||||
- R25. When the user picks an idea to brainstorm, the ideation doc is updated to mark that idea as "explored" with a reference to the resulting brainstorm session date, so future revisits show which ideas have been acted on.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- A user can invoke `/ce:ideate` with no arguments on any project and receive genuinely surprising, high-quality improvement ideas grounded in the actual codebase
|
||||
- Ideas that survive the filter are meaningfully better than what the user would get from a naive "give me 10 ideas" prompt
|
||||
- The workflow uses agent intelligence to widen the candidate pool without obscuring the core generate -> reject -> survivors mechanism
|
||||
- The user sees and can question the surviving candidates before they are written into the durable artifact
|
||||
- The ideation artifact persists and provides value when revisited weeks later
|
||||
- The skill composes naturally with the existing pipeline: ideate → brainstorm → plan → work
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- ce:ideate does NOT produce requirements, plans, or code — it produces ranked ideas
|
||||
- ce:ideate does NOT modify ce:brainstorm's behavior — discovery of ce:ideate is handled through the skill description and catalog, not by altering other skills
|
||||
- The skill does not do external research (competitive analysis, similar projects) in v1 — this could be a future enhancement but adds cost and latency without proven need
|
||||
- No configurable depth modes in v1 — fixed volume with argument-based override is sufficient
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Standalone skill, not a mode within ce:brainstorm**: The workflows are fundamentally different cognitive modes (proactive/divergent vs. reactive/convergent) with different phases, outputs, and success criteria. Combining them would make ce:brainstorm harder to maintain and blur its identity.
|
||||
- **Durable artifact in docs/ideation/**: Discarding ideation results is anti-compounding. The file is cheap to write and provides value when revisiting un-acted-on ideas or avoiding re-exploration of rejected ones.
|
||||
- **Artifact written after candidate review, not before initial presentation**: The first survivor presentation is collaborative review, not archival finalization. The artifact should be written only after the candidate set is good enough to preserve, but always before handoff, sharing, or session end.
|
||||
- **Always route to ce:brainstorm for follow-up**: At ideation depth, ideas are one-paragraph concepts — never detailed enough to skip requirements refinement.
|
||||
- **Survivors + rejection summary output format**: Full transparency on what was considered without overwhelming with detailed analysis of rejected ideas.
|
||||
- **Freeform optional argument**: A concept, a path, or nothing at all — the skill interprets whatever it gets as context. No artificial distinction between "focus area" and "target path."
|
||||
- **Agent intelligence as support, not replacement**: The value comes from the proven ideation-and-rejection mechanism. Parallel sub-agents help produce a richer candidate pool and stronger critique, but the orchestrator remains responsible for synthesis, scoring, and final ranking.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R3][Technical] Which research agents should always run for codebase grounding in v1 beyond `repo-research-analyst` and `learnings-researcher`, if any?
|
||||
- [Affects R21][Technical] What exact structured output schema should ideation sub-agents return so the orchestrator can merge and score consistently without overfitting the format too early?
|
||||
- [Affects R6][Technical] Should the structured analysis per surviving idea include "suggested next steps" or "what this would unlock" beyond the current fields (description, rationale, downsides, confidence, complexity)?
|
||||
- [Affects R2][Technical] How should the skill detect volume overrides in the freeform argument vs. focus-area hints? Simple heuristic or explicit parsing?
|
||||
|
||||
## Next Steps
|
||||
|
||||
→ `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,65 @@
|
||||
---
|
||||
date: 2026-03-16
|
||||
topic: issue-grounded-ideation
|
||||
---
|
||||
|
||||
# Issue-Grounded Ideation Mode for ce:ideate
|
||||
|
||||
## Problem Frame
|
||||
|
||||
When a team wants to ideate on improvements, their issue tracker holds rich signal about real user pain, recurring failures, and severity patterns — but ce:ideate currently only looks at the codebase and past learnings. Teams have to manually synthesize issue patterns before ideating, or they ideate without that context and miss what their users are actually hitting.
|
||||
|
||||
The goal is not "fix individual bugs" but "generate strategic improvement ideas grounded in the patterns your issue tracker reveals." 25 duplicate bugs about the same failure mode is a signal about collaboration reliability, not 25 separate problems.
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. When the user's argument indicates they want issue-tracker data as input (e.g., "bugs", "github issues", "open issues", "what users are reporting", "issue patterns"), ce:ideate activates an issue intelligence step alongside the existing Phase 1 scans
|
||||
- R2. A new **issue intelligence agent** fetches, clusters, deduplicates, and analyzes issues, returning structured theme analysis — not a list of individual issues
|
||||
- R3. The agent fetches **open issues** plus **recently closed issues** (approximately 30 days), filtering out issues closed as duplicate, won't-fix, or not-planned. Recently fixed issues are included because they show which areas had enough pain to warrant action.
|
||||
- R4. Issue clusters drive the ideation frames in Phase 2 using a **hybrid strategy**: derive frames from clusters, pad with default frames (e.g., "assumption-breaking", "leverage/compounding") when fewer than 4 clusters exist. This ensures ideas are grounded in real pain patterns while maintaining ideation diversity.
|
||||
- R5. The existing Phase 1 scans (codebase context + learnings search) still run in parallel — issue analysis is additive context, not a replacement
|
||||
- R6. The issue intelligence agent detects the repository from the current directory's git remote
|
||||
- R7. Start with GitHub issues via `gh` CLI. Design the agent prompt and output structure so Linear or other trackers can be added later without restructuring the ideation flow.
|
||||
- R8. The issue intelligence agent is independently useful outside of ce:ideate — it can be dispatched directly by a user or other workflows to summarize issue themes, understand the current landscape, or reason over recent activity. Its output should be self-contained, not coupled to ideation-specific context.
|
||||
- R9. The agent's output must communicate at the **theme level**, not the individual-issue level. Each theme should convey: what the pattern is, why it matters (user impact, severity, frequency, trend direction), and what it signals about the system. The output should help a human or agent fully understand the importance and shape of each theme without needing to read individual issues.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Running `/ce:ideate bugs` on a repo with noisy/duplicate issues (like proof's 25+ LIVE_DOC_UNAVAILABLE variants) produces clustered themes, not a rehash of individual issues
|
||||
- Surviving ideas are strategic improvements ("invest in collaboration reliability infrastructure") not bug fixes ("fix LIVE_DOC_UNAVAILABLE")
|
||||
- The issue intelligence agent's output is structured enough that ideation sub-agents can engage with themes meaningfully
|
||||
- Ideation quality is at least as good as the default mode, with the added benefit of issue grounding
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- GitHub issues only in v1 (Linear is a future extension)
|
||||
- No issue triage or management — this is read-only analysis for ideation input
|
||||
- No changes to Phase 3 (adversarial filtering) or Phase 4 (presentation) — only Phase 1 and Phase 2 frame derivation are affected
|
||||
- The issue intelligence agent is a new agent file, not a modification to an existing research agent
|
||||
- The agent is designed as a standalone capability that ce:ideate composes, not an ideation-internal module
|
||||
- Assumes `gh` CLI is available and authenticated in the environment
|
||||
- When a repo has too few issues to cluster meaningfully (e.g., < 5 open+recent), the agent should report that and ce:ideate should fall back to default ideation with a note to the user
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Pattern-first, not issue-first**: The output is improvement ideas grounded in bug patterns, not a prioritized bug list. The ideation instructions already prevent "just fix bug #534" thinking.
|
||||
- **Hybrid frame strategy**: Clusters derive ideation frames, padded with defaults when thin. Pure cluster-derived frames risk too few frames; pure default frames risk ignoring the issue signal.
|
||||
- **Flexible argument detection**: Use intent-based parsing ("reasonable interpretation rather than formal parsing") consistent with the existing volume hint system. No rigid keyword matching.
|
||||
- **Open + recently closed**: Including recently fixed issues provides richer pattern data — shows which areas warranted action, not just what's currently broken.
|
||||
- **Additive to Phase 1**: Issue analysis runs as a third parallel agent alongside codebase scan and learnings search. All three feed the grounding summary.
|
||||
- **Titles + labels + sample bodies**: Read titles and labels for all issues (cheap), then read full bodies for 2-3 representative issues per emerging cluster. This handles both well-labeled repos (labels drive clustering, bodies confirm) and poorly-labeled repos (bodies drive clustering). Avoids reading all bodies which is expensive at scale.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R2][Technical] What structured output format should the issue intelligence agent return? Likely theme clusters with: theme name, issue count, severity distribution, representative issue titles, and a one-line synthesis.
|
||||
- [Affects R3][Technical] How to detect GitHub close reasons (completed vs not-planned vs duplicate) via `gh` CLI? May need `gh issue list --state closed --json stateReason` or label-based filtering.
|
||||
- [Affects R4][Technical] What's the threshold for "too few clusters"? Current thinking: pad with default frames when fewer than 4 clusters, but this may need tuning.
|
||||
- [Affects R6][Technical] How to extract the GitHub repo from git remote? Standard `gh repo view --json nameWithOwner` or parse the remote URL.
|
||||
- [Affects R7][Needs research] What would a Linear integration look like? Just swapping the fetch mechanism, or does Linear's project/cycle structure change the clustering approach?
|
||||
- [Affects R2][Technical] Exact number of sample bodies per cluster to read (starting point: 2-3 per cluster).
|
||||
|
||||
## Next Steps
|
||||
|
||||
→ `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,89 @@
|
||||
---
|
||||
date: 2026-03-17
|
||||
topic: release-automation
|
||||
---
|
||||
|
||||
# Release Automation and Changelog Ownership
|
||||
|
||||
## Problem Frame
|
||||
|
||||
The repository currently has one automated release flow for the npm CLI, but the broader release story is split across CI, manual maintainer workflows, stale docs, and multiple version surfaces. That makes it hard to batch releases intentionally, hard for multiple maintainers to share release responsibility, and easy for changelogs, plugin manifests, and derived metadata like component counts to drift out of sync. The goal is to move to a release model that supports intentional batching, independent component versioning, centralized history, and CI-owned release authority without forcing version bumps for untouched plugins.
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. The release process must be manually triggered; merging to `main` must not automatically publish a release.
|
||||
- R2. The release system must support batching: releasable merges may accumulate on `main` until maintainers decide to cut a release.
|
||||
- R3. The release system must maintain a single release PR for the whole repo that stays open until merged and automatically accumulates additional releasable changes merged to `main`.
|
||||
- R4. The release system must support independent version bumps for these components: `cli`, `compound-engineering`, `coding-tutor`, and `marketplace`.
|
||||
- R5. The release system must not bump untouched plugins or unrelated components.
|
||||
- R6. The release system must preserve one centralized root `CHANGELOG.md` as the canonical changelog for the repository.
|
||||
- R7. The root changelog must record releases as top-level entries per component version, rather than requiring separate changelog files per plugin.
|
||||
- R8. Existing root changelog history must be preserved during the migration; the new release model must not discard or rewrite historical entries in a way that loses continuity.
|
||||
- R9. `plugins/compound-engineering/CHANGELOG.md` must no longer be treated as the canonical changelog after the migration.
|
||||
- R10. The release process must replace the current `release-docs` workflow; `release-docs` must no longer act as a release authority or required release step.
|
||||
- R11. Narrow scripts must replace `release-docs` responsibilities, including metadata synchronization, count calculation, docs generation where still needed, and validation.
|
||||
- R12. Release automation must be the sole authority for version bumps, changelog writes, and computed metadata updates such as counts of agents, skills, commands, or similar release-owned descriptions.
|
||||
- R13. The release flow must support a dry-run mode that summarizes what would happen without publishing, tagging, or committing release changes.
|
||||
- R14. Dry run output must clearly summarize which components would release, the proposed version bumps, the changelog entries that would be added, and any blocking validation failures.
|
||||
- R15. Marketplace version bumps must happen only for marketplace-level changes, such as marketplace metadata changes or adding/removing plugins from the catalog.
|
||||
- R16. Updating a plugin version alone must not require a marketplace version bump.
|
||||
- R17. Plugin-only content changes must be releasable without requiring a CLI version bump when the CLI code itself has not changed.
|
||||
- R18. The release model must remain compatible with the current install behavior where `bunx @every-env/compound-plugin install ...` runs the npm CLI but fetches named plugin content from the GitHub repository at runtime.
|
||||
- R19. The release process must be triggerable by a maintainer or an AI agent through CI without requiring a local maintainer-only skill.
|
||||
- R20. The resulting model must scale to future plugins without requiring the repo to special-case `compound-engineering` forever.
|
||||
- R21. The release model must continue to rely on conventional release intent signals (`feat`, `fix`, breaking changes, etc.), but component scopes in commit or PR titles must remain optional rather than required.
|
||||
- R22. Release automation must infer component ownership primarily from changed files, not from commit or PR title scopes alone.
|
||||
- R23. The repo should enforce parseable conventional PR or merge titles strongly enough for release tooling to classify change type, while avoiding mandatory component scoping on every change.
|
||||
- R24. The manual CI-driven release workflow must support explicit bump overrides for exceptional cases, at least `patch`, `minor`, and `major`, without requiring maintainers to create fake or empty commits purely to coerce a release.
|
||||
- R25. Bump overrides must be expressible per component rather than only as a repo-wide override.
|
||||
- R26. Dry run output must clearly show both the inferred bump and any applied manual override for each affected component.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Maintainers can let multiple PRs merge to `main` without immediately cutting a release.
|
||||
- At any point, maintainers can inspect a release PR or dry run and understand what would ship next.
|
||||
- A change to `coding-tutor` does not force a version bump to `compound-engineering`.
|
||||
- A plugin version bump does not force a marketplace version bump unless marketplace-level files changed.
|
||||
- Release-owned metadata and counts stay in sync without relying on a local slash command.
|
||||
- The root changelog remains readable and continuous before and after the migration.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- This work does not require changing how Claude Code itself consumes plugin and marketplace versions.
|
||||
- This work does not require solving end-user auto-update discovery for non-Claude harnesses in v1.
|
||||
- This work does not require adding dedicated per-plugin changelog files as the canonical history model.
|
||||
- This work does not require immediate future automation of release timing; manual release remains the default.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Use `release-please` rather than a single release-line flow**: The repo now has multiple independently versioned components, and the release PR model matches the need to batch merges on `main` until a release is intentionally cut.
|
||||
- **One release PR for the whole repo**: Centralized release visibility matters more than separate PRs per component, and a single release PR can still carry multiple component bumps.
|
||||
- **Manual release timing**: The release process should prepare and accumulate the next release automatically, but the decision to cut that release should remain explicit.
|
||||
- **Root changelog stays canonical**: Centralized history is more important than per-plugin changelog isolation for the current repo shape.
|
||||
- **Top-level changelog entries per component version**: This preserves one changelog file while keeping independent component version history readable.
|
||||
- **Retire `release-docs`**: Its responsibilities are too broad, stale, and conflated. Release logic, docs logic, and metadata synchronization should be separated.
|
||||
- **Scripts for narrow responsibilities**: Explicit scripts are easier to validate, automate, and reuse from CI than a local repo-maintenance skill.
|
||||
- **Marketplace version is catalog-scoped**: Plugin version bumps alone should not imply a marketplace release.
|
||||
- **Conventional type required, component scope optional**: Release intent should still come from conventional commit semantics, but requiring `(compound-engineering)` on most repo changes would add unnecessary wording overhead. Component detection should remain file-driven.
|
||||
- **Manual bump override is an explicit escape hatch**: Automatic bump inference remains the default, but maintainers should be able to override a component's release level in CI for exceptional cases without awkward synthetic commits.
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- The current install flow for named plugins continues to fetch plugin content from GitHub at runtime, so plugin content releases can remain independent from CLI releases unless CLI behavior also changes.
|
||||
- Claude Code already respects marketplace and plugin versions, so those version surfaces remain meaningful release signals.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R3][Technical] Should the release PR be updated automatically on every push to `main`, or via a manually triggered maintenance workflow that refreshes the release PR state on demand?
|
||||
- [Affects R7][Technical] What exact root changelog format best balances readability and automation for multiple component-version entries in one file?
|
||||
- [Affects R11][Technical] Which responsibilities should become distinct scripts versus steps embedded directly in the CI workflow?
|
||||
- [Affects R12][Technical] Which release-owned metadata fields should be computed automatically versus validated and left untouched when no count change is needed?
|
||||
- [Affects R9][Technical] Should `plugins/compound-engineering/CHANGELOG.md` be deleted, frozen, or replaced with a short pointer note after the migration?
|
||||
- [Affects R21][Technical] Should conventional-format enforcement happen on PR titles, squash-merge titles, commits, or some combination of them?
|
||||
- [Affects R24][Technical] Should manual bump overrides be implemented as workflow inputs that shape the generated release PR directly, or as an internal generated release-control commit on the release branch only?
|
||||
|
||||
## Next Steps
|
||||
|
||||
→ `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,50 @@
|
||||
---
|
||||
date: 2026-03-18
|
||||
topic: auto-memory-integration
|
||||
---
|
||||
|
||||
# Auto Memory Integration for ce:compound and ce:compound-refresh
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Claude Code's Auto Memory feature passively captures debugging insights, fix patterns, and preferences across sessions in `~/.claude/projects/<project>/memory/`. The ce:compound and ce:compound-refresh skills currently don't leverage this data source, even though it contains exactly the kind of raw material these workflows need: notes about problems solved, approaches tried, and patterns discovered.
|
||||
|
||||
After long sessions or compaction, auto memory may preserve insights that conversation context has lost. For ce:compound-refresh, auto memory may contain newer observations that signal drift in existing docs/solutions/ learnings without anyone explicitly flagging it.
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. **ce:compound uses auto memory as supplementary evidence.** The orchestrator reads MEMORY.md before launching Phase 1 subagents, scans for entries related to the problem being documented, and passes relevant memory content as additional context to the Context Analyzer and Solution Extractor subagents. Those subagents treat memory notes as supplementary evidence alongside conversation history.
|
||||
- R2. **ce:compound-refresh investigation subagents check auto memory.** When investigating a candidate learning's staleness, investigation subagents also check auto memory for notes in the same problem domain. A memory note describing a different approach than what the learning recommends is treated as a drift signal.
|
||||
- R3. **Graceful absence handling.** If auto memory doesn't exist for the project (no memory directory or empty MEMORY.md), all skills proceed exactly as they do today with no errors or warnings.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- ce:compound produces richer documentation when auto memory contains relevant notes about the fix, especially after sessions involving compaction
|
||||
- ce:compound-refresh surfaces staleness signals that would otherwise require manual discovery
|
||||
- No regression when auto memory is absent or empty
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- **Not changing auto memory's output location or format** -- these skills consume it as-is
|
||||
- **Read-only** -- neither skill writes to auto memory; ce:compound writes to docs/solutions/ (team-shared, structured), which serves a different purpose than machine-local auto memory
|
||||
- **Not adding a new subagent** -- existing subagents are augmented with memory-checking instructions
|
||||
- **Not changing the structure of docs/solutions/ output** -- the final artifacts are the same
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- Claude knows its auto memory directory path from the system prompt context in every session -- no path discovery logic needed in the skills
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Augment existing subagents, not a new one**: ce:compound-refresh investigation subagents need memory context during their own investigation (not as a separate report), so a dedicated Memory Scanner subagent would be awkward. For ce:compound, the orchestrator pre-reads MEMORY.md once and passes relevant excerpts to subagents, avoiding redundant reads while keeping the same subagent count.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R1][Technical] How should the orchestrator determine which MEMORY.md entries are "related" to the current problem? Keyword matching against the problem description, or broader heuristic?
|
||||
- [Affects R2][Technical] Should ce:compound-refresh investigation subagents read the full MEMORY.md or only topic files matching the learning's domain? The 200-line MEMORY.md is small enough to read in full, but topic files may be more targeted.
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,187 @@
|
||||
# Frontend Design Skill Improvement
|
||||
|
||||
**Date:** 2026-03-22
|
||||
**Status:** Design approved, pending implementation plan
|
||||
**Scope:** Rewrite `frontend-design` skill + surgical addition to `ce:work-beta`
|
||||
|
||||
## Context
|
||||
|
||||
The current `frontend-design` skill (43 lines) is a brief aesthetic manifesto forked from the Anthropic official skill. It emphasizes bold design and avoiding AI slop but lacks practical structure, concrete constraints, context-specific guidance, and any verification mechanism.
|
||||
|
||||
Two external sources informed this redesign:
|
||||
- **Anthropic's official frontend-design skill** -- nearly identical to ours, same gaps
|
||||
- **OpenAI's frontend skill** (from their "Designing Delightful Frontends with GPT-5.4" article, March 2026) -- dramatically more comprehensive with composition rules, context modules, card philosophy, copy guidelines, motion specifics, and litmus checks
|
||||
|
||||
Additionally, the beta workflow (`ce:plan-beta` -> `deepen-plan-beta` -> `ce:work-beta`) has no mechanism to invoke the frontend-design skill. The old `deepen-plan` discovered and applied it dynamically; `deepen-plan-beta` uses deterministic agent mapping and skips skill discovery entirely. The skill is effectively orphaned in the beta workflow.
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### Authority Hierarchy
|
||||
|
||||
Every rule in the skill is a default, not a mandate:
|
||||
1. **Existing design system / codebase patterns** -- highest priority, always respected
|
||||
2. **User's explicit instructions** -- override skill defaults
|
||||
3. **Skill defaults** -- only fully apply in greenfield or when user asks for design guidance
|
||||
|
||||
This addresses a key weakness in OpenAI's approach: their rules read as absolutes ("No cards by default", "Full-bleed hero only") without escape hatches. Users who want cards in the hero shouldn't fight their own tooling.
|
||||
|
||||
### Layered Architecture
|
||||
|
||||
The skill is structured as layers:
|
||||
|
||||
- **Layer 0: Context Detection** -- examine codebase for existing design signals before doing anything. Short-circuits opinionated guidance when established patterns exist.
|
||||
- **Layer 1: Pre-Build Planning** -- visual thesis + content plan + interaction plan (3 short statements). Adapts to greenfield vs existing codebase.
|
||||
- **Layer 2: Design Guidance Core** -- always-applicable principles (typography, color, composition, motion, accessibility, imagery). All yield to existing systems.
|
||||
- **Context Modules** -- agent selects one based on what's being built:
|
||||
- Module A: Landing pages & marketing (greenfield)
|
||||
- Module B: Apps & dashboards (greenfield)
|
||||
- Module C: Components & features (default when working inside an existing app, regardless of what's being built)
|
||||
|
||||
### Layer 0: Detection Signals (Concrete Checklist)
|
||||
|
||||
The agent looks for these specific signals when classifying the codebase:
|
||||
|
||||
- **Design tokens / CSS variables**: `--color-*`, `--spacing-*`, `--font-*` custom properties, theme files
|
||||
- **Component libraries**: shadcn/ui, Material UI, Chakra, Ant Design, Radix, or project-specific component directories
|
||||
- **CSS frameworks**: `tailwind.config.*`, `styled-components` theme, Bootstrap imports, CSS modules with consistent naming
|
||||
- **Typography**: Font imports in HTML/CSS, `@font-face` declarations, Google Fonts links
|
||||
- **Color palette**: Defined color scales, brand color files, design token exports
|
||||
- **Animation libraries**: Framer Motion, GSAP, anime.js, Motion One, Vue Transition imports
|
||||
- **Spacing / layout patterns**: Consistent spacing scale usage, grid systems, layout components
|
||||
|
||||
**Mode classification:**
|
||||
- **Existing system**: 4+ signals detected across multiple categories. Defer to it.
|
||||
- **Partial system**: 1-3 signals detected. Apply skill defaults where no convention was detected; yield to detected conventions where they exist.
|
||||
- **Greenfield**: No signals detected. Full skill guidance applies.
|
||||
- **Ambiguous**: Signals are contradictory or unclear. Ask the user.
|
||||
|
||||
### Interaction Method for User Questions
|
||||
|
||||
When Layer 0 needs to ask the user (ambiguous detection), use the platform's blocking question tool:
|
||||
- Claude Code: `AskUserQuestion`
|
||||
- Codex: `request_user_input`
|
||||
- Gemini CLI: `ask_user`
|
||||
- Fallback: If no question tool is available, assume "partial" mode and proceed conservatively.
|
||||
|
||||
### Where We Improve Beyond OpenAI
|
||||
|
||||
1. **Accessibility as a first-class concern** -- OpenAI's skill is pure aesthetics. We include semantic HTML, contrast ratios, focus states as peers of typography and color.
|
||||
|
||||
2. **Existing codebase integration** -- OpenAI has one exception line buried in the rules. We make context detection the first step and add Module C specifically for "adding a feature to an existing app" -- the most common real-world case that both OpenAI and Anthropic ignore entirely.
|
||||
|
||||
3. **Defaults with escape hatches** -- Two-tier anti-pattern system: "default against" (overridable preferences) vs "always avoid" (genuine quality failures). OpenAI mixes these in a flat list.
|
||||
|
||||
4. **Framework-aware animation defaults** -- OpenAI assumes Framer Motion. We detect existing animation libraries first. When no existing library is found, the default is framework-conditional: CSS animations as the universal baseline, Framer Motion for React, Vue Transition / Motion One for Vue, Svelte transitions for Svelte.
|
||||
|
||||
5. **Visual self-verification** -- Neither OpenAI nor Anthropic have any verification. We add a browser-based screenshot + assessment step with a tool preference cascade:
|
||||
1. Existing project browser tooling (Playwright, Puppeteer, etc.)
|
||||
2. Browser MCP tools (claude-in-chrome, etc.)
|
||||
3. agent-browser CLI (default when nothing else exists -- load the `agent-browser` skill for setup)
|
||||
4. Mental review against litmus checks (last resort)
|
||||
|
||||
6. **Responsive guidance** -- kept light (trust smart models) but present, unlike OpenAI's single mention.
|
||||
|
||||
7. **Performance awareness** -- careful balance, noting that heavy animations and multiple font imports have costs, without being prescriptive about specific thresholds.
|
||||
|
||||
8. **Copy guidance without arbitrary thresholds** -- OpenAI says "if deleting 30% of the copy improves the page, keep deleting." We use: "Every sentence should earn its place. Default to less copy, not more."
|
||||
|
||||
### Scope Control on Verification
|
||||
|
||||
Visual verification is a sanity check, not a pixel-perfect review. One pass. If there's a glaring issue, fix it. If it looks solid, move on. The goal is catching "this clearly doesn't work" before the user sees it.
|
||||
|
||||
### ce:work-beta Integration
|
||||
|
||||
A small addition to Phase 2 (Execute), after the existing Figma Design Sync section:
|
||||
|
||||
**UI task detection heuristic:** A task is a "UI task" if any of these are true:
|
||||
- The task's implementation files include view, template, component, layout, or page files
|
||||
- The task creates new user-visible routes or pages
|
||||
- The plan text contains explicit "UI", "frontend", "design", "layout", or "styling" language
|
||||
- The task references building or modifying something the user will see in a browser
|
||||
|
||||
The agent uses judgment -- these are heuristics, not a rigid classifier.
|
||||
|
||||
**What ce:work-beta adds:**
|
||||
|
||||
> For UI tasks without a Figma design, load the `frontend-design` skill before implementing. Follow its detection, guidance, and verification flow.
|
||||
|
||||
This is intentionally minimal:
|
||||
- Doesn't duplicate skill content into ce:work-beta
|
||||
- Doesn't load the skill for non-UI tasks
|
||||
- Doesn't load the skill when Figma designs exist (Figma sync covers that)
|
||||
- Doesn't change any other phase
|
||||
|
||||
**Verification screenshot reuse:** The frontend-design skill's visual verification screenshot satisfies ce:work-beta Phase 4's screenshot requirement. The agent does not need to screenshot twice -- the skill's verification output is reused for the PR.
|
||||
|
||||
**Relationship to design-iterator agent:** The frontend-design skill's verification is a single sanity-check pass. For iterative refinement beyond that (multiple rounds of screenshot-assess-fix), see the `design-iterator` agent. The skill does not invoke design-iterator automatically.
|
||||
|
||||
## Files Changed
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `plugins/compound-engineering/skills/frontend-design/SKILL.md` | Full rewrite |
|
||||
| `plugins/compound-engineering/skills/ce-work-beta/SKILL.md` | Add ~5 lines to Phase 2 |
|
||||
|
||||
## Skill Description (Optimized)
|
||||
|
||||
```yaml
|
||||
name: frontend-design
|
||||
description: Build web interfaces with genuine design quality, not AI slop. Use for
|
||||
any frontend work: landing pages, web apps, dashboards, admin panels, components,
|
||||
interactive experiences. Activates for both greenfield builds and modifications to
|
||||
existing applications. Detects existing design systems and respects them. Covers
|
||||
composition, typography, color, motion, and copy. Verifies results via screenshots
|
||||
before declaring done.
|
||||
```
|
||||
|
||||
## Skill Structure (frontend-design/SKILL.md)
|
||||
|
||||
```
|
||||
Frontmatter (name, description)
|
||||
Preamble (what, authority hierarchy, workflow preview)
|
||||
Layer 0: Context Detection
|
||||
- Detect existing design signals
|
||||
- Choose mode: existing / partial / greenfield
|
||||
- Ask user if ambiguous
|
||||
Layer 1: Pre-Build Planning
|
||||
- Visual thesis (one sentence)
|
||||
- Content plan (what goes where)
|
||||
- Interaction plan (2-3 motion ideas)
|
||||
Layer 2: Design Guidance Core
|
||||
- Typography (2 typefaces max, distinctive choices, yields to existing)
|
||||
- Color & Theme (CSS variables, one accent, no purple bias, yields to existing)
|
||||
- Composition (poster mindset, cardless default, whitespace before chrome)
|
||||
- Motion (2-3 intentional motions, use existing library, framework-conditional defaults)
|
||||
- Accessibility (semantic HTML, WCAG AA contrast, focus states)
|
||||
- Imagery (real photos, stable tonal areas, image generation when available)
|
||||
Context Modules (select one)
|
||||
- A: Landing Pages & Marketing (greenfield -- hero rules, section sequence, copy as product language)
|
||||
- B: Apps & Dashboards (greenfield -- calm surfaces, utility copy, minimal chrome)
|
||||
- C: Components & Features (default in existing apps -- match existing, inherit tokens, focus on states)
|
||||
Hard Rules & Anti-Patterns
|
||||
- Default against (overridable): generic card grids, purple bias, overused fonts, etc.
|
||||
- Always avoid (quality floor): prompt language in UI, broken contrast, missing focus states
|
||||
Litmus Checks
|
||||
- Context-sensitive self-review questions
|
||||
Visual Verification
|
||||
- Tool cascade: existing > MCP > agent-browser > mental review
|
||||
- One iteration, sanity check scope
|
||||
- Include screenshot in deliverable
|
||||
```
|
||||
|
||||
## What We Keep From Current Skill
|
||||
|
||||
- Strong anti-AI-slop identity and messaging
|
||||
- Creative energy / encouragement to be bold in greenfield work
|
||||
- Tone-picking exercise (brutally minimal, maximalist chaos, retro-futuristic...)
|
||||
- "Differentiation" prompt: what makes this unforgettable?
|
||||
- Framework-agnostic approach (HTML/CSS/JS, React, Vue, etc.)
|
||||
|
||||
## Cross-Agent Compatibility
|
||||
|
||||
Per AGENTS.md rules:
|
||||
- Describe tools by capability class with platform hints, not Claude-specific names alone
|
||||
- Use platform-agnostic question patterns (name known equivalents + fallback)
|
||||
- No shell recipes for routine exploration
|
||||
- Reference co-located scripts with relative paths
|
||||
- Skill is written once, copied as-is to other platforms
|
||||
@@ -0,0 +1,84 @@
|
||||
---
|
||||
date: 2026-03-23
|
||||
topic: plan-review-personas
|
||||
---
|
||||
|
||||
# Persona-Based Plan Review for document-review
|
||||
|
||||
## Problem Frame
|
||||
|
||||
The `document-review` skill currently uses a single-voice evaluator with five generic criteria (Clarity, Completeness, Specificity, Appropriate Level, YAGNI). This catches surface-level issues but misses role-specific concerns: a security engineer, product leader, and design reviewer each see different problems in the same plan. The ce:review skill already demonstrates that multi-persona review produces richer, more actionable feedback for code. The same architecture should apply to plan review.
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. Replace the current single-voice `document-review` with a persona pipeline that dispatches specialized reviewer agents in parallel against the target document.
|
||||
|
||||
- R2. Implement 2 always-on personas that run on every document review:
|
||||
- **coherence**: Internal consistency, contradictions, terminology drift, structural issues, ambiguity. Checks whether readers would diverge on interpretation.
|
||||
- **feasibility**: Can this actually be built? Architecture decisions, external dependencies, performance requirements, migration strategies. Absorbs the "tech-plan implementability" angle (can an implementer code from this?).
|
||||
|
||||
- R3. Implement 4 conditional personas that activate based on document content analysis:
|
||||
- **product-lens**: Activates when the document contains user-facing features, market claims, scope decisions, or prioritization. Opens with a "premise challenge" -- 3 diagnostic questions that challenge whether the plan solves the right problem. Asks: "What's the 10-star version? What's the narrowest wedge that proves demand?"
|
||||
- **design-lens**: Activates when the document contains UI/UX work, frontend changes, or user flows. Uses a "rate 0-10 and describe what 10 looks like" dimensional rating method. Rates design dimensions concretely, identifies what "great" looks like for each.
|
||||
- **security-lens**: Activates when the document contains auth, data handling, external APIs, or payments. Evaluates threat model at the plan level, not code level. Surfaces what the plan fails to account for.
|
||||
- **scope-guardian**: Activates when the document contains multiple priority levels, unclear boundaries, or goals that don't align with requirements. Absorbs the "skeptic" angle -- challenges unnecessary complexity, premature abstractions, and frameworks ahead of need. Opens with a "what already exists?" check against the codebase.
|
||||
|
||||
- R4. The skill auto-detects which conditional personas are relevant by analyzing the document content. No user configuration required for persona selection.
|
||||
|
||||
- R5. Hybrid action model after persona findings are synthesized:
|
||||
- **Auto-fix**: Document quality issues (contradictions, terminology drift, structural problems, missing details that can be inferred). These are unambiguously improvements.
|
||||
- **Present for user decision**: Strategic/product questions (problem framing, scope challenges, priority conflicts, "is this the right thing to build?"). These require human judgment.
|
||||
|
||||
- R6. Each persona returns structured findings with confidence scores. The orchestrator deduplicates overlapping findings across personas and synthesizes into a single prioritized report.
|
||||
|
||||
- R7. Maintain backward compatibility with all existing callers:
|
||||
- `ce-brainstorm` Phase 4 "Review and refine" option
|
||||
- `ce-plan` / `ce-plan-beta` post-generation "Review and refine" option
|
||||
- `deepen-plan-beta` post-deepening "Review and refine" option
|
||||
- Standalone invocation
|
||||
- Returns "Review complete" when done, as callers expect
|
||||
|
||||
- R8. Pipeline-compatible: When called from automated pipelines (e.g., future lfg/slfg integration), auto-fixes run silently and only genuinely blocking strategic questions surface to the user.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Running document-review on a plan surfaces role-specific issues that the current single-voice evaluator misses (e.g., security gaps, product framing problems, scope concerns).
|
||||
- Conditional personas activate only when relevant -- a backend refactor plan does not spawn design-lens.
|
||||
- Auto-fix changes improve the document without requiring user approval for every edit.
|
||||
- Strategic findings are presented as clear questions, not vague observations.
|
||||
- All existing callers (brainstorm, plan, plan-beta, deepen-plan-beta) work without modification.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Not adding new callers or pipeline integrations beyond maintaining existing ones.
|
||||
- Not changing how deepen-plan-beta works (it strengthens with research; document-review reviews for issues).
|
||||
- Not adding user configuration for persona selection (auto-detection only for now).
|
||||
- Not inventing new review frameworks -- incorporating established review patterns (premise challenge, dimensional rating, existing-code check) into the respective personas.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Replace, don't layer**: document-review is fully replaced by the persona pipeline, not enhanced with an optional mode. Simpler mental model, one behavior.
|
||||
- **2 always-on + 4 conditional**: Coherence and feasibility run on every document. Product-lens, design-lens, security-lens, and scope-guardian activate based on content. Keeps cost proportional to document complexity.
|
||||
- **Hybrid action model**: Auto-fix document quality issues, present strategic questions. Matches the natural split between what personas surface.
|
||||
- **Absorb skeptic into scope-guardian**: Both challenge whether the plan is right-sized. One persona with both angles avoids redundancy.
|
||||
- **Absorb tech-plan implementability into feasibility**: Both ask "can this work?" One persona with both angles.
|
||||
- **Review patterns as persona behavior, not separate mechanisms**: Premise challenge goes into product-lens, dimensional rating goes into design-lens, existing-code check goes into scope-guardian.
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- Assumes the ce:review agent orchestration pattern (parallel dispatch, synthesis, dedup) can be adapted for plan review without fundamental changes.
|
||||
- Assumes plan/requirements documents are text-based and contain enough signal for content-based conditional persona selection.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R6][Technical] What is the exact structured output format for persona findings? Should it mirror ce:review's P1/P2/P3 severity model or use a different classification?
|
||||
- [Affects R4][Needs research] What content signals reliably detect each conditional persona's relevance? Need to define the heuristics (keyword-based, section-based, or semantic).
|
||||
- [Affects R1][Technical] Should personas be implemented as compound-engineering agents (like code review agents) or as inline prompt sections within the skill? Agents enable parallel dispatch; inline is simpler.
|
||||
- [Affects R5][Technical] How should the auto-fix mechanism work -- direct inline edits like current document-review, or a separate "apply fixes" pass after synthesis?
|
||||
- [Affects R7][Technical] Do any of the 4 existing callers need minor updates to handle the new output format, or is the "Review complete" contract sufficient?
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> /ce:plan for structured implementation planning
|
||||
@@ -0,0 +1,172 @@
|
||||
---
|
||||
date: 2026-03-25
|
||||
topic: config-storage-redesign
|
||||
---
|
||||
|
||||
# Config and Worktree-Safe Storage Redesign
|
||||
|
||||
## Problem Frame
|
||||
|
||||
The current branch improves `/ce-doctor` and `/ce-setup`, but it still assumes two foundations that do not hold up:
|
||||
|
||||
1. Plugin state lives inside the repo under `.context/compound-engineering/` or `todos/`, which breaks across git worktrees and Conductor-managed parallel checkouts.
|
||||
2. Older plugin flows wrote `compound-engineering.local.md`, and parts of the repo still reference it, but main no longer treats review-agent selection as an active setup concern. Any new repo/user-level config system should not revive that removed model.
|
||||
|
||||
This work is broader than dependency setup alone. It needs one coherent model for:
|
||||
|
||||
- user-level defaults
|
||||
- repo-level overrides
|
||||
- machine-local overrides
|
||||
- worktree-safe durable storage
|
||||
- setup and doctor behavior
|
||||
- skill instructions, docs, and tests that currently hardcode `compound-engineering.local.md` or `.context/compound-engineering/...`
|
||||
|
||||
Terminology for this document:
|
||||
|
||||
- `user_state_dir` = the user-level Compound Engineering directory, defaulting to `~/.compound-engineering`
|
||||
- `repo_state_dir` = the repo-local Compound Engineering directory at `<repo>/.compound-engineering`
|
||||
- per-project storage path = `<user_state_dir>/projects/<project-slug>/`
|
||||
|
||||
## Consolidation Notes
|
||||
|
||||
This document is the active consolidated requirements doc for the setup, config, and worktree-safe storage work. It replaces the earlier setup-dependency-management and todo-path-consolidation brainstorm docs and incorporates the external worktree-safe storage draft from the parallel `gwangju` workspace.
|
||||
|
||||
It changes the direction of two earlier efforts:
|
||||
|
||||
- The dependency-management work remains in scope, but `/ce-setup` can no longer write `compound-engineering.local.md`; any surviving YAML config is optional and minimal.
|
||||
- The todo-path consolidation work is superseded by home-directory storage. The dual-read migration logic still matters for durable todo files, but `.context/compound-engineering/todos/` is no longer the end state.
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. Any new plugin config introduced by this work must use plain YAML files under `repo_state_dir`, specifically `config.yaml` and `config.local.yaml`. Config is data, not a markdown document.
|
||||
- R2. Config must support a three-layer cascade with `local > project > global` precedence and first-found wins per key:
|
||||
- `<user_state_dir>/config.yaml`
|
||||
- `<repo_state_dir>/config.yaml`
|
||||
- `<repo_state_dir>/config.local.yaml`
|
||||
- R3. The config model must persist only active plugin-level behavior that truly needs durable storage, starting with minimal compatibility metadata if such metadata is still needed after planning. Deterministic path derivation under `user_state_dir` is runtime logic, not config data.
|
||||
- R4. The new config model must not reintroduce removed review-agent selection or review-context storage behavior. Reviewer selection is now automatic in `/ce:review`, and project-specific guidance belongs in `CLAUDE.md` or `AGENTS.md`, not plugin-managed config files.
|
||||
- R5. The YAML config shape may reorganize keys (for example, grouping review-related settings under a `review` object), but any such reshape must be applied consistently across all skills, docs, and tests that read or write config.
|
||||
- R6. The new config format must include only the minimum compatibility metadata needed for the plugin to decide whether `/ce-setup` must be run again.
|
||||
- R7. Compatibility checks must not rely only on plugin semver. If explicit versioning is needed, prefer a single setup or config contract revision that answers the practical question "is rerunning `/ce-setup` required?" Optional diagnostic metadata may be stored separately, but the requirements should not assume multiple independent version counters unless planning proves they are necessary.
|
||||
- R8. `/ce-setup` must treat legacy `compound-engineering.local.md` as obsolete. If the surviving CE contract still requires machine-local persisted state, `/ce-setup` may write `repo_state_dir/config.local.yaml`; otherwise it should not invent stored values just to mirror deterministic runtime path derivation. Because the legacy file no longer contains any valid first-class CE settings, `/ce-setup` should explain that it is obsolete and delete it as part of cleanup rather than attempting a semantic migration.
|
||||
- R9. `/ce-setup` must be the canonical place that executes config cleanup and any remaining compatibility migration. This flow should be safe to re-run, and it should handle at least these cases:
|
||||
- legacy `compound-engineering.local.md` exists and no repo-local CE files exist yet
|
||||
- legacy `compound-engineering.local.md` exists alongside `repo_state_dir/config.local.yaml`
|
||||
- no repo-local CE files exist yet, but deterministic storage derivation still works
|
||||
- R10. When legacy `compound-engineering.local.md` and new repo-local CE files both exist, the new CE contract is authoritative. `/ce-setup` should explain that the legacy file is obsolete and delete it rather than attempting to merge removed settings back into the new model.
|
||||
|
||||
- R11. `AGENTS.md` must define the config/storage contract section as a standard skill authoring criterion: every skill should include the approved compact header even if that specific skill does not currently consume config values, so the contract stays consistent across the plugin.
|
||||
- R12. The standard config section and its instructions must be coding-agent cross-compatible. They must not assume Claude Code-only or Codex-only tool names, interaction patterns, or permission models.
|
||||
- R13. The standard config section must be written to optimize for speed and execution reliability:
|
||||
- prefer a minimal number of reads/tool calls
|
||||
- avoid unnecessary shell fallbacks once config is established
|
||||
- reduce permission prompts where the platform makes that possible
|
||||
- keep wording concise so agents are more likely to execute it correctly
|
||||
- R14. Independently invocable skills that depend on config or storage must use one standard full preamble that:
|
||||
- prefers caller-passed resolved values
|
||||
- deterministically resolves `repo_state_dir`, `user_state_dir`, and the per-project storage path
|
||||
- reads local, project, and global YAML layers with the same precedence rules when those layers exist
|
||||
- warns and routes to `/ce-setup` when migration or rerun is needed
|
||||
- continues with degraded behavior rather than writing to legacy or guessed fallback paths when canonical config or storage cannot be resolved safely
|
||||
`AGENTS.md` must also define and enforce the delegation rule: when a parent skill spawns an agent that needs configuration or storage values, the parent skill must pass the resolved values into the agent prompt rather than making the spawned agent re-resolve them unless that agent is independently invocable.
|
||||
- R15. Migration warning behavior must be centralized rather than duplicated across the entire plugin. A small set of core entry skills, including `/ce-setup`, `/ce-doctor`, `/ce:brainstorm`, `/ce:plan`, `/ce:work`, and `/ce:review`, must detect legacy-only or conflicting config states and direct the user to run `/ce-setup` to migrate. Non-core skills should not each implement their own migration flow.
|
||||
- R16. Core entry skills and `/ce-doctor` must use the compatibility metadata to distinguish the actionable states that matter to the user:
|
||||
- no new config exists yet
|
||||
- legacy-only or conflicting config exists and `/ce-setup` must migrate it
|
||||
- new config exists but is below the required contract and `/ce-setup` must be rerun
|
||||
- config is current and no rerun is needed
|
||||
|
||||
- R17. All durable plugin storage must resolve outside the repo tree under `user_state_dir`, with this fallback chain for determining `user_state_dir`:
|
||||
- `$COMPOUND_ENGINEERING_HOME`
|
||||
- `$XDG_DATA_HOME/compound-engineering` when `XDG_DATA_HOME` is set
|
||||
- `~/.compound-engineering`
|
||||
- R18. Durable per-project storage must live under `<user_state_dir>/projects/<project-slug>/`, where the slug is deterministic and stable across worktrees of the same repo.
|
||||
- R19. Project identity must resolve from shared repo identity so all worktrees for the same repo share the same per-project storage path under `user_state_dir`. The primary identity source is `git rev-parse --path-format=absolute --git-common-dir`, and the directory-safe slug should be derived as `<sanitized-repo-name>-<short-hash>`. Non-git contexts must have a deterministic fallback.
|
||||
- R20. The standard full preamble must be sufficient for independently invocable skills to deterministically resolve the canonical per-project storage path without requiring `/ce-setup` to pre-write that path into config.
|
||||
- R21. Skills that read or write durable plugin state must use the per-project storage path under `user_state_dir` instead of repo-local `.context/compound-engineering/...` or `todos/` paths.
|
||||
- R22. Durable todo files must retain legacy read compatibility from repo-local `todos/` and `.context/compound-engineering/todos/` until they drain naturally. New todo writes must go only to `<user_state_dir>/projects/<project-slug>/todos/`.
|
||||
- R23. Per-run scratch and run-artifact directories do not need active migration from repo-local `.context/compound-engineering/...`; new writes move to `<user_state_dir>/projects/<project-slug>/<workflow>/...`.
|
||||
|
||||
- R24. `/ce-doctor` must remain a standalone entry point and expand from dependency/env checks to also report config and storage health:
|
||||
- resolved config layers
|
||||
- resolved `user_state_dir`
|
||||
- resolved `repo_state_dir`
|
||||
- resolved per-project storage path
|
||||
- presence of legacy `compound-engineering.local.md`
|
||||
- whether no repo-local CE file exists yet
|
||||
- whether setup attention is needed because a legacy file still exists or compatibility metadata is stale
|
||||
- whether rerunning setup is required because the stored compatibility metadata is below the required contract
|
||||
- whether `.compound-engineering/config.local.yaml` is safely gitignored
|
||||
- R25. `/ce-doctor` must continue to use a centralized dependency registry that lists known CLIs, MCP-backed capabilities, related environment variables, install guidance, tiering, and the skills/agents that depend on them.
|
||||
- R26. `/ce-doctor` remains informational only. It reports dependency, env, config, and storage status, but it does not install tools or mutate user config beyond diagnostics.
|
||||
- R27. `/ce-setup` must continue to include the dependency and environment flow already designed in this branch, but its output and guidance must target the new storage contract and any surviving YAML config state without inventing persisted path values that skills can derive deterministically.
|
||||
- R28. If `.compound-engineering/config.local.yaml` is part of the surviving CE contract and is not safely gitignored, `/ce-setup` must explain why that file is machine-local and offer to add an appropriate `.gitignore` entry for it.
|
||||
- R29. `/ce-setup` must present missing installable dependencies by tier, offer installation one item at a time with user approval, verify each install, and prompt for related environment variables at the appropriate point in the flow.
|
||||
- R30. For dependencies with both MCP and CLI paths, diagnostics and setup must detect MCP availability first, then CLI availability, and only offer CLI installation if neither satisfies the dependency.
|
||||
- R31. Dependency and env checks must always scan fresh on each run rather than relying on persisted installation state.
|
||||
|
||||
- R32. Skill content, docs, and tests must stop treating `.context/compound-engineering/...` and `compound-engineering.local.md` as the stable contract.
|
||||
- R33. The config and storage contract must stay tool-agnostic across Claude Code, Codex, Gemini CLI, OpenCode, Copilot, and Conductor worktrees. This work should not introduce new provider-specific config paths.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- A user can run `/ce-setup` in the main checkout or any worktree and end up with the same resolved project storage location.
|
||||
- Independently invocable skills that need CE state can derive the same canonical per-project storage path without requiring `/ce-setup` to pre-write that path.
|
||||
- Users on the legacy config format get a clear migration path through `/ce-setup` without needing every individual skill to invent its own migration behavior.
|
||||
- Core skills and `/ce-doctor` can determine whether `/ce-setup` must run again without relying on raw plugin semver comparisons or multiple unnecessary version counters.
|
||||
- Todos and other durable workflow artifacts remain available across worktrees without symlinks, git hooks, or manual copying.
|
||||
- Existing users with repo-local todo files do not lose access to unresolved work.
|
||||
- Legacy `compound-engineering.local.md` files are cleaned up by `/ce-setup` after a brief explanation, without reviving removed review-agent selection behavior.
|
||||
- `/ce-doctor` can explain both dependency gaps and config/storage misconfiguration in one report.
|
||||
- `/ce-setup` can bring `.compound-engineering/config.local.yaml` under gitignore safely instead of only warning later.
|
||||
- The dependency registry remains the single source of truth for `/ce-doctor` and `/ce-setup` rather than splitting dependency metadata across multiple docs or skills.
|
||||
- Provider conversion tests and plugin docs reflect the new contract instead of the old file/path names.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Do not add a full team-managed authoring workflow for tracked project config in `/ce-setup`; reading the project layer is in scope, authoring it is a separate effort.
|
||||
- Do not auto-migrate per-run scratch or historical run artifacts out of `.context/compound-engineering/...`.
|
||||
- Do not add storage garbage collection or project-directory pruning in this change.
|
||||
- Do not preserve markdown-frontmatter config as a long-term supported format after migration; legacy support is for import/migration, not dual-write.
|
||||
- Do not introduce provider-specific config directories for this feature.
|
||||
- Do not auto-install dependencies without explicit user approval.
|
||||
- Do not expand this work into project dependency management such as `bundle install`, `npm install`, or app-specific environment setup.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Home-directory storage is the durable answer:** repo-local `.context` is fine for scratch in a single checkout, but it is the wrong primitive for shared multi-worktree state.
|
||||
- **Plain YAML replaces the legacy markdown config format:** if this work introduces plugin-managed config, it should do so with files in `repo_state_dir`, not by extending `compound-engineering.local.md`.
|
||||
- **Legacy review config is not the target model:** main has already removed setup-managed reviewer selection. The new config system should focus on current setup-owned state such as storage and compatibility metadata, not on recreating reviewer preferences in a new file.
|
||||
- **Compatibility metadata should stay minimal:** plugin semver alone is too coarse, but the fix is not to add version fields everywhere. Keep only the metadata needed to answer whether `/ce-setup` must run again.
|
||||
- **Migration should have one owner:** `/ce-setup` should perform migration, `/ce-doctor` should report migration state, and a small set of entry skills should warn. Spreading migration logic across every skill creates drift and inconsistent user experience.
|
||||
- **Todo migration deserves special handling:** unlike per-run artifacts, todo files have a multi-session lifecycle. Read compatibility is worth keeping during the transition.
|
||||
- **Standard preamble, not universal prompt bloat:** use one shared config-loading pattern for independently invocable config/storage consumers and have parent skills pass resolved values to delegates. Requiring every skill to load config even when it does nothing with it adds carrying cost without enough value.
|
||||
- **Standard section belongs in AGENTS.md:** the skill-level config instructions should be codified as a repo authoring rule so future skills inherit the same structure instead of drifting.
|
||||
- **Cross-agent and low-friction wording matters:** the config section should be written against capability classes, minimal reads, and low-prompt execution patterns so it works well across Claude Code, Codex, Gemini, OpenCode, Copilot, and Conductor.
|
||||
- **`/ce-doctor` and `/ce-setup` stay coupled but distinct:** doctor diagnoses; setup installs/configures. The new architecture should deepen that relationship, not replace it.
|
||||
- **The dependency design from this branch carries forward:** registry-driven checks, tiered installs, env var prompting, and MCP-first detection still belong in scope. They just need to target the new config/storage contract.
|
||||
- **Gitignore safety is part of the feature, not a follow-up:** if `/ce-setup` writes `.compound-engineering/config.local.yaml` into repos, the plugin must also verify that users will not accidentally commit it. The gitignore rule should target that machine-local file, not the entire `.compound-engineering/` directory.
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- The current `/ce-doctor` dependency registry and install flow remain the starting point for the dependency portion of this work.
|
||||
- Skills and docs that currently reference `.context/compound-engineering/...` or `compound-engineering.local.md` will need an inventory-based update pass.
|
||||
- Converter and contract tests that assert old config names or old storage paths are part of the affected surface, not incidental cleanup.
|
||||
- `git worktree` metadata is available in normal git repos; planning still needs to define the exact fallback behavior for non-git contexts and edge cases.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R3][Technical] Choose the exact YAML shape for any surviving setup-owned config such as compatibility metadata and any future plugin-level keys that still belong in plugin-managed config.
|
||||
- [Affects R5][Technical] Define the smallest compatibility metadata shape that reliably tells the plugin whether `/ce-setup` must run again, and add extra diagnostic metadata only if it materially improves behavior.
|
||||
- [Affects R15][Technical] Decide when a plugin change should bump the setup or migration requirement versus when it should be treated as backward-compatible.
|
||||
- [Affects R17][Technical] Define the precise slugging and fallback algorithm for git repos, linked worktrees, and non-git directories.
|
||||
- [Affects R21][Technical] Decide how long legacy todo read compatibility remains and where to document eventual removal.
|
||||
- [Affects R13][Technical] Build the inventory of independently invocable skills that need direct config/storage loading versus parent-passed values.
|
||||
- [Affects R23][Technical] Define the doctor output format for config/storage warnings and migration guidance.
|
||||
- [Affects R30][Needs research] Inventory all docs, tests, and conversion fixtures that encode the old config/storage contract.
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for a phased implementation plan that starts by codifying the new config schema and migration strategy, then updates `/ce-setup` and `/ce-doctor`, then migrates storage consumers and tests.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
date: 2026-03-25
|
||||
topic: onboarding-skill
|
||||
---
|
||||
|
||||
# Onboarding: Codebase Onboarding Document Generator
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Onboarding is a general problem in software, but it is more acute in fast-moving codebases where code is written faster than documentation — whether through AI-assisted development, rapid prototyping, or simply a team that ships faster than it documents. The traditional assumption that the creator can explain the codebase breaks down when they didn't fully understand it to begin with, or when the codebase has evolved beyond any one person's mental model. New team members (and AI agents brought into the project) are left without the mental model they need to contribute effectively.
|
||||
|
||||
The primary audience is human developers. A document that works for human comprehension is also effective as agent context, but the inverse is not true.
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. A skill named `onboarding` that crawls a repository and generates `ONBOARDING.md` at the repo root
|
||||
- R2. The skill always regenerates the full document from scratch — no surgical updates or diffing against a previous version
|
||||
- R3. The document has a fixed filename (`ONBOARDING.md`) so the skill can detect whether one already exists; existence is the only state — no separate mode flag
|
||||
- R4. The document contains exactly five sections, each earning its place by answering a question a new contributor will ask in their first hour:
|
||||
- **What is this thing?** — Purpose, who it's for, what problem it solves
|
||||
- **How is it organized?** — Architecture, key modules, how they connect, and what the system depends on externally (databases, APIs, services, env vars)
|
||||
- **Key concepts and abstractions** — The vocabulary and architectural patterns needed to talk about and reason about this codebase
|
||||
- **Primary flow** — One concrete path through the system showing how the pieces connect (the main thing the app does)
|
||||
- **Where do I start?** — Dev setup, how to run it, where to make common types of changes
|
||||
- R5. During the crawl, if `docs/solutions/` or other existing documentation is discovered and is directly relevant to a section's content, link to it inline within that section. Do not create a separate references/further-reading section. If no relevant docs exist, the document stands on its own without mentioning their absence.
|
||||
- R6. The document is written for human comprehension first — clear prose, not agent-formatted structured data
|
||||
- R7. Use visual aids — ASCII diagrams, markdown tables — where they improve readability over prose. Architecture overviews and flow traces especially benefit from diagrams.
|
||||
- R8. Use proper markdown formatting throughout — backticks for file names, paths, commands, code references, and technical terms. Consistent styling maximizes legibility.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- A new contributor can read `ONBOARDING.md` and understand the codebase well enough to start making changes without needing the creator to explain it
|
||||
- The document is useful even when the creator themselves doesn't fully understand the architecture
|
||||
- Running the skill again on an evolved codebase produces an accurate, current document (no stale information carried over)
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Does not attempt to infer or fabricate design rationale ("why was X chosen over Y") — the creator may not know, and presenting guesses as fact is worse than saying nothing
|
||||
- Does not assess fragility or risk areas — that requires judgment about production behavior the agent doesn't have
|
||||
- Does not generate README.md, CLAUDE.md, AGENTS.md, or any other document — only `ONBOARDING.md`
|
||||
- Does not preserve hand-edits from a previous version on regeneration — if users want durable authored context, it belongs in other docs (which the skill may discover and link to)
|
||||
- No `ce:` prefix — this is a standalone utility skill, not part of the core workflow
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Always regenerate, never update**: Reading the old document to update it means the agent does two jobs (understand the codebase + fact-check the old doc). That's slower and more error-prone than regenerating.
|
||||
- **Five sections, no more**: Every section must earn its place by answering a question a new person will actually ask. No speculative sections "just in case."
|
||||
- **Inline linking only**: Existing docs are surfaced within relevant sections, not collected in an appendix. This is opportunistic — works fine when nothing exists to link to.
|
||||
- **Human-first writing**: The document targets human readers. Agent utility is a natural side effect of clear prose, not a separate design goal.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R1][Technical] How should the skill orchestrate the crawl — single-pass or dispatch sub-agents for different sections?
|
||||
- [Affects R4][Technical] What crawl strategy produces the best "Primary flow" section — entry point tracing, route analysis, or something else?
|
||||
- [Affects R4][Needs research] What's the right depth/length target for each section to be useful without becoming a wall of text?
|
||||
- [Affects R5][Technical] What heuristic determines whether a discovered doc is "directly relevant" to a section versus noise?
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,56 @@
|
||||
---
|
||||
date: 2026-03-26
|
||||
topic: merge-deepen-into-plan
|
||||
---
|
||||
|
||||
# Merge Deepen-Plan Into ce:plan
|
||||
|
||||
## Problem Frame
|
||||
|
||||
The ce:plan and deepen-plan skills form a sequential workflow where the user is offered a choice ("want to deepen?") that they can't evaluate better than the agent can. When deepen-plan runs, it already evaluates whether deepening is warranted and gates itself accordingly. The user decision adds friction without adding value.
|
||||
|
||||
With current model capabilities, the original concern about over-investing in planning is no longer a meaningful risk — the deepening skill already self-gates on scope and confidence scoring.
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. ce:plan automatically evaluates and deepens its own output after the initial plan is written, without asking the user for approval.
|
||||
- R2. When deepening runs, ce:plan reports what sections it's strengthening and why (transparency without requiring a decision).
|
||||
- R3. Deepening is skipped for Lightweight plans unless high-risk topics are detected (preserving the existing gate logic from deepen-plan).
|
||||
- R4. For Standard and Deep plans, ce:plan scores confidence gaps using deepen-plan's checklist-first, risk-weighted scoring. If no gaps exceed the threshold, it reports "confidence check passed" and moves on.
|
||||
- R5. When gaps are found, ce:plan dispatches targeted research agents (deepen-plan's deterministic agent mapping) to strengthen only the weak sections.
|
||||
- R6. The deepen-plan skill is removed as a standalone command. Re-deepening an existing plan is handled by re-running ce:plan in resume mode. In resume mode, ce:plan applies the same confidence-gap evaluation as on a fresh plan — it deepens only if gaps warrant it, unless the user explicitly requests deepening.
|
||||
- R7. The "Run deepen-plan" post-generation option in ce:plan is removed. Post-generation options become simpler.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- ce:plan produces plans at least as strong as the old ce:plan + manual deepen-plan flow
|
||||
- Users never need to decide whether to deepen — the agent handles it
|
||||
- Users see what's being strengthened (no black box)
|
||||
- One fewer skill to know about, simpler workflow
|
||||
- No regression in plan quality for any scope tier (Lightweight, Standard, Deep)
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- This does not change what deepening does — only where it lives and who decides to run it
|
||||
- No changes to the deepening logic itself (confidence scoring, agent selection, section rewriting)
|
||||
- No changes to ce:brainstorm or ce:work
|
||||
- The planning boundary (no code, no commands) is preserved
|
||||
- deepen-plan scratch space (`.context/compound-engineering/deepen-plan/`) moves under ce:plan's namespace
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Agent decides, user informed**: The agent evaluates whether deepening adds value and proceeds automatically. The user sees a brief status message about what's being strengthened but doesn't approve it. Why: the user can't evaluate this better than the agent, and the existing gate logic already prevents wasteful deepening.
|
||||
- **No standalone deepen command**: Re-deepening existing plans is handled through ce:plan's resume mode. Why: simpler mental model, one entry point for all planning work.
|
||||
- **Absorb, don't invoke**: The deepening logic is folded into ce:plan as a new phase rather than ce:plan invoking deepen-plan as a sub-skill. Why: eliminates a skill boundary and simplifies maintenance.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R1][Technical] Where exactly in ce:plan's phase structure should the confidence check and deepening phase land — as a new Phase 5 before the current post-generation options, or integrated into Phase 4 (plan writing)?
|
||||
- [Affects R6][Technical] How should ce:plan's resume mode distinguish "resume an incomplete plan" from "re-deepen a completed plan"? Likely frontmatter-based (`deepened: YYYY-MM-DD` presence).
|
||||
- [Affects R5][Technical] Should deepen-plan's artifact-backed research mode (for larger scope) use `.context/compound-engineering/ce-plan/deepen/` or a per-run subdirectory?
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> /ce:plan for structured implementation planning
|
||||
@@ -0,0 +1,232 @@
|
||||
---
|
||||
date: 2026-03-27
|
||||
topic: ce-skill-prefix-rename
|
||||
---
|
||||
|
||||
# Consistent `ce-` Prefix for All Skills and Agents
|
||||
|
||||
## Problem Frame
|
||||
|
||||
As the Claude Code plugin ecosystem grows, generic skill names like `setup`, `plan`, `review`, and `frontend-design` collide when users have multiple plugins installed. Typing `/plan` surfaces every plugin's plan skill, forcing users to scan descriptions. Agent names also collide across plugins — generic names like `adversarial-reviewer` or `security-reviewer` are common enough that multiple plugins could define them. The compound-engineering plugin currently uses an inconsistent mix: 8 core workflow skills have a `ce:` colon prefix, while 33 others have no prefix at all. Agents use verbose 3-segment references (`compound-engineering:<category>:<agent-name>`) that are cumbersome and can be simplified now that agents will have a unique `ce-` prefix. This creates collision risk, a confusing naming taxonomy, and unnecessarily verbose agent references.
|
||||
|
||||
Standardizing on a `ce-` hyphen prefix for all owned skills and agents eliminates collisions, creates a consistent namespace, simplifies agent references, and removes the colon character that requires filesystem sanitization on Windows.
|
||||
|
||||
Related: [GitHub Issue #337](https://github.com/EveryInc/compound-engineering-plugin/issues/337)
|
||||
|
||||
## Requirements
|
||||
|
||||
When doing renames of files and folders, you are required to use `git mv` whenever possible for simplicity and explicit intent and history preservation. You can fallback provided you notify when it happens and why.
|
||||
|
||||
### Naming Rules
|
||||
|
||||
- R1. All compound-engineering-owned skills and agents adopt a `ce-` hyphen prefix
|
||||
- R2. Skills currently using `ce:` colon prefix change to `ce-` hyphen prefix (e.g., `ce:plan` -> `ce-plan`)
|
||||
- R3. Skills and Agents currently without a prefix get `ce-` prepended (e.g., `setup` -> `ce-setup`, `frontend-design` -> `ce-frontend-design`, `repo-research-analyst` -> `ce-repo-research-analyst`)
|
||||
- R4. `git-*` skills replace the `git-` prefix with `ce-` (e.g., `git-commit` -> `ce-commit`, `git-worktree` -> `ce-worktree`)
|
||||
- R5. `report-bug-ce` normalizes to `ce-report-bug` (drops redundant suffix)
|
||||
|
||||
### Exclusions
|
||||
|
||||
- R6. `agent-browser` and `rclone` are excluded (sourced from upstream, not our skills)
|
||||
- R7. `lfg` and `slfg` are excluded from renaming (short memorable workflow entry points), but their internal skill invocations must be updated per R12
|
||||
|
||||
### Propagation
|
||||
|
||||
- R8. The skill and agent frontmatter `name:` field must match after rename (no more colon-vs-hyphen divergence). Directories need to reflect the new names as well when applicable.
|
||||
- R9. All cross-references updated: skill-to-skill invocations (`/ce:plan` -> `/ce-plan`), fully-qualified references (`/compound-engineering:todo-resolve` -> `/compound-engineering:ce-todo-resolve`), `Skill("compound-engineering:...")` programmatic invocations, prose mentions, skill `description:` frontmatter fields, and intra-skill path references (`${CLAUDE_PLUGIN_ROOT}/skills/<old-name>/...`)
|
||||
- R10. Active documentation updated: root README, plugin README, AGENTS.md. Note: the AGENTS.md "Why `ce:`?" rationale section (lines 53-60) needs a conceptual rewrite explaining the `ce-` convention, not just find-and-replace. Historical docs in `docs/` (past brainstorms, plans, solutions) are left as-is -- they are records of past decisions.
|
||||
- R11. Agent prompt files updated where they reference skill names.
|
||||
- R11b. Skill prompt files updated where they reference Agent names.
|
||||
- R11c. Agent references drop the `compound-engineering:` plugin prefix and keep the category. The agent name itself gets the `ce-` prefix. (e.g. `compound-engineering:review:adversarial-reviewer` -> `review:ce-adversarial-reviewer`)
|
||||
- R12. lfg and slfg orchestration chains updated to use new skill names (lfg/slfg themselves are not renamed per R7, but their internal skill and agent invocations must reflect new names)
|
||||
- R13. Converter infrastructure preserved: `sanitizePathName()` and colon-handling logic stays as future protection, not removed. Add a test assertion that no skill `name:` field contains a colon, so the sanitizer is defense-in-depth rather than a silent workaround.
|
||||
- R17. Codex converter's `isCanonicalCodexWorkflowSkill()` and `toCanonicalWorkflowSkillName()` in `src/converters/claude-to-codex.ts` updated to match `ce-` prefix pattern (currently hardcodes `ce:` prefix check). Related test fixtures in `tests/codex-converter.test.ts` and `tests/codex-writer.test.ts` updated accordingly.
|
||||
|
||||
### Testing
|
||||
|
||||
- R14. Path sanitization tests updated to reflect new naming (collision detection still works)
|
||||
- R15. `bun test` passes after all changes
|
||||
- R16. `bun run release:validate` passes after all changes
|
||||
- R18. Converter test fixtures that hardcode `ce:plan` etc. updated to `ce-plan` where they test compound-engineering plugin behavior. Fixtures testing abstract colon-handling for other plugins may remain.
|
||||
- R19. Sanity check and for every skill and agent name, grep to confirm new names are correct and old names do not persist except in historical planning, requirements, etc docs.
|
||||
|
||||
---
|
||||
|
||||
## Complete Rename Map
|
||||
|
||||
### Excluded (no change) - 4 skills
|
||||
|
||||
| Current Name | Reason |
|
||||
|---|---|
|
||||
| `agent-browser` | External/upstream |
|
||||
| `rclone` | External/upstream |
|
||||
| `lfg` | Exception (memorable name) |
|
||||
| `slfg` | Exception (memorable name) |
|
||||
|
||||
### `ce:` -> `ce-` (frontmatter only, dirs already match) - 8 skills
|
||||
|
||||
| Current Name | New Name | Dir Rename? |
|
||||
|---|---|---|
|
||||
| `ce:brainstorm` | `ce-brainstorm` | No |
|
||||
| `ce:compound` | `ce-compound` | No |
|
||||
| `ce:compound-refresh` | `ce-compound-refresh` | No |
|
||||
| `ce:ideate` | `ce-ideate` | No |
|
||||
| `ce:plan` | `ce-plan` | No |
|
||||
| `ce:review` | `ce-review` | No |
|
||||
| `ce:work` | `ce-work` | No |
|
||||
| `ce:work-beta` | `ce-work-beta` | No |
|
||||
|
||||
### `git-*` -> `ce-*` (replace prefix) - 4 skills
|
||||
|
||||
| Current Name | New Name | Dir Rename |
|
||||
|---|---|---|
|
||||
| `git-clean-gone-branches` | `ce-clean-gone-branches` | `git-clean-gone-branches/` -> `ce-clean-gone-branches/` |
|
||||
| `git-commit` | `ce-commit` | `git-commit/` -> `ce-commit/` |
|
||||
| `git-commit-push-pr` | `ce-commit-push-pr` | `git-commit-push-pr/` -> `ce-commit-push-pr/` |
|
||||
| `git-worktree` | `ce-worktree` | `git-worktree/` -> `ce-worktree/` |
|
||||
|
||||
### Special normalization - 1 skill
|
||||
|
||||
| Current Name | New Name | Dir Rename |
|
||||
|---|---|---|
|
||||
| `report-bug-ce` | `ce-report-bug` | `report-bug-ce/` -> `ce-report-bug/` |
|
||||
|
||||
### Standard prefix addition - 24 skills
|
||||
|
||||
| Current Name | New Name | Dir Rename |
|
||||
|---|---|---|
|
||||
| `agent-native-architecture` | `ce-agent-native-architecture` | `agent-native-architecture/` -> `ce-agent-native-architecture/` |
|
||||
| `agent-native-audit` | `ce-agent-native-audit` | `agent-native-audit/` -> `ce-agent-native-audit/` |
|
||||
| `andrew-kane-gem-writer` | `ce-andrew-kane-gem-writer` | `andrew-kane-gem-writer/` -> `ce-andrew-kane-gem-writer/` |
|
||||
| `changelog` | `ce-changelog` | `changelog/` -> `ce-changelog/` |
|
||||
| `claude-permissions-optimizer` | `ce-claude-permissions-optimizer` | `claude-permissions-optimizer/` -> `ce-claude-permissions-optimizer/` |
|
||||
| `deploy-docs` | `ce-deploy-docs` | `deploy-docs/` -> `ce-deploy-docs/` |
|
||||
| `dhh-rails-style` | `ce-dhh-rails-style` | `dhh-rails-style/` -> `ce-dhh-rails-style/` |
|
||||
| `document-review` | `ce-document-review` | `document-review/` -> `ce-document-review/` |
|
||||
| `dspy-ruby` | `ce-dspy-ruby` | `dspy-ruby/` -> `ce-dspy-ruby/` |
|
||||
| `every-style-editor` | `ce-every-style-editor` | `every-style-editor/` -> `ce-every-style-editor/` |
|
||||
| `feature-video` | `ce-feature-video` | `feature-video/` -> `ce-feature-video/` |
|
||||
| `frontend-design` | `ce-frontend-design` | `frontend-design/` -> `ce-frontend-design/` |
|
||||
| `gemini-imagegen` | `ce-gemini-imagegen` | `gemini-imagegen/` -> `ce-gemini-imagegen/` |
|
||||
| `onboarding` | `ce-onboarding` | `onboarding/` -> `ce-onboarding/` |
|
||||
| `orchestrating-swarms` | `ce-orchestrating-swarms` | `orchestrating-swarms/` -> `ce-orchestrating-swarms/` |
|
||||
| `proof` | `ce-proof` | `proof/` -> `ce-proof/` |
|
||||
| `reproduce-bug` | `ce-reproduce-bug` | `reproduce-bug/` -> `ce-reproduce-bug/` |
|
||||
| `resolve-pr-feedback` | `ce-resolve-pr-feedback` | `resolve-pr-feedback/` -> `ce-resolve-pr-feedback/` |
|
||||
| `setup` | `ce-setup` | `setup/` -> `ce-setup/` |
|
||||
| `test-browser` | `ce-test-browser` | `test-browser/` -> `ce-test-browser/` |
|
||||
| `test-xcode` | `ce-test-xcode` | `test-xcode/` -> `ce-test-xcode/` |
|
||||
| `todo-create` | `ce-todo-create` | `todo-create/` -> `ce-todo-create/` |
|
||||
| `todo-resolve` | `ce-todo-resolve` | `todo-resolve/` -> `ce-todo-resolve/` |
|
||||
| `todo-triage` | `ce-todo-triage` | `todo-triage/` -> `ce-todo-triage/` |
|
||||
|
||||
**Total: 37 skills renamed, 4 excluded (41 skills total)**
|
||||
|
||||
### Agent renames - 49 agents
|
||||
|
||||
All agents are renamed with `ce-` prefix within their existing category subdirs. The `compound-engineering:` plugin prefix is dropped from references, keeping the `<category>:ce-<agent-name>` format. Category subdirs are preserved for organization.
|
||||
|
||||
| Current File | New File | Old Reference | New Reference |
|
||||
|---|---|---|---|
|
||||
| `agents/design/design-implementation-reviewer.md` | `agents/design/ce-design-implementation-reviewer.md` | `compound-engineering:design:design-implementation-reviewer` | `design:ce-design-implementation-reviewer` |
|
||||
| `agents/design/design-iterator.md` | `agents/design/ce-design-iterator.md` | `compound-engineering:design:design-iterator` | `design:ce-design-iterator` |
|
||||
| `agents/design/figma-design-sync.md` | `agents/design/ce-figma-design-sync.md` | `compound-engineering:design:figma-design-sync` | `design:ce-figma-design-sync` |
|
||||
| `agents/docs/ankane-readme-writer.md` | `agents/docs/ce-ankane-readme-writer.md` | `compound-engineering:docs:ankane-readme-writer` | `docs:ce-ankane-readme-writer` |
|
||||
| `agents/document-review/adversarial-document-reviewer.md` | `agents/document-review/ce-adversarial-document-reviewer.md` | `compound-engineering:document-review:adversarial-document-reviewer` | `document-review:ce-adversarial-document-reviewer` |
|
||||
| `agents/document-review/coherence-reviewer.md` | `agents/document-review/ce-coherence-reviewer.md` | `compound-engineering:document-review:coherence-reviewer` | `document-review:ce-coherence-reviewer` |
|
||||
| `agents/document-review/design-lens-reviewer.md` | `agents/document-review/ce-design-lens-reviewer.md` | `compound-engineering:document-review:design-lens-reviewer` | `document-review:ce-design-lens-reviewer` |
|
||||
| `agents/document-review/feasibility-reviewer.md` | `agents/document-review/ce-feasibility-reviewer.md` | `compound-engineering:document-review:feasibility-reviewer` | `document-review:ce-feasibility-reviewer` |
|
||||
| `agents/document-review/product-lens-reviewer.md` | `agents/document-review/ce-product-lens-reviewer.md` | `compound-engineering:document-review:product-lens-reviewer` | `document-review:ce-product-lens-reviewer` |
|
||||
| `agents/document-review/scope-guardian-reviewer.md` | `agents/document-review/ce-scope-guardian-reviewer.md` | `compound-engineering:document-review:scope-guardian-reviewer` | `document-review:ce-scope-guardian-reviewer` |
|
||||
| `agents/document-review/security-lens-reviewer.md` | `agents/document-review/ce-security-lens-reviewer.md` | `compound-engineering:document-review:security-lens-reviewer` | `document-review:ce-security-lens-reviewer` |
|
||||
| `agents/research/best-practices-researcher.md` | `agents/research/ce-best-practices-researcher.md` | `compound-engineering:research:best-practices-researcher` | `research:ce-best-practices-researcher` |
|
||||
| `agents/research/framework-docs-researcher.md` | `agents/research/ce-framework-docs-researcher.md` | `compound-engineering:research:framework-docs-researcher` | `research:ce-framework-docs-researcher` |
|
||||
| `agents/research/git-history-analyzer.md` | `agents/research/ce-git-history-analyzer.md` | `compound-engineering:research:git-history-analyzer` | `research:ce-git-history-analyzer` |
|
||||
| `agents/research/issue-intelligence-analyst.md` | `agents/research/ce-issue-intelligence-analyst.md` | `compound-engineering:research:issue-intelligence-analyst` | `research:ce-issue-intelligence-analyst` |
|
||||
| `agents/research/learnings-researcher.md` | `agents/research/ce-learnings-researcher.md` | `compound-engineering:research:learnings-researcher` | `research:ce-learnings-researcher` |
|
||||
| `agents/research/repo-research-analyst.md` | `agents/research/ce-repo-research-analyst.md` | `compound-engineering:research:repo-research-analyst` | `research:ce-repo-research-analyst` |
|
||||
| `agents/review/adversarial-reviewer.md` | `agents/review/ce-adversarial-reviewer.md` | `compound-engineering:review:adversarial-reviewer` | `review:ce-adversarial-reviewer` |
|
||||
| `agents/review/agent-native-reviewer.md` | `agents/review/ce-agent-native-reviewer.md` | `compound-engineering:review:agent-native-reviewer` | `review:ce-agent-native-reviewer` |
|
||||
| `agents/review/api-contract-reviewer.md` | `agents/review/ce-api-contract-reviewer.md` | `compound-engineering:review:api-contract-reviewer` | `review:ce-api-contract-reviewer` |
|
||||
| `agents/review/architecture-strategist.md` | `agents/review/ce-architecture-strategist.md` | `compound-engineering:review:architecture-strategist` | `review:ce-architecture-strategist` |
|
||||
| `agents/review/cli-agent-readiness-reviewer.md` | `agents/review/ce-cli-agent-readiness-reviewer.md` | `compound-engineering:review:cli-agent-readiness-reviewer` | `review:ce-cli-agent-readiness-reviewer` |
|
||||
| `agents/review/cli-readiness-reviewer.md` | `agents/review/ce-cli-readiness-reviewer.md` | `compound-engineering:review:cli-readiness-reviewer` | `review:ce-cli-readiness-reviewer` |
|
||||
| `agents/review/code-simplicity-reviewer.md` | `agents/review/ce-code-simplicity-reviewer.md` | `compound-engineering:review:code-simplicity-reviewer` | `review:ce-code-simplicity-reviewer` |
|
||||
| `agents/review/correctness-reviewer.md` | `agents/review/ce-correctness-reviewer.md` | `compound-engineering:review:correctness-reviewer` | `review:ce-correctness-reviewer` |
|
||||
| `agents/review/data-integrity-guardian.md` | `agents/review/ce-data-integrity-guardian.md` | `compound-engineering:review:data-integrity-guardian` | `review:ce-data-integrity-guardian` |
|
||||
| `agents/review/data-migration-expert.md` | `agents/review/ce-data-migration-expert.md` | `compound-engineering:review:data-migration-expert` | `review:ce-data-migration-expert` |
|
||||
| `agents/review/data-migrations-reviewer.md` | `agents/review/ce-data-migrations-reviewer.md` | `compound-engineering:review:data-migrations-reviewer` | `review:ce-data-migrations-reviewer` |
|
||||
| `agents/review/deployment-verification-agent.md` | `agents/review/ce-deployment-verification-agent.md` | `compound-engineering:review:deployment-verification-agent` | `review:ce-deployment-verification-agent` |
|
||||
| `agents/review/dhh-rails-reviewer.md` | `agents/review/ce-dhh-rails-reviewer.md` | `compound-engineering:review:dhh-rails-reviewer` | `review:ce-dhh-rails-reviewer` |
|
||||
| `agents/review/julik-frontend-races-reviewer.md` | `agents/review/ce-julik-frontend-races-reviewer.md` | `compound-engineering:review:julik-frontend-races-reviewer` | `review:ce-julik-frontend-races-reviewer` |
|
||||
| `agents/review/kieran-python-reviewer.md` | `agents/review/ce-kieran-python-reviewer.md` | `compound-engineering:review:kieran-python-reviewer` | `review:ce-kieran-python-reviewer` |
|
||||
| `agents/review/kieran-rails-reviewer.md` | `agents/review/ce-kieran-rails-reviewer.md` | `compound-engineering:review:kieran-rails-reviewer` | `review:ce-kieran-rails-reviewer` |
|
||||
| `agents/review/kieran-typescript-reviewer.md` | `agents/review/ce-kieran-typescript-reviewer.md` | `compound-engineering:review:kieran-typescript-reviewer` | `review:ce-kieran-typescript-reviewer` |
|
||||
| `agents/review/maintainability-reviewer.md` | `agents/review/ce-maintainability-reviewer.md` | `compound-engineering:review:maintainability-reviewer` | `review:ce-maintainability-reviewer` |
|
||||
| `agents/review/pattern-recognition-specialist.md` | `agents/review/ce-pattern-recognition-specialist.md` | `compound-engineering:review:pattern-recognition-specialist` | `review:ce-pattern-recognition-specialist` |
|
||||
| `agents/review/performance-oracle.md` | `agents/review/ce-performance-oracle.md` | `compound-engineering:review:performance-oracle` | `review:ce-performance-oracle` |
|
||||
| `agents/review/performance-reviewer.md` | `agents/review/ce-performance-reviewer.md` | `compound-engineering:review:performance-reviewer` | `review:ce-performance-reviewer` |
|
||||
| `agents/review/previous-comments-reviewer.md` | `agents/review/ce-previous-comments-reviewer.md` | `compound-engineering:review:previous-comments-reviewer` | `review:ce-previous-comments-reviewer` |
|
||||
| `agents/review/project-standards-reviewer.md` | `agents/review/ce-project-standards-reviewer.md` | `compound-engineering:review:project-standards-reviewer` | `review:ce-project-standards-reviewer` |
|
||||
| `agents/review/reliability-reviewer.md` | `agents/review/ce-reliability-reviewer.md` | `compound-engineering:review:reliability-reviewer` | `review:ce-reliability-reviewer` |
|
||||
| `agents/review/schema-drift-detector.md` | `agents/review/ce-schema-drift-detector.md` | `compound-engineering:review:schema-drift-detector` | `review:ce-schema-drift-detector` |
|
||||
| `agents/review/security-reviewer.md` | `agents/review/ce-security-reviewer.md` | `compound-engineering:review:security-reviewer` | `review:ce-security-reviewer` |
|
||||
| `agents/review/security-sentinel.md` | `agents/review/ce-security-sentinel.md` | `compound-engineering:review:security-sentinel` | `review:ce-security-sentinel` |
|
||||
| `agents/review/testing-reviewer.md` | `agents/review/ce-testing-reviewer.md` | `compound-engineering:review:testing-reviewer` | `review:ce-testing-reviewer` |
|
||||
| `agents/workflow/bug-reproduction-validator.md` | `agents/workflow/ce-bug-reproduction-validator.md` | `compound-engineering:workflow:bug-reproduction-validator` | `workflow:ce-bug-reproduction-validator` |
|
||||
| `agents/workflow/lint.md` | `agents/workflow/ce-lint.md` | `compound-engineering:workflow:lint` | `workflow:ce-lint` |
|
||||
| `agents/workflow/pr-comment-resolver.md` | `agents/workflow/ce-pr-comment-resolver.md` | `compound-engineering:workflow:pr-comment-resolver` | `workflow:ce-pr-comment-resolver` |
|
||||
| `agents/workflow/spec-flow-analyzer.md` | `agents/workflow/ce-spec-flow-analyzer.md` | `compound-engineering:workflow:spec-flow-analyzer` | `workflow:ce-spec-flow-analyzer` |
|
||||
|
||||
**Total: 49 agents renamed in place (category subdirs preserved)**
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Every owned skill (except the 4 exclusions) has a `ce-` prefix in both directory name and frontmatter
|
||||
- Every agent has a `ce-` prefix in filename and frontmatter within its category subdir
|
||||
- All cross-references across skills, agents, docs, and orchestration chains use new names
|
||||
- All 3-segment agent references (`compound-engineering:<category>:<agent>`) simplified to `<category>:ce-<agent>`
|
||||
- `bun test` and `bun run release:validate` pass
|
||||
- No colon characters remain in any skill `name:` field (though sanitization infra is preserved)
|
||||
- Slash command invocations work with new names (e.g., `/ce-plan`)
|
||||
- lfg and slfg orchestration chains reference new skill and agent names (R12)
|
||||
- Grep sanity check confirms no old names persist in active code (R19)
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- **Not removing sanitization infrastructure** — `sanitizePathName()` stays as future protection for any colons
|
||||
- **Not adding backward-compatibility aliases** — No alias mechanism exists; this is a clean break
|
||||
- **Not renaming external skills** — `agent-browser` and `rclone` are upstream
|
||||
- **Not renaming lfg/slfg** — Kept as memorable exceptions
|
||||
- **Historical docs are not updated** — Past brainstorms, plans, and solutions in `docs/` reference old names; this is expected and acceptable (they're historical records). R10 applies only to active docs (README, AGENTS.md), not historical docs.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Hyphen over colon**: `ce-` not `ce:` — eliminates filesystem sanitization divergence and is more portable
|
||||
- **git-* replaces prefix**: `git-commit` -> `ce-commit` rather than `ce-git-commit` — avoids verbose double-prefix
|
||||
- **report-bug-ce normalizes**: Drops redundant `-ce` suffix -> `ce-report-bug`
|
||||
- **Agents renamed in place**: Category subdirs preserved for organization. Agent files get `ce-` prefix within their category dir. 3-segment refs drop plugin prefix: `compound-engineering:review:adversarial-reviewer` -> `review:ce-adversarial-reviewer`.
|
||||
- **Major version bump**: This is a breaking change; plugin version will bump the major version to signal it.
|
||||
- **Clean break, no aliases**: Users learn new names immediately; the old names stop working
|
||||
- **Preserve sanitization**: Keep colon-handling code even though no skills currently use colons — future-proofing
|
||||
- **git mv required**: All renames use `git mv` for history preservation. Fallback only with notification.
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- Skill directory renames via `git mv` preserve git history. Commit strategy (single vs multiple commits) deferred to planning.
|
||||
- lfg/slfg reference other skills both by short name (`/ce:plan`) and fully-qualified (`/compound-engineering:todo-resolve`) — both patterns need updating
|
||||
- README may contain stale skill references (e.g., `/sync`) — clean up during R10 documentation pass
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R9][Needs research] Exact inventory of every cross-reference in every SKILL.md, agent file, and doc that needs updating — planner should grep comprehensively
|
||||
- [Affects R8][Technical] Should directory renames be done via `git mv` in a single commit or spread across multiple commits for reviewability?
|
||||
- [Affects R14, R18][Technical] What specific test assertions reference skill names and need updating? Which test fixtures test compound-engineering behavior (should update) vs abstract colon-handling (may keep)?
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for structured implementation planning (will itself be renamed to `/ce-plan` as part of this work)
|
||||
@@ -0,0 +1,58 @@
|
||||
---
|
||||
date: 2026-03-28
|
||||
topic: ce-review-headless-mode
|
||||
---
|
||||
|
||||
# ce:review Headless Mode
|
||||
|
||||
## Problem Frame
|
||||
|
||||
ce:review currently has three modes (interactive, autofix, report-only), but all assume some level of direct user interaction or have mode-specific behaviors that don't fit programmatic callers. When another skill needs code review results as structured input, there's no way to invoke ce:review without it trying to prompt a user or applying fixes with interactive-session assumptions.
|
||||
|
||||
document-review solved this same problem in PR #425 with a `mode:headless` pattern. ce:review needs the same capability so it can be used as a utility skill by other workflows.
|
||||
|
||||
## Requirements
|
||||
|
||||
**Argument Parsing**
|
||||
- R1. Add `mode:headless` argument, parsed alongside existing mode flags
|
||||
|
||||
**Runtime Behavior**
|
||||
- R2. In headless mode, apply `safe_auto` fixes silently (matching autofix behavior)
|
||||
- R4. No `AskUserQuestion` or other interactive prompts in headless mode
|
||||
- R5. End with a clear completion signal so callers can detect when the review is done
|
||||
|
||||
**Output Format**
|
||||
- R3. Return all non-auto findings (`gated_auto`, `manual`, `advisory`) as structured text output, preserving their original classifications (severity, autofix_class, owner, confidence, evidence[], pre_existing)
|
||||
- R6. Follow document-review's structural output pattern (same envelope format, same section headings, similar parsing heuristics) while adapting per-finding fields to ce:review's own schema
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Another skill can invoke ce:review with `mode:headless`, receive structured findings, and act on them without any user interaction
|
||||
- Output envelope (section headings, severity grouping, completion signal) is structurally consistent with document-review's headless output so callers can use a similar consumption pattern for both, while per-finding fields reflect ce:review's own schema
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Not changing the existing three modes (interactive, autofix, report-only)
|
||||
- Not adding new reviewer personas or changing the review pipeline itself
|
||||
- Not building a specific caller workflow in this change — just enabling the capability
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Apply safe_auto fixes in headless**: Matches document-review's pattern where auto-fixes are applied silently and everything else is returned for the caller to handle
|
||||
- **Structural consistency with document-review, not schema compatibility**: Same envelope and section headings, but per-finding fields use ce:review's own schema (which has different autofix_class values, owner, pre_existing, etc.). Callers will need skill-aware parsing for individual findings
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R3][Technical] Exact structured output format — should it mirror document-review's text format verbatim, or adapt to ce:review's richer findings schema (which includes fields like `autofix_class`, `evidence[]`, `pre_existing` that document-review doesn't have)?
|
||||
- [Affects R1][Technical] How `mode:headless` interacts with the existing mode parsing — is it a fourth mode, or an overlay that modifies report-only/autofix behavior?
|
||||
- [Affects R5][Technical] What the completion signal looks like — "Review complete (headless mode)" text, or a more structured envelope?
|
||||
- [Affects R2][Technical] Should headless mode write run artifacts (`.context/compound-engineering/ce-review/<run-id>/`) and create durable todo files like autofix, or suppress them like report-only?
|
||||
- [Affects R1][Technical] How should headless mode handle checkout/branch switching in Stage 1? Programmatic callers may need the checkout to stay stable (like report-only) even though headless applies fixes (like autofix).
|
||||
- [Affects R1][Technical] Error behavior when headless receives conflicting mode flags (e.g., `mode:headless` + existing mode flags) or missing diff scope (no changes, no PR).
|
||||
- [Affects R2][Technical] Should headless mode support bounded re-review rounds (max_rounds: 2) like autofix, or be single-pass?
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,977 @@
|
||||
# Iterative Optimization Loop Skill — Requirements Brainstorm
|
||||
|
||||
## Problem Statement
|
||||
|
||||
CE has strong knowledge-compounding (learn from past work) and multi-agent review (quality gates), but no skill for **metric-driven iterative optimization** — the pattern where you define a measurable goal, build measurement scaffolding, then run an automated loop that tries many approaches, measures each, keeps improvements, and converges toward the best solution.
|
||||
|
||||
### Motivating Example
|
||||
|
||||
A project builds issue/PR clusters for a large open-source repo. Currently only ~20% of issues/PRs land in clusters with >1 item. The suspected achievable target is ~95%. Getting there requires testing many hypotheses:
|
||||
|
||||
- Extracting signal (unique user-entered text) from noise (PR/issue template boilerplate that makes all vectors too similar)
|
||||
- Using issue-to-PR links as a new clustering signal
|
||||
- Adjusting similarity thresholds
|
||||
- Trying different embedding models or chunking strategies
|
||||
- Combining multiple signals (text similarity + link graph + label overlap + author patterns)
|
||||
- Pre-filtering or normalizing template sections before embedding
|
||||
|
||||
No single hypothesis will get from 20% to 95%. It requires systematic experimentation — trying dozens or hundreds of variations, measuring each, and building on successes.
|
||||
|
||||
## Landscape Analysis
|
||||
|
||||
### Karpathy's AutoResearch (March 2026, 21k+ stars)
|
||||
|
||||
The simplest and most influential model. Core design:
|
||||
|
||||
- **One mutable file** (`train.py`) — the agent edits only this
|
||||
- **One immutable evaluator** (`prepare.py`) — the agent cannot touch measurement
|
||||
- **One instruction file** (`program.md`) — defines objectives, constraints, stopping criteria
|
||||
- **One metric** (`val_bpb`) — scalar, lower is better
|
||||
- **Linear keep/revert loop**: modify -> commit -> run -> measure -> if improved keep, else `git reset`
|
||||
- **History**: `results.tsv` accumulates all experiment results; git log preserves successful commits
|
||||
- **Result**: 700 experiments in 2 days, 20 discovered optimizations, ~12 experiments/hour
|
||||
|
||||
**Strengths**: Dead simple. Git-native history. Easy to understand and debug.
|
||||
**Weaknesses**: Linear — can't explore multiple directions simultaneously. Single scalar metric. No backtracking to earlier promising states.
|
||||
|
||||
### AIDE / WecoAI
|
||||
|
||||
- **Tree search** in solution space — each script is a node, LLM patches spawn children
|
||||
- Can backtrack to any previous node and explore alternatives
|
||||
- 4x more Kaggle medals than linear agents on MLE-Bench
|
||||
- More complex but better at escaping local optima
|
||||
|
||||
### Sakana AI Scientist v2
|
||||
|
||||
- **Agentic tree search** with parallel experiment execution
|
||||
- VLM feedback for analyzing figures
|
||||
- Full paper generation with automated peer review
|
||||
- Overkill for code optimization but shows the value of tree-structured exploration
|
||||
|
||||
### DSPy (Stanford)
|
||||
|
||||
- Automated prompt/weight optimization for LLM programs
|
||||
- Bayesian optimization (MIPROv2), iterative feedback (GEPA), coordinate ascent (COPRO)
|
||||
- Shows that different optimization strategies suit different problem shapes
|
||||
|
||||
### Existing Claude Code AutoResearch Forks
|
||||
|
||||
- `uditgoenka/autoresearch` — packages the pattern as a Claude Code skill
|
||||
- `autoexp` — generalized for any project with a quantifiable metric
|
||||
- Multiple teams report 50-80% improvements over 30-70 iterations overnight
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
### 1. Linear vs. Tree Search
|
||||
|
||||
| Approach | Pros | Cons |
|
||||
|---|---|---|
|
||||
| Linear (autoresearch) | Simple, easy to understand, git-native | Can't explore multiple directions, stuck in local optima |
|
||||
| Tree search (AIDE) | Can backtrack, explore alternatives | More complex state management, harder to review |
|
||||
| Hybrid: linear with manual branch points | Best of both — simple default, user chooses when to fork | Requires user interaction to fork |
|
||||
|
||||
**Recommendation**: Start with linear keep/revert (Karpathy model) as the default. Add optional "branch point" support where the user can snapshot the current best and start a new exploration direction. Each direction is its own branch. This keeps the core loop simple while allowing multi-direction exploration when needed.
|
||||
|
||||
### 2. What Gets Measured — The Three-Tier Metric Architecture
|
||||
|
||||
AutoResearch uses a single scalar metric (val_bpb). That works when you have an objective function with clear ground truth. Most real-world optimization problems don't — especially when the quality of the output requires human judgment.
|
||||
|
||||
**Key insight**: Hard scalar metrics are often the wrong optimization target. For clustering, "bigger clusters" isn't inherently better. "Fewer singletons" isn't inherently better. A solution with 35% singletons where every cluster is coherent beats a solution with 5% singletons where clusters are garbage. Hard metrics catch *degenerate* solutions; *quality* requires judgment.
|
||||
|
||||
**Three tiers**:
|
||||
|
||||
1. **Degenerate-case gates** (hard, cheap, fully automated):
|
||||
- Catch obviously broken solutions before expensive evaluation
|
||||
- Examples: "all items in 1 cluster" (degenerate merge), "all singletons" (degenerate split), "runtime > 10 minutes" (performance regression)
|
||||
- These are fast boolean checks: pass/fail. If any gate fails, the experiment is immediately reverted without running the expensive judge
|
||||
- Think of these as "sanity checks" not "optimization targets"
|
||||
|
||||
2. **LLM-as-judge quality score** (the actual optimization target):
|
||||
- For problems where quality requires judgment, this IS the primary metric
|
||||
- Cost-controlled via stratified sampling (not exhaustive)
|
||||
- Produces a scalar score the loop can optimize against
|
||||
- Can include multiple dimensions (coherence, granularity, completeness)
|
||||
- See detailed design below
|
||||
|
||||
3. **Diagnostics** (logged for understanding, not gated on):
|
||||
- Distribution stats, counts, histograms
|
||||
- Useful for understanding WHY a judge score changed
|
||||
- Examples: median cluster size, singleton %, largest cluster size, cluster count
|
||||
- Logged in the experiment record but never used for keep/revert decisions
|
||||
|
||||
**When to use which configuration**:
|
||||
|
||||
| Problem Type | Degenerate Gates | Primary Metric | Example |
|
||||
|---|---|---|---|
|
||||
| Objective function exists | Yes | Hard metric (scalar) | Build time, test pass rate, API latency |
|
||||
| Quality requires judgment | Yes | LLM-as-judge score | Clustering quality, search relevance, content generation |
|
||||
| Hybrid | Yes | Hard metric + LLM-judge as guard rail | Latency (optimize) + response quality (must not drop) |
|
||||
|
||||
**Recommendation**: Support all three tiers. The user declares whether the primary optimization target is a hard metric or an LLM-judge score. Degenerate gates always run first (cheap). Judge runs only on experiments that pass gates.
|
||||
|
||||
### 3. What the Agent Can Edit
|
||||
|
||||
AutoResearch constrains the agent to one file. This is elegant but too restrictive for most software projects.
|
||||
|
||||
**Recommendation**: Define an explicit allowlist of mutable files/directories and an explicit denylist (measurement harness, test fixtures, evaluation data). The agent operates within the allowlist. The measurement harness is immutable — the agent cannot game the metric by changing how it's measured.
|
||||
|
||||
### 4. Measurement Scaffolding First
|
||||
|
||||
This is critical and distinguishes this from "just run the code in a loop":
|
||||
|
||||
1. **Define the measurement spec** before any optimization begins
|
||||
2. **Build and validate the measurement harness** — ensure it produces reliable, reproducible results
|
||||
3. **Establish baseline** — run the harness on the current code to get starting metrics
|
||||
4. Only then begin the optimization loop
|
||||
|
||||
**Recommendation**: Make this a hard phase gate. The skill refuses to enter the optimization loop until the measurement harness passes a validation check (runs successfully, produces expected metric types, baseline is recorded).
|
||||
|
||||
### 5. History and Memory
|
||||
|
||||
What gets remembered across iterations:
|
||||
|
||||
- **Results log**: Every experiment's metrics, hypothesis, and outcome (kept/reverted)
|
||||
- **Git history**: Successful experiments are commits; branches are preserved
|
||||
- **Hypothesis log**: What was tried, why, what was learned — prevents re-trying failed approaches
|
||||
- **Strategy evolution**: As the agent learns what works, it should adapt its exploration strategy
|
||||
|
||||
**Recommendation**: A structured experiment log (YAML or JSON) that captures: iteration number, hypothesis, changes made, metrics before/after, outcome (kept/reverted/error), and learnings. The agent reads this before proposing the next hypothesis. Git branches are preserved for all kept experiments.
|
||||
|
||||
### 6. How Long It Runs
|
||||
|
||||
- AutoResearch runs "indefinitely until manually stopped"
|
||||
- Real-world needs: time budgets, iteration budgets, metric targets, or "until no improvement for N iterations"
|
||||
|
||||
**Recommendation**: Support multiple stopping criteria (any can trigger stop):
|
||||
- Target metric reached
|
||||
- Max iterations
|
||||
- Max wall-clock time
|
||||
- No improvement for N consecutive iterations
|
||||
- Manual stop (user interrupts)
|
||||
|
||||
### 7. Parallelism
|
||||
|
||||
AutoResearch is single-threaded. AIDE and AI Scientist run parallel experiments. For CE:
|
||||
|
||||
- **Phase 1 (v1)**: Single-threaded linear loop. Simple, debuggable, works with git worktrees.
|
||||
- **Phase 2 (future)**: Parallel experiments using multiple worktrees or Codex sandboxes. Each experiment is independent.
|
||||
|
||||
**Recommendation**: Start single-threaded. Design the experiment log and branching model to support parallelism later.
|
||||
|
||||
### 8. Integration with Existing CE Skills
|
||||
|
||||
The optimization loop should compose with existing CE capabilities:
|
||||
|
||||
- **`/ce:ideate`** or **`/ce:brainstorm`** to generate initial hypothesis space
|
||||
- **Learnings researcher** to check if similar optimization was done before
|
||||
- **`/ce:compound`** to capture the winning strategy as institutional knowledge after the loop completes
|
||||
- **`/ce:review`** optionally on the final winning diff before it's merged
|
||||
|
||||
## Proposed Skill: `/ce-optimize`
|
||||
|
||||
### Workflow Phases
|
||||
|
||||
```
|
||||
Phase 0: Setup
|
||||
|-- Read/create optimization spec (target metric, guard rails, mutable files, constraints)
|
||||
|-- Search learnings for prior related optimization attempts
|
||||
'-- Validate spec completeness
|
||||
|
||||
Phase 1: Measurement Scaffolding (HARD GATE - user must approve before Phase 2)
|
||||
|-- If user provides harness:
|
||||
| |-- Review docs (or document usage if undocumented)
|
||||
| |-- Run harness once against current implementation
|
||||
| '-- Confirm baseline measurement is accurate with user
|
||||
|-- If agent builds harness:
|
||||
| |-- Build measurement harness (immutable evaluator)
|
||||
| |-- Run validation: harness executes, produces expected metric types
|
||||
| '-- Establish baseline metrics
|
||||
|-- Parallelism readiness probe:
|
||||
| |-- Check for hardcoded ports -> parameterize via env var
|
||||
| |-- Check for shared DB files (SQLite, etc.) -> plan copy strategy
|
||||
| |-- Check for shared external services -> warn user
|
||||
| |-- Check for exclusive resource needs (GPU, etc.)
|
||||
| '-- Produce parallel_readiness assessment
|
||||
|-- Stability validation (if mode: repeat):
|
||||
| |-- Run harness repeat_count times
|
||||
| |-- Verify variance is within noise_threshold
|
||||
| '-- Confirm aggregation method produces stable baseline
|
||||
'-- GATE: Present baseline + parallel readiness to user. Refuse to proceed until approved.
|
||||
|
||||
Phase 2: Hypothesis Generation + Dependency Approval
|
||||
|-- Analyze the problem space (read code, understand current approach)
|
||||
|-- Generate initial hypothesis list (agent + optionally /ce:ideate)
|
||||
|-- Prioritize by expected impact and feasibility
|
||||
|-- Identify new dependencies across ALL planned hypotheses
|
||||
|-- Present dependency list for bulk approval
|
||||
'-- Record hypothesis backlog (with dep approval status per hypothesis)
|
||||
|
||||
Phase 3: Optimization Loop (repeats in parallel batches)
|
||||
|-- Select batch of hypotheses (batch_size = min(backlog, max_concurrent))
|
||||
| '-- Prefer diversity: mix different hypothesis categories per batch
|
||||
|-- For each experiment in batch (PARALLEL by default):
|
||||
| |-- Create worktree or Codex sandbox
|
||||
| |-- Copy shared resources (DB files, data files)
|
||||
| |-- Apply parameterization (ports, env vars)
|
||||
| |-- Implement hypothesis (within mutable scope)
|
||||
| |-- Run measurement harness (respecting stability config)
|
||||
| '-- Collect metrics + diff
|
||||
|-- Wait for batch completion
|
||||
|-- Evaluate results:
|
||||
| |-- Rank by primary metric improvement
|
||||
| |-- Filter by guard rails (reject any that violate)
|
||||
| |-- If best > current: KEEP (merge to optimization branch)
|
||||
| |-- If best has unapproved dep: mark deferred_needs_approval
|
||||
| '-- All others: REVERT (log results, clean up worktrees)
|
||||
|-- Handle unapproved deps:
|
||||
| '-- Set aside, don't block pipeline, batch-ask at end or check-in
|
||||
|-- Update experiment log with ALL results (kept + reverted)
|
||||
|-- Re-baseline: remaining hypotheses evaluated against new best
|
||||
|-- Generate new hypotheses based on learnings from this batch
|
||||
|-- Check stopping criteria
|
||||
'-- Next batch
|
||||
|
||||
Phase 4: Wrap-Up
|
||||
|-- Present deferred hypotheses needing dep approval (if any)
|
||||
|-- Summarize results: baseline -> final metrics, total iterations, kept improvements
|
||||
|-- Preserve ALL experiment branches for reference
|
||||
|-- Optionally run /ce:review on cumulative diff
|
||||
|-- Optionally run /ce:compound to capture winning strategy as learning
|
||||
'-- Report to user
|
||||
```
|
||||
|
||||
### Optimization Spec File Format
|
||||
|
||||
See "Updated Spec File Format" in the Resolved Design Decisions section below for the full spec with parallel execution and stability config.
|
||||
|
||||
### Experiment Log Format
|
||||
|
||||
```yaml
|
||||
# .context/compound-engineering/optimize/experiment-log.yaml
|
||||
spec: "improve-issue-clustering"
|
||||
|
||||
baseline:
|
||||
timestamp: "2026-03-29T10:00:00Z"
|
||||
gates:
|
||||
largest_cluster_pct: 0.02
|
||||
singleton_pct: 0.79
|
||||
cluster_count: 342
|
||||
runtime_seconds: 45
|
||||
diagnostics:
|
||||
singleton_pct: 0.79
|
||||
median_cluster_size: 2
|
||||
cluster_count: 342
|
||||
avg_cluster_size: 2.8
|
||||
p95_cluster_size: 7
|
||||
judge:
|
||||
mean_score: 3.1
|
||||
pct_scoring_4plus: 0.33
|
||||
mean_distinct_topics: 1.8
|
||||
singleton_false_negative_pct: 0.45 # 45% of sampled singletons should be clustered
|
||||
sample_seed: 42
|
||||
judge_cost_usd: 0.42
|
||||
|
||||
experiments:
|
||||
- iteration: 1
|
||||
batch: 1
|
||||
hypothesis: "Remove PR template boilerplate before embedding to reduce noise"
|
||||
category: "signal-extraction"
|
||||
changes:
|
||||
- file: "src/preprocessing/text_cleaner.py"
|
||||
summary: "Added template detection and removal using common PR template patterns"
|
||||
gates:
|
||||
largest_cluster_pct: 0.03
|
||||
singleton_pct: 0.62
|
||||
cluster_count: 489
|
||||
runtime_seconds: 48
|
||||
gates_passed: true
|
||||
diagnostics:
|
||||
singleton_pct: 0.62
|
||||
median_cluster_size: 3
|
||||
cluster_count: 489
|
||||
avg_cluster_size: 3.4
|
||||
judge:
|
||||
mean_score: 3.8
|
||||
pct_scoring_4plus: 0.57
|
||||
mean_distinct_topics: 1.4
|
||||
singleton_false_negative_pct: 0.31
|
||||
judge_cost_usd: 0.38
|
||||
outcome: "kept"
|
||||
primary_delta: "+0.7" # mean_score: 3.1 -> 3.8
|
||||
learnings: "Template removal significantly improved coherence. Clusters now group by actual issue content rather than shared boilerplate. Singleton rate dropped 17pp."
|
||||
commit: "abc123"
|
||||
|
||||
- iteration: 2
|
||||
batch: 1 # same batch as iteration 1 (ran in parallel)
|
||||
hypothesis: "Lower similarity threshold from 0.85 to 0.75"
|
||||
category: "clustering-algorithm"
|
||||
changes:
|
||||
- file: "config/clustering.yaml"
|
||||
summary: "Changed similarity_threshold from 0.85 to 0.75"
|
||||
gates:
|
||||
largest_cluster_pct: 0.08
|
||||
singleton_pct: 0.35
|
||||
cluster_count: 210
|
||||
runtime_seconds: 47
|
||||
gates_passed: true
|
||||
diagnostics:
|
||||
singleton_pct: 0.35
|
||||
median_cluster_size: 5
|
||||
cluster_count: 210
|
||||
judge:
|
||||
mean_score: 2.4
|
||||
pct_scoring_4plus: 0.13
|
||||
mean_distinct_topics: 3.1 # clusters covering too many unrelated topics
|
||||
singleton_false_negative_pct: 0.12
|
||||
judge_cost_usd: 0.41
|
||||
outcome: "reverted"
|
||||
primary_delta: "-0.7" # mean_score: 3.1 -> 2.4
|
||||
learnings: "Lower threshold pulled in more items but destroyed coherence. Clusters became grab-bags. The hard metrics looked good (fewer singletons!) but judge correctly identified the quality drop. Validates that singleton_pct alone is a misleading optimization target."
|
||||
|
||||
- iteration: 3
|
||||
batch: 2 # new batch, runs on top of iteration 1's changes
|
||||
hypothesis: "Use issue-to-PR link graph as additional clustering signal"
|
||||
category: "graph-signals"
|
||||
changes:
|
||||
- file: "src/clustering/signals.py"
|
||||
summary: "Added link-graph signal extraction from issue-PR references"
|
||||
- file: "src/clustering/merger.py"
|
||||
summary: "Combined text similarity with link-graph signal using weighted average"
|
||||
gates:
|
||||
largest_cluster_pct: 0.04
|
||||
singleton_pct: 0.48
|
||||
cluster_count: 520
|
||||
runtime_seconds: 52
|
||||
gates_passed: true
|
||||
diagnostics:
|
||||
singleton_pct: 0.48
|
||||
median_cluster_size: 3
|
||||
cluster_count: 520
|
||||
judge:
|
||||
mean_score: 4.1
|
||||
pct_scoring_4plus: 0.70
|
||||
mean_distinct_topics: 1.2
|
||||
singleton_false_negative_pct: 0.22
|
||||
judge_cost_usd: 0.39
|
||||
outcome: "kept"
|
||||
primary_delta: "+0.3" # mean_score: 3.8 -> 4.1 (from iteration 1 baseline)
|
||||
learnings: "Link graph is a strong complementary signal. Issues referencing the same PR are almost always related. Judge scores jumped — 70% of clusters now score 4+. Singleton false negatives dropped further."
|
||||
commit: "def456"
|
||||
|
||||
- iteration: 4
|
||||
batch: 2
|
||||
hypothesis: "Add scikit-learn HDBSCAN for hierarchical density clustering"
|
||||
category: "clustering-algorithm"
|
||||
changes: []
|
||||
gates_passed: false # not evaluated — deferred
|
||||
outcome: "deferred_needs_approval"
|
||||
deferred_reason: "Requires unapproved dependency: scikit-learn"
|
||||
learnings: "Set aside for batch approval at end of loop."
|
||||
|
||||
best:
|
||||
iteration: 3
|
||||
judge:
|
||||
mean_score: 4.1
|
||||
pct_scoring_4plus: 0.70
|
||||
total_judge_cost_usd: 1.60 # running total across all experiments
|
||||
```
|
||||
|
||||
## Hypothesis Generation Strategies
|
||||
|
||||
For the clustering example, here's the kind of hypothesis space the agent should explore:
|
||||
|
||||
### Signal Extraction
|
||||
- Remove PR/issue template boilerplate before embedding
|
||||
- Extract only user-authored text (strip auto-generated sections)
|
||||
- Weight title more heavily than body
|
||||
- Use code snippets / file paths mentioned as signals
|
||||
- Extract error messages and stack traces as high-signal features
|
||||
|
||||
### Graph-Based Signals
|
||||
- Issue-to-PR links (issues referencing same PR are related)
|
||||
- Cross-references between issues (`#123` mentions)
|
||||
- Author patterns (same author filing similar issues)
|
||||
- Label co-occurrence
|
||||
- Milestone/project board grouping
|
||||
|
||||
### Embedding & Similarity
|
||||
- Try different embedding models (different size/quality tradeoffs)
|
||||
- Chunk long issues before embedding vs. truncate vs. summarize
|
||||
- Weighted combination of multiple similarity signals
|
||||
- Asymmetric similarity (issue-to-PR vs. issue-to-issue)
|
||||
|
||||
### Clustering Algorithm
|
||||
- Adjust similarity thresholds (per-signal or combined)
|
||||
- Try hierarchical clustering vs. graph-based community detection
|
||||
- Two-pass: coarse clusters then split/merge refinement
|
||||
- Minimum cluster size constraints
|
||||
- Handle outlier issues that genuinely don't cluster
|
||||
|
||||
### Pre-processing
|
||||
- Normalize markdown formatting
|
||||
- Deduplicate near-identical issues before clustering
|
||||
- Language detection and translation for multilingual repos
|
||||
- Time-decay weighting (recent issues weighted more)
|
||||
|
||||
## Resolved Design Decisions
|
||||
|
||||
### D1: Measurement Harness Ownership -> DECIDED: Agent builds, user validates
|
||||
|
||||
The agent builds the measurement harness in Phase 1 and evaluates it against the current implementation. If the user provides an existing harness, the agent documents how to use it (or reviews existing docs), runs it once, and confirms the baseline measurement is accurate. Either way, the user reviews and approves before the loop starts. This is a hard gate.
|
||||
|
||||
### D2: Flaky Metrics -> DECIDED: User-configurable, default stable
|
||||
|
||||
The spec supports a `stability` block:
|
||||
|
||||
```yaml
|
||||
measurement:
|
||||
command: "python evaluate.py"
|
||||
stability:
|
||||
mode: "stable" # default: run once, trust the result
|
||||
# mode: "repeat" # run N times, aggregate
|
||||
# repeat_count: 5 # how many runs
|
||||
# aggregation: "median" # median | mean | min | max | custom
|
||||
# noise_threshold: 0.02 # improvement must exceed this to count
|
||||
```
|
||||
|
||||
When `mode: repeat`, the harness runs `repeat_count` times. The `aggregation` function reduces results to a single value per metric. The `noise_threshold` prevents accepting improvements within the noise floor. Default is `stable` — run once, trust it.
|
||||
|
||||
### D3: New Dependencies -> DECIDED: Pre-approve expected, defer surprises
|
||||
|
||||
During Phase 2 (Hypothesis Generation), the agent outlines expected new dependencies across all planned variations and gets bulk approval up front. If an experiment during the loop discovers it needs an unapproved dependency, the agent:
|
||||
1. Sets that hypothesis aside (marks it `deferred_needs_approval` in the experiment log)
|
||||
2. Continues with other hypotheses that don't need new deps
|
||||
3. At the end of the loop (or at a user check-in), presents the deferred hypotheses and their dep requirements for batch approval
|
||||
4. If approved, those hypotheses enter the next iteration batch
|
||||
|
||||
This prevents blocking the pipeline on interactive approval during long unattended runs.
|
||||
|
||||
### D4: LLM-as-Judge -> DECIDED: Include in v1 (cost-controlled via sampling)
|
||||
|
||||
LLM-as-judge is essential for problems where quality requires judgment — it's often the *actual* optimization target, not a nice-to-have. Hard metrics catch degenerate cases but can't tell you whether clusters are coherent or search results are relevant.
|
||||
|
||||
**Cost control via stratified sampling**:
|
||||
- Don't judge every output item — sample a representative set
|
||||
- Stratified sampling ensures coverage of edge cases (small clusters, large clusters, singletons)
|
||||
- Default: ~30 samples per evaluation (configurable)
|
||||
- At ~$0.01-0.03 per judgment call, 30 samples = ~$0.30-0.90 per experiment
|
||||
- Over 100 experiments = $30-90 total — manageable
|
||||
|
||||
**Sampling strategy**:
|
||||
```yaml
|
||||
judge:
|
||||
sample_size: 30
|
||||
stratification:
|
||||
- bucket: "small" # 2-3 items
|
||||
count: 10
|
||||
- bucket: "medium" # 4-10 items
|
||||
count: 10
|
||||
- bucket: "large" # 11+ items
|
||||
count: 10
|
||||
# For singletons: sample 10 and ask "should any of these be in a cluster?"
|
||||
singleton_sample: 10
|
||||
```
|
||||
|
||||
**Rubric-based scoring** (user-defined, per problem):
|
||||
```yaml
|
||||
judge:
|
||||
rubric: |
|
||||
Rate this cluster 1-5:
|
||||
- 5: All items clearly about the same issue/feature
|
||||
- 4: Strong theme, minor outliers
|
||||
- 3: Related but covers 2-3 sub-topics
|
||||
- 2: Weak connection
|
||||
- 1: Unrelated items grouped together
|
||||
|
||||
Also answer:
|
||||
- How many distinct sub-topics does this cluster represent?
|
||||
- Should any items be removed from this cluster?
|
||||
|
||||
scoring:
|
||||
primary: "mean_score" # mean of 1-5 ratings
|
||||
secondary: "pct_scoring_4plus" # % of samples scoring 4 or 5
|
||||
output_format: "json" # {"score": 4, "distinct_topics": 1, "remove_items": []}
|
||||
```
|
||||
|
||||
**Judge execution order**:
|
||||
1. Run degenerate-case gates (fast, free) -- reject obviously broken solutions
|
||||
2. Run hard metrics (fast, free) -- collect diagnostics
|
||||
3. Only if gates pass: run LLM-as-judge on sampled outputs (slow, costs money)
|
||||
4. Keep/revert decision uses judge score as primary metric
|
||||
|
||||
**Judge consistency**:
|
||||
- Use the same sample indices across experiments when possible (same random seed)
|
||||
- This reduces noise from sample variance — you're comparing the same clusters across runs
|
||||
- When the output structure changes (different number of clusters), re-sample but log the seed change
|
||||
|
||||
**Judge model selection**:
|
||||
- Default: Haiku (fast, cheap, good enough for rubric-based scoring)
|
||||
- Option: Sonnet for nuanced judgment (2-3x cost)
|
||||
- The judge prompt is part of the immutable measurement harness — the agent cannot modify it
|
||||
|
||||
**Singleton evaluation** (the non-obvious case):
|
||||
- Low singleton % isn't automatically good. High singleton % isn't automatically bad.
|
||||
- Sample singletons and ask the judge: "Given these other clusters, should this item be in one of them? Which one? Or is it genuinely unique?"
|
||||
- This catches false-negative clustering (items that should cluster but don't) AND validates true singletons
|
||||
|
||||
### D5: Codex Support -> DECIDED: Include from v1
|
||||
|
||||
Based on patterns from PRs #364/#365 in the compound-engineering plugin:
|
||||
|
||||
**Dispatch pattern**: Write experiment prompt to a temp file, pipe to `codex exec` via stdin:
|
||||
```bash
|
||||
cat /tmp/optimize-exp-XXXXX.txt | codex exec --skip-git-repo-check - 2>&1
|
||||
```
|
||||
|
||||
**Security posture**: User selects once per session (same as ce-work-beta):
|
||||
- Workspace write (`--full-auto`)
|
||||
- Full access (`--dangerously-bypass-approvals-and-sandbox`)
|
||||
|
||||
**Result collection**: Inspect working directory diff after `codex exec` completes. No structured result format — Codex writes files, orchestrator reads the diff and runs the measurement harness.
|
||||
|
||||
**Guard rails**:
|
||||
- Check for `CODEX_SANDBOX` / `CODEX_SESSION_ID` env vars to prevent recursive delegation
|
||||
- 3 consecutive delegate failures auto-disable Codex for remaining experiments
|
||||
- Orchestrator retains control of git operations, measurement, and keep/revert decisions
|
||||
|
||||
### D6: Parallel Execution -> DECIDED: Parallel by default
|
||||
|
||||
Experiments run in parallel by default. The user can specify serial execution if the system under test requires it. The skill actively probes for parallelism blockers.
|
||||
|
||||
See full parallel execution design below.
|
||||
|
||||
---
|
||||
|
||||
## Parallel Execution Design
|
||||
|
||||
### Default: Parallel Experiments
|
||||
|
||||
The optimization loop dispatches multiple experiments simultaneously unless the user explicitly requests serial execution. This is the primary throughput lever — running 4-8 experiments in parallel vs. 1 at a time means 4-8x more iterations per hour.
|
||||
|
||||
### Isolation Strategy
|
||||
|
||||
Each parallel experiment needs full filesystem isolation. Two mechanisms, selectable per session:
|
||||
|
||||
**Local worktrees** (default):
|
||||
```
|
||||
.claude/worktrees/optimize-exp-001/ # full repo copy
|
||||
.claude/worktrees/optimize-exp-002/
|
||||
.claude/worktrees/optimize-exp-003/
|
||||
```
|
||||
- Created via `git worktree add` with a unique branch per experiment
|
||||
- Each worktree gets its own copy of shared resources (see below)
|
||||
- Cleaned up after measurement: kept experiments merge to the optimization branch, reverted experiments have their worktree removed
|
||||
|
||||
**Codex sandboxes** (opt-in):
|
||||
- Each experiment dispatched as an independent `codex exec` invocation
|
||||
- Codex provides built-in filesystem isolation
|
||||
- Orchestrator collects diffs after completion
|
||||
- Best for maximizing parallelism (no local resource limits)
|
||||
|
||||
**Hybrid** (future):
|
||||
- Use Codex for implementation, local worktree for measurement
|
||||
- Useful when measurement requires local resources (GPU, specific hardware, large datasets)
|
||||
|
||||
### Parallelism Blocker Detection (Phase 1)
|
||||
|
||||
During Phase 1 (Measurement Scaffolding), the skill actively probes for common parallelism blockers:
|
||||
|
||||
**Port conflicts**:
|
||||
- Run the measurement harness and check if it binds to fixed ports
|
||||
- Search config and code for hardcoded port numbers
|
||||
- If found: parameterize via environment variable (e.g., `PORT=0` for random, or `BASE_PORT + experiment_index`)
|
||||
- Add to spec: `parallel.port_strategy: "parameterized"` with the env var name
|
||||
|
||||
**Shared database files**:
|
||||
- Check for SQLite databases, local file-based stores
|
||||
- If found: each experiment gets a copy of the database in its worktree
|
||||
- Cleanup: remove copies after measurement
|
||||
- Add to spec: `parallel.shared_files: ["data/clusters.db"]` with copy strategy
|
||||
|
||||
**Shared external services**:
|
||||
- Check if the system writes to a shared external database, API, or queue
|
||||
- If found: warn user, suggest serial mode or test database isolation
|
||||
- This is a hard blocker for parallel unless the user confirms isolation
|
||||
|
||||
**Resource contention**:
|
||||
- Check for GPU usage, large memory requirements
|
||||
- If the system needs exclusive access to a resource, serial mode is required
|
||||
- Add to spec: `parallel.exclusive_resources: ["gpu"]`
|
||||
|
||||
**Detection output**: Phase 1 produces a `parallel_readiness` assessment:
|
||||
```yaml
|
||||
parallel:
|
||||
mode: "parallel" # parallel | serial | user-decision
|
||||
max_concurrent: 4 # default, adjustable
|
||||
blockers_found: [] # or list of issues
|
||||
mitigations_applied:
|
||||
- type: "port_parameterization"
|
||||
env_var: "EVAL_PORT"
|
||||
strategy: "base_port_plus_index"
|
||||
base: 9000
|
||||
- type: "database_copy"
|
||||
source: "data/clusters.db"
|
||||
strategy: "copy_per_worktree"
|
||||
blockers_unresolved: [] # these force serial unless user resolves
|
||||
```
|
||||
|
||||
### Parallel Loop Mechanics
|
||||
|
||||
```
|
||||
Orchestrator (main branch)
|
||||
|
|
||||
|-- Batch N experiments from hypothesis backlog
|
||||
| (batch_size = min(backlog_size, max_concurrent))
|
||||
|
|
||||
|-- For each experiment in batch (parallel):
|
||||
| |-- Create worktree / Codex sandbox
|
||||
| |-- Copy shared resources (DB files, etc.)
|
||||
| |-- Apply parameterization (ports, env vars)
|
||||
| |-- Implement hypothesis (agent edits mutable files)
|
||||
| |-- Run measurement harness
|
||||
| |-- Collect metrics + diff
|
||||
| |-- Clean up shared resource copies
|
||||
|
|
||||
|-- Wait for all experiments in batch to complete
|
||||
|
|
||||
|-- Evaluate results:
|
||||
| |-- Rank by primary metric improvement
|
||||
| |-- Filter by guard rails
|
||||
| |-- Select best experiment that passes all guards
|
||||
| |-- If best > current best: KEEP (merge to optimization branch)
|
||||
| |-- All others: REVERT (remove worktrees, log results)
|
||||
| |-- If none improve: log all results, advance to next batch
|
||||
|
|
||||
|-- Update experiment log with all results (kept + reverted)
|
||||
|-- Update hypothesis backlog based on learnings from ALL experiments
|
||||
|-- Check stopping criteria
|
||||
|-- Next batch
|
||||
```
|
||||
|
||||
### Parallel-Aware Keep/Revert
|
||||
|
||||
With parallel experiments, multiple experiments might improve the metric but conflict with each other (they modify the same files in incompatible ways). Resolution strategy:
|
||||
|
||||
1. **Non-overlapping changes**: If the best experiment's changes don't overlap with the second-best, consider keeping both (merge sequentially, re-measure after merge to confirm)
|
||||
2. **Overlapping changes**: Keep only the best. Log the second-best as "promising but conflicts with experiment N" for potential future retry on top of the new baseline
|
||||
3. **Re-baseline**: After keeping any experiment, all remaining experiments in the batch that were reverted get re-measured mentally against the new baseline — their hypotheses go back into the backlog for potential retry
|
||||
|
||||
### Experiment Prompt Template (for Codex dispatch)
|
||||
|
||||
```markdown
|
||||
# Optimization Experiment #{iteration}
|
||||
|
||||
## Context
|
||||
You are running experiment #{iteration} for optimization target: {spec.name}
|
||||
Current best metrics: {current_best_metrics}
|
||||
Baseline metrics: {baseline_metrics}
|
||||
|
||||
## Your Hypothesis
|
||||
{hypothesis.description}
|
||||
|
||||
## What To Change
|
||||
Modify ONLY files in the mutable scope:
|
||||
{spec.scope.mutable}
|
||||
|
||||
DO NOT modify:
|
||||
{spec.scope.immutable}
|
||||
|
||||
## Constraints
|
||||
{spec.constraints}
|
||||
{approved_dependencies}
|
||||
|
||||
## Previous Experiments (for context)
|
||||
{recent_experiment_summaries}
|
||||
|
||||
## Instructions
|
||||
1. Implement the hypothesis
|
||||
2. Do NOT run the measurement harness (orchestrator handles this)
|
||||
3. Do NOT commit (orchestrator handles this)
|
||||
4. Run `git diff --stat` when done so the orchestrator can see your changes
|
||||
```
|
||||
|
||||
### Concurrency Limits
|
||||
|
||||
```yaml
|
||||
parallel:
|
||||
max_concurrent: 4 # default for local worktrees
|
||||
# max_concurrent: 8 # default for Codex (no local resource limits)
|
||||
codex_rate_limit: 10 # max Codex invocations per minute
|
||||
worktree_cleanup: "immediate" # or "batch" (clean up after full batch)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Updated Spec File Format
|
||||
|
||||
### Example A: Hard-Metric Primary (build performance, test pass rate)
|
||||
|
||||
```yaml
|
||||
# .context/compound-engineering/optimize/spec.yaml
|
||||
name: "reduce-build-time"
|
||||
description: "Reduce CI build time while maintaining test pass rate"
|
||||
|
||||
metric:
|
||||
primary:
|
||||
type: "hard" # hard | judge
|
||||
name: "build_time_seconds"
|
||||
direction: "minimize"
|
||||
baseline: null # filled by Phase 1
|
||||
target: 60 # optional target to stop at
|
||||
|
||||
degenerate_gates: # fast boolean checks, run first
|
||||
- name: "test_pass_rate"
|
||||
check: ">= 1.0" # all tests must pass
|
||||
- name: "build_exits_zero"
|
||||
check: "== true"
|
||||
|
||||
diagnostics:
|
||||
- name: "cache_hit_rate"
|
||||
- name: "slowest_step"
|
||||
- name: "total_test_count"
|
||||
|
||||
measurement:
|
||||
command: "python evaluate.py"
|
||||
timeout_seconds: 600
|
||||
output_format: "json"
|
||||
stability:
|
||||
mode: "stable"
|
||||
```
|
||||
|
||||
### Example B: LLM-Judge Primary (clustering quality, search relevance)
|
||||
|
||||
```yaml
|
||||
# .context/compound-engineering/optimize/spec.yaml
|
||||
name: "improve-issue-clustering"
|
||||
description: "Improve coherence and coverage of issue/PR clusters"
|
||||
|
||||
metric:
|
||||
primary:
|
||||
type: "judge"
|
||||
name: "cluster_coherence"
|
||||
direction: "maximize"
|
||||
baseline: null
|
||||
target: 4.2 # mean judge score (1-5 scale)
|
||||
|
||||
degenerate_gates: # cheap checks that reject obviously broken solutions
|
||||
- name: "largest_cluster_pct"
|
||||
description: "% of all items in the single largest cluster"
|
||||
check: "<= 0.10" # if >10% of items are in one cluster, it's degenerate
|
||||
- name: "singleton_pct"
|
||||
description: "% of items that are singletons"
|
||||
check: "<= 0.80" # if >80% singletons, clustering isn't working at all
|
||||
- name: "cluster_count"
|
||||
check: ">= 10" # fewer than 10 clusters for 18k items is degenerate
|
||||
- name: "runtime_seconds"
|
||||
check: "<= 600"
|
||||
|
||||
diagnostics: # logged for understanding, never gated on
|
||||
- name: "singleton_pct" # note: same metric can be diagnostic AND gate
|
||||
- name: "median_cluster_size"
|
||||
- name: "cluster_count"
|
||||
- name: "avg_cluster_size"
|
||||
- name: "p95_cluster_size"
|
||||
|
||||
judge:
|
||||
model: "haiku" # haiku (cheap) | sonnet (nuanced)
|
||||
sample_size: 30
|
||||
stratification:
|
||||
- bucket: "small" # 2-3 items per cluster
|
||||
count: 10
|
||||
- bucket: "medium" # 4-10 items
|
||||
count: 10
|
||||
- bucket: "large" # 11+ items
|
||||
count: 10
|
||||
singleton_sample: 10 # also sample singletons to check false negatives
|
||||
sample_seed: 42 # fixed seed for cross-experiment consistency
|
||||
rubric: |
|
||||
Rate this cluster 1-5:
|
||||
- 5: All items clearly about the same issue/feature
|
||||
- 4: Strong theme, minor outliers
|
||||
- 3: Related but covers 2-3 sub-topics
|
||||
- 2: Weak connection
|
||||
- 1: Unrelated items grouped together
|
||||
|
||||
Also answer in JSON:
|
||||
- "score": your 1-5 rating
|
||||
- "distinct_topics": how many distinct sub-topics this cluster represents
|
||||
- "outlier_count": how many items don't belong
|
||||
singleton_rubric: |
|
||||
This item is currently a singleton (not in any cluster).
|
||||
Given the cluster titles listed below, should this item be in one of them?
|
||||
|
||||
Answer in JSON:
|
||||
- "should_cluster": true/false
|
||||
- "best_cluster_id": cluster ID it belongs in (or null)
|
||||
- "confidence": 1-5 how confident you are
|
||||
scoring:
|
||||
primary: "mean_score" # what the loop optimizes
|
||||
secondary:
|
||||
- "pct_scoring_4plus" # % of samples scoring 4+
|
||||
- "mean_distinct_topics" # lower is better (tighter clusters)
|
||||
- "singleton_false_negative_pct" # % of sampled singletons that should be clustered
|
||||
|
||||
measurement:
|
||||
command: "python evaluate.py" # outputs JSON with gate + diagnostic metrics
|
||||
timeout_seconds: 600
|
||||
output_format: "json"
|
||||
stability:
|
||||
mode: "stable"
|
||||
|
||||
scope:
|
||||
mutable:
|
||||
- "src/clustering/"
|
||||
- "src/preprocessing/"
|
||||
- "config/clustering.yaml"
|
||||
immutable:
|
||||
- "evaluate.py"
|
||||
- "tests/fixtures/"
|
||||
- "data/"
|
||||
|
||||
execution:
|
||||
mode: "parallel"
|
||||
backend: "worktree"
|
||||
max_concurrent: 4
|
||||
codex_security: null
|
||||
|
||||
parallel:
|
||||
port_strategy: null
|
||||
shared_files: ["data/clusters.db"]
|
||||
exclusive_resources: []
|
||||
|
||||
dependencies:
|
||||
approved: []
|
||||
|
||||
constraints:
|
||||
- "Do not change the output format of clusters"
|
||||
- "Preserve backward compatibility with existing cluster consumers"
|
||||
|
||||
stopping:
|
||||
max_iterations: 100
|
||||
max_hours: 8
|
||||
plateau_iterations: 10
|
||||
target_reached: true
|
||||
```
|
||||
|
||||
### Evaluation Execution Order (per experiment)
|
||||
|
||||
```
|
||||
1. Run measurement command (evaluate.py)
|
||||
-> Produces JSON with gate metrics + diagnostics
|
||||
-> Fast, free
|
||||
|
||||
2. Check degenerate gates
|
||||
-> If ANY gate fails: REVERT immediately, log as "degenerate"
|
||||
-> Do NOT run the judge (saves money)
|
||||
|
||||
3. If primary type is "judge": Run LLM-as-judge
|
||||
-> Sample outputs according to stratification config
|
||||
-> Send each sample to judge model with rubric
|
||||
-> Aggregate scores per scoring config
|
||||
-> This is the number the loop optimizes against
|
||||
|
||||
4. Keep/revert decision
|
||||
-> Based on primary metric (hard or judge score)
|
||||
-> Must also pass all degenerate gates (already checked in step 2)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Open Questions (Remaining)
|
||||
|
||||
1. **Should the agent propose hypotheses, or should the user provide them?**
|
||||
- Both — agent generates from analysis, user can inject ideas, agent prioritizes
|
||||
|
||||
2. **Judge calibration across experiments**
|
||||
- LLM judges can drift or be inconsistent across calls
|
||||
- Should we include "anchor samples" — a fixed set of clusters with known scores — in every judge batch to detect drift?
|
||||
- If anchor scores shift >0.5 from baseline, re-calibrate or flag for user review
|
||||
|
||||
3. **Judge rubric iteration**
|
||||
- The rubric itself might need improvement after seeing early results
|
||||
- But changing the rubric mid-loop invalidates comparisons to earlier experiments
|
||||
- Solution: if rubric changes, re-judge the current best with the new rubric to re-baseline?
|
||||
|
||||
4. **Relationship to `/lfg` and `/slfg`?**
|
||||
- `/lfg` is autonomous execution of a single task
|
||||
- `/ce-optimize` is autonomous execution of an iterative search
|
||||
- `/ce-optimize` can delegate each experiment to Codex (decided D5)
|
||||
- Local experiments use subagent dispatch similar to `/ce:review`
|
||||
|
||||
5. **Branch strategy details?**
|
||||
- Main optimization branch: `optimize/<spec-name>`
|
||||
- Each kept experiment is a commit on that branch
|
||||
- Branch points create `optimize/<spec-name>/direction-<N>`
|
||||
- All branches preserved for later reference and comparison
|
||||
|
||||
6. **Batch size adaptation?**
|
||||
- Should the batch size grow/shrink based on success rate?
|
||||
- High success rate -> larger batches (more exploration)
|
||||
- Low success rate -> smaller batches (more focused)
|
||||
- Or keep it simple and let the user tune `max_concurrent`
|
||||
|
||||
7. **Hypothesis diversity within a batch?**
|
||||
- Should parallel experiments in the same batch be intentionally diverse?
|
||||
- E.g., one threshold tweak + one new signal + one preprocessing change
|
||||
- Or let the prioritization algorithm decide naturally?
|
||||
|
||||
8. **Judge cost budgets?**
|
||||
- Should the spec include a `max_judge_cost_usd` budget?
|
||||
- When budget is exhausted, switch to hard-metrics-only mode or stop?
|
||||
- Or just track cost in the log and let the user decide?
|
||||
|
||||
## What Makes This Different From "Just Using AutoResearch"
|
||||
|
||||
AutoResearch is designed for ML training on a single GPU. CE's version needs to handle:
|
||||
|
||||
1. **Multi-file changes** — real code changes span multiple files
|
||||
2. **Complex metrics** — not just one scalar, but primary + guard rails + diagnostics
|
||||
3. **Varied execution environments** — not just `python train.py` but arbitrary commands
|
||||
4. **Integration with existing workflows** — learnings, review, ideation
|
||||
5. **User-in-the-loop** — pause for approval on scope-expanding changes, inject new hypotheses
|
||||
6. **Knowledge capture** — document what worked and why for the team, not just for the agent's context
|
||||
7. **Non-ML domains** — clustering, search quality, API performance, test coverage, build times, etc.
|
||||
|
||||
## Success Criteria for This Skill
|
||||
|
||||
- User can define an optimization target in <15 minutes
|
||||
- Measurement scaffolding is validated before the loop starts
|
||||
- Loop runs unattended for hours, producing measurable improvement
|
||||
- All experiments are preserved in git for later reference
|
||||
- The winning strategy is documented as a learning
|
||||
- A human reviewing the experiment log can understand what was tried and why
|
||||
- The skill handles failures gracefully (bad experiments don't corrupt state)
|
||||
|
||||
## Lessons from First Run (2026-03-30)
|
||||
|
||||
The skill was tested on the clustering problem for ~90 minutes. Results:
|
||||
|
||||
**What worked:**
|
||||
- Ran 16 experiments, improved multi_member_pct from 31.4% to 72.1%
|
||||
- Explored multiple algorithm modes (basic, refine, bounded union-find)
|
||||
- Correctly identified size-bounded union-find as the winning approach
|
||||
- Hypothesis diversity across parameter sweeps was reasonable
|
||||
|
||||
**What failed:**
|
||||
|
||||
1. **No LLM-as-judge evaluation** -- The skill defaulted to `type: hard` and optimized `multi_member_pct` as the primary metric. This is a proxy metric that can mislead. A solution that puts 72% of items in clusters is useless if the clusters are incoherent. The Phase 0.2 interactive spec creation did not actively probe whether the target was qualitative or guide toward judge mode.
|
||||
|
||||
**Fix applied**: Phase 0.2 now includes explicit qualitative vs quantitative detection, concrete examples of when to use each type, sampling strategy guidance with walkthrough questions, and rubric design guidance. The skill now strongly recommends `type: judge` for qualitative targets.
|
||||
|
||||
2. **No disk persistence** -- Experiment results existed only in the conversation context (as a table dumped to chat). If the session had been compacted or crashed, all 90 minutes of results would have been lost. This directly contradicts the Karpathy model where `results.tsv` is written after every single experiment.
|
||||
|
||||
**Fix applied**: Added mandatory disk checkpoints (CP-0 through CP-5) at every phase boundary. Each checkpoint requires a write-then-verify cycle: write the file, read it back, confirm the content is present. The persistence discipline section now explicitly states "If you produce a results table in the conversation without writing those results to disk first, you have a bug."
|
||||
|
||||
3. **Sampling strategy not prompted** -- Even if `type: judge` had been used, the skill didn't guide the user through designing a sampling strategy. For clustering, the user wants stratified sampling across: top clusters by size (check for mega-clusters), mid-range clusters (representative quality), small clusters (check if connections are real), and singletons (check for false negatives). This domain-specific guidance was missing.
|
||||
|
||||
**Fix applied**: Phase 0.2 now walks through sampling strategy design with concrete questions and domain-specific examples.
|
||||
|
||||
**Key takeaway**: The skill had all the right machinery in the schema and templates but the SKILL.md instructions didn't forcefully enough guide the agent toward using that machinery. Instructions that say "if judge type, do X" are ignored when the skill silently defaults to hard type. Instructions need to actively detect the right path and guide toward it.
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Re-test with the clustering use case using `type: judge` to validate the judge loop works end-to-end
|
||||
2. Verify disk persistence works on a long run (2+ hours) with context compaction
|
||||
3. Test with a second use case (e.g., prompt optimization, build performance) to validate generality
|
||||
4. Consider adding anchor samples for judge calibration across experiments (Open Question #2)
|
||||
5. Consider judge cost budgets (Open Question #8)
|
||||
@@ -0,0 +1,82 @@
|
||||
---
|
||||
date: 2026-03-29
|
||||
topic: testing-addressed-gate
|
||||
---
|
||||
|
||||
# Close the Testing Gap in ce:work and ce:plan
|
||||
|
||||
## Problem Frame
|
||||
|
||||
ce:work has extensive testing instructions -- test discovery, test-first execution posture, system-wide test checks, and a test scenario completeness checklist. But two narrow gaps let untested behavioral changes slip through silently:
|
||||
|
||||
1. **ce:work's quality gate says "All tests pass"** -- which is vacuously true when no tests exist. A passing empty test suite is indistinguishable from a passing comprehensive one. "No tests" can be a deliberate decision or an accidental omission, and the skill doesn't distinguish between the two.
|
||||
|
||||
2. **ce:plan allows blank test scenarios without annotation** -- when a plan unit has no test scenarios, it's ambiguous whether the planner assessed testing and determined none were needed, or simply didn't think about it. ce:plan already requires test scenarios for feature-bearing units (Plan Quality Bar, Phase 5.1 review), but non-feature-bearing units legitimately omit them, and the template doesn't require saying so.
|
||||
|
||||
The testing-reviewer in ce:review catches some of these after the fact by examining diffs for untested branches and missing edge case coverage. But it doesn't specifically flag the broader pattern: behavioral changes with no corresponding test additions at all.
|
||||
|
||||
The existing testing instructions are thorough but generic. The gap isn't volume of instructions -- it's specificity at the right moments. This targets focused changes at three layers: planning (ce:plan annotation), execution (ce:work per-task deliberation), and review (testing-reviewer detection).
|
||||
|
||||
## Requirements
|
||||
|
||||
**ce:plan -- Handle the Blank Case**
|
||||
|
||||
- R1. When a plan unit has no test scenarios, the planner should annotate why (e.g., "Test expectation: none -- config-only, no behavioral change") rather than leaving the field blank
|
||||
- R2. A blank or missing test scenarios field on a feature-bearing unit should be treated as incomplete during ce:plan's Phase 5.1 review, not silently accepted
|
||||
|
||||
---
|
||||
|
||||
**ce:work -- Per-Task Testing Deliberation**
|
||||
|
||||
- R3. Before marking a task done, ce:work's execution loop should include an explicit testing deliberation: did this task change behavior? If yes, were tests written or updated? If no tests were added, why not? This is a prompt for deliberation at the point of action, not a formal artifact
|
||||
- R4. The Phase 3 quality checklist item "Tests pass (run project's test command)" and the Final Validation item "All tests pass" should both be updated to "Testing addressed -- tests pass AND new/changed behavior has corresponding test coverage (or an explicit justification for why tests are not needed)"
|
||||
- R5. Apply R3 and R4 to ce:work-beta (AGENTS.md requires explicit sync decisions for beta counterparts)
|
||||
|
||||
---
|
||||
|
||||
**testing-reviewer -- Flag the Missing-Test Pattern**
|
||||
|
||||
- R6. The testing-reviewer agent should add a new check: when the diff contains behavioral code changes (new logic branches, state mutations, API changes) with zero corresponding test additions or modifications, flag it as a finding
|
||||
- R7. This check complements the existing checks (untested branches, weak assertions, brittle tests, missing edge cases) -- it catches the case those miss: no tests at all for new behavior
|
||||
|
||||
**Contract Tests -- Practice What We Preach**
|
||||
|
||||
- R8. Add contract tests verifying each behavioral change ships as intended. Following the existing pattern in `pipeline-review-contract.test.ts` and `review-skill-contract.test.ts` (string assertions against skill/agent file content):
|
||||
- ce:work includes per-task testing deliberation in the execution loop (R3)
|
||||
- ce:work checklist says "Testing addressed", not "Tests pass" or "All tests pass" (R4)
|
||||
- ce:work-beta mirrors the testing deliberation and checklist changes (R5)
|
||||
- ce:plan Phase 5.1 review treats blank test scenarios on feature-bearing units as incomplete (R2)
|
||||
- testing-reviewer agent includes the behavioral-changes-with-no-test-additions check (R6)
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- A diff with behavioral changes and no test changes gets flagged by the testing-reviewer (R6) -- the detective layer catches it on real artifacts
|
||||
- ce:plan units without test scenarios either have an explicit annotation or get flagged during plan review (R1-R2) -- the preventive layer operates at planning time
|
||||
- ce:work's execution loop prompts testing deliberation per task, and the checklist makes the agent explicitly consider whether testing was addressed, not just whether the suite is green (R3-R4)
|
||||
- "No tests needed" with justification remains a valid outcome -- the goal is deliberate decisions, not forced ceremony
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Not adding CI-level enforcement or programmatic gates -- these are prompt-level changes
|
||||
- Not adding new abstractions like "testing assessment artifacts" or structured output schemas
|
||||
- Not mandating coverage thresholds or specific testing frameworks
|
||||
- Not changing the testing-reviewer's output format -- adding one check within its existing review protocol
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Layered approach -- deliberation + detection**: ce:work's per-task deliberation (R3) prompts the agent to think about testing at the point of action. The testing-reviewer (R6) operates on the actual diff as a backstop. Instruction specificity at the right moment matters -- "did you address testing for this task?" is a much more targeted prompt than "tests pass."
|
||||
- **Targeted edits over a new system**: Rather than introducing a "testing assessment gate" abstraction, make focused changes to ce:plan, ce:work, and testing-reviewer that close the identified gaps.
|
||||
- **Deliberate omission is a first-class outcome**: "No tests needed" with justification is valid. The goal is making "no tests" a deliberate decision, not an accidental one.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R1][Technical] What's the lightest-weight annotation for plan units that genuinely need no tests -- a field, a comment, or a convention?
|
||||
- [Affects R6][Needs research] Review the testing-reviewer's current check implementation to determine where the new "behavioral changes with no test changes" check fits in its analysis protocol
|
||||
- [Affects R3][Technical] Where in ce:work's execution loop (Phase 2 task loop) does the testing deliberation prompt fit -- after "Run tests after changes" or as part of "Mark task as completed"?
|
||||
- [Affects R4-R5][Resolved] ce:work's Phase 3 checklist is plaintext markdown in SKILL.md (line ~433 and ~289). ce:work-beta has the same pattern. The change is editing bullet points, no dynamic infrastructure.
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,65 @@
|
||||
---
|
||||
date: 2026-03-30
|
||||
topic: cli-readiness-review-persona
|
||||
---
|
||||
|
||||
# CLI Agent-Readiness Review Persona in ce:review
|
||||
|
||||
## Problem Frame
|
||||
|
||||
The `cli-agent-readiness-reviewer` agent exists as a standalone deep-audit tool, but developers only benefit from it if they know it exists and invoke it explicitly. Most CLI code gets reviewed through `ce:review`, which has no CLI-specific lens. Agent-readiness issues (prose-only output, missing `--json`, interactive prompts without bypass, unbounded list output) ship undetected because no review persona covers them.
|
||||
|
||||
Adding CLI readiness as a conditional persona in ce:review makes this expertise automatic -- the developer runs their normal review and gets CLI agent-readiness findings alongside security, performance, and other concerns.
|
||||
|
||||
## Requirements
|
||||
|
||||
**Persona Selection**
|
||||
|
||||
- R1. ce:review's orchestrator selects the CLI readiness persona based on diff analysis (same pattern as security-reviewer, performance-reviewer, etc.) -- not always-on
|
||||
- R2. Activation signals: diff touches CLI command definitions, argument parsing, CLI framework usage, or command handler implementations. The orchestrator uses judgment (not keyword matching), consistent with how all other conditional personas are activated
|
||||
- R3. Non-overlapping scope with agent-native-reviewer: CLI readiness evaluates CLI command structure and agent-friendliness; agent-native evaluates UI/agent tool parity. Both may activate on the same diff if it touches both CLI and UI code -- their findings address different concerns. Overlap is possible and handled during synthesis rather than prevented mechanically
|
||||
|
||||
**Persona Behavior**
|
||||
|
||||
- R4. Once dispatched, the persona self-scopes: identifies the framework, detects changed commands from the diff, and evaluates against the 7 principles from the standalone `cli-agent-readiness-reviewer` agent (used as reference material, not dispatched directly)
|
||||
- R5. The persona returns findings in ce:review's standard JSON findings schema (same as all other conditional personas). For design-level findings that span multiple files or concern missing capabilities, use the most relevant command handler file as the canonical location
|
||||
- R6. Severity mapping: Blocker -> P1, Friction -> P2, Optimization -> P3. The severity ceiling is P1 -- CLI readiness issues make the CLI harder for agents to use, they do not crash or corrupt
|
||||
- R7. Autofix class: all findings use autofix_class `manual` or `advisory` with owner `human`. CLI readiness findings are design decisions (JSON schema design, flag semantics, error message content) that should not be auto-applied
|
||||
- R8. Framework-idiomatic recommendations: findings reference the specific framework's patterns (e.g., "add `@click.option('--json', ...)` " for Click, not generic "add a --json flag")
|
||||
|
||||
**Integration**
|
||||
|
||||
- R9. Create a new lightweight persona agent file in `agents/review/` that distills the 7 principles into a code-review-oriented persona producing structured JSON findings. Add it to `ce-review/references/persona-catalog.md` in the cross-cutting conditional section with activation description and severity guidance
|
||||
- R10. The existing standalone `cli-agent-readiness-reviewer` agent stays unchanged -- it remains available for direct invocation and whole-CLI audits. The new persona references the same principles but is optimized for ce:review's dispatch pattern and output format
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- A ce:review run on a PR that modifies CLI command handlers includes CLI readiness findings in the review report without the user asking
|
||||
- A ce:review run on a PR that only modifies React components or Rails views does not dispatch the CLI readiness persona
|
||||
- Findings use framework-specific language matching the CLI's detected framework
|
||||
- All findings have severity P1, P2, or P3 (never P0) and autofix_class `manual` or `advisory`
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- This does not modify the standalone `cli-agent-readiness-reviewer` agent
|
||||
- This does not add CLI awareness to ce:brainstorm or ce:plan (deferred -- ce:review alone covers the highest-value case)
|
||||
- This does not introduce autofix for CLI readiness findings
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **New persona agent file**: A lightweight agent in `agents/review/` that distills the standalone agent's 7 principles into structured JSON findings. This matches how every other conditional persona works (security-reviewer, performance-reviewer, etc. are all separate agent files). The standalone agent's narrative report format doesn't match ce:review's JSON findings schema, and prompt surgery at dispatch time would be fragile.
|
||||
- **Conditional, not always-on**: Follows the existing pattern where the orchestrator selects personas based on diff content. The persona never runs on non-CLI diffs.
|
||||
- **Persona self-scopes**: The persona does its own framework detection and subcommand identification after dispatch. ce:review's orchestrator only decides whether to dispatch, not what framework is in use.
|
||||
- **No autofix**: All findings route to human review. CLI readiness issues require design judgment.
|
||||
- **Severity ceiling is P1**: CLI readiness issues don't crash the software -- they make it harder for agents to use. The highest reasonable severity is P1 (should fix), not P0 (must fix before merge).
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R9][Needs research] How much of the standalone agent's content should the new persona include directly vs. reference? The standalone agent is 24K+ (the largest review agent) -- the persona should be much smaller, distilling the principles into code-review-oriented checks rather than reproducing the full Framework Idioms Reference.
|
||||
- [Affects R4][Needs research] Should the persona evaluate all 7 principles on every dispatch, or should it prioritize principles by command type (as the standalone agent does) and cap findings to avoid flooding the review with low-signal items?
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,236 @@
|
||||
---
|
||||
date: 2026-03-31
|
||||
topic: codex-delegation
|
||||
---
|
||||
|
||||
# Codex Delegation Mode for ce:work
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Users running ce:work from Claude Code (or other non-Codex agents) may want to delegate the actual code-writing to Codex. Two motivations: (1) Codex may produce better code for certain tasks, and (2) delegating token-heavy implementation work to Codex conserves tokens on the user's current model.
|
||||
|
||||
PR #364 attempted this via a separate `ce-work-beta` skill with prose-based delegation instructions. The agent improvises CLI syntax each run, producing non-deterministic results confirmed as flaky in the PR author's own testing. The root cause: describing Codex CLI invocation in prose lets the agent guess differently every time.
|
||||
|
||||
ce-work-beta does have a structured 7-step External Delegate Mode (environment guards, availability checks, prompt file writing, circuit breaker), but the CLI invocation step itself is prose-based, causing the non-determinism. This feature ports the useful structural elements (guards, circuit breaker pattern) while replacing prose invocations with concrete bash templates.
|
||||
|
||||
> **Implementation note (2026-03-31):** The final rollout was redirected to `ce:work-beta` so stable `ce:work` remains unchanged during beta. `ce:work-beta` must be invoked manually; `ce:plan` and workflow handoffs stay on stable `ce:work` until promotion.
|
||||
|
||||
## Delegation Flow
|
||||
|
||||
```
|
||||
/ce:work delegate:codex ~/plan.md
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────┐
|
||||
│ Parse arguments │
|
||||
│ - Extract delegate flag │
|
||||
│ - Require plan file │
|
||||
│ - Check local.md default │
|
||||
│ - Resolution chain: │
|
||||
│ flag > local.md > off │
|
||||
└────────┬─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────┐ ┌───────────────────────┐
|
||||
│ Environment guard │────>│ Notify if explicit, │
|
||||
│ $CODEX_SANDBOX set? │ yes │ use standard mode │
|
||||
│ $CODEX_SESSION_ID set? │ └───────────────────────┘
|
||||
└────────┬─────────────────┘
|
||||
│ no
|
||||
▼
|
||||
┌──────────────────────────┐ ┌───────────────────────┐
|
||||
│ Availability check │────>│ Fall back to │
|
||||
│ command -v codex │ no │ standard mode + notify│
|
||||
└────────┬─────────────────┘ └───────────────────────┘
|
||||
│ yes
|
||||
▼
|
||||
┌──────────────────────────┐ ┌───────────────────────┐
|
||||
│ Consent + mode selection │────>│ Ask: disable │
|
||||
│ work_delegate_consent set? │ no │ delegation? │
|
||||
│ Show warning + sandbox │ │ Set local.md │
|
||||
│ mode choice (yolo/full- │ └───────────────────────┘
|
||||
│ auto). Recommend yolo. │
|
||||
│ (headless: require prior) │
|
||||
└────────┬─────────────────┘
|
||||
│ accepted
|
||||
▼
|
||||
┌──────────────────────────┐
|
||||
│ Per-unit execution loop │
|
||||
│ (SERIAL, not parallel) │
|
||||
│ For each implementation │
|
||||
│ unit in the plan: │
|
||||
│ │
|
||||
│ 1. Check unit eligibility │
|
||||
│ (out-of-repo? trivial?)│
|
||||
│ -> local if ineligible │
|
||||
│ 2. Named stash snapshot │
|
||||
│ 3. Write prompt + schema │
|
||||
│ to .context/compound- │
|
||||
│ engineering/codex- │
|
||||
│ delegation/ │
|
||||
│ 4. codex exec w/ flags │
|
||||
│ 5. Classify result: │
|
||||
│ CLI fail | task fail | │
|
||||
│ verify fail | success │
|
||||
│ 6. Pass: commit, drop │
|
||||
│ stash, clean scratch │
|
||||
│ Fail: rollback, │
|
||||
│ increment ctr │
|
||||
│ 7. If 3 consecutive │
|
||||
│ failures: fall back │
|
||||
│ to standard mode │
|
||||
└──────────────────────────┘
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
**Activation and Configuration**
|
||||
|
||||
- R1. Codex delegation is an optional mode within ce:work, not a separate skill. ce-work-beta is superseded: its delegation logic is replaced by this feature; its non-delegation features (e.g., Frontend Design Guidance) should be ported to ce:work as a separate concern if valuable. Disposition of ce-work-beta (delete vs. retain without delegation) is a planning decision, not a product decision.
|
||||
- R2. Delegation is triggered via a resolution chain: (1) per-invocation argument wins, (2) `work_delegate` setting in `.claude/compound-engineering.local.md` is fallback, (3) hard default is `false` (off).
|
||||
- R3. Canonical activation argument is `delegate:codex`. The skill also recognizes fuzzy variants: `codex mode`, `codex`, `delegate codex`, and similar intent expressions. Agent intent recognition handles the fuzzy matching — the set does not need to be exhaustively enumerated.
|
||||
- R4. Canonical deactivation argument is `delegate:local`. Also recognizes fuzzy variants like `no codex`, `local mode`, `standard mode`.
|
||||
- R5. Delegation only applies to structured plan execution. Ad-hoc prompts without a plan file always use standard mode regardless of the delegation setting. When delegation mode is active for a plan, each implementation unit is delegated to Codex by default. The agent may execute a unit locally in standard mode when: (a) the unit explicitly requires modifications outside the repository root, or (b) the unit is trivially small (single-file config change, simple substitution) where delegation overhead exceeds the work. The agent states which mode it's using for each unit before execution.
|
||||
|
||||
**Environment Safety**
|
||||
|
||||
- R6. When running inside a Codex sandbox (detected by `$CODEX_SANDBOX` or `$CODEX_SESSION_ID` environment variables), delegation is disabled and ce:work proceeds in standard mode. If the user explicitly requested delegation (via argument), emit a brief notification: "Already inside Codex sandbox — using standard mode." If delegation was only enabled via local.md default, proceed silently.
|
||||
- R7. All delegation logic lives in the skill itself. Converters do not modify skill behavior for cross-platform compatibility — the environment guard handles platform detection at runtime.
|
||||
|
||||
**Availability and Fallback**
|
||||
|
||||
- R8. Before delegation, check `command -v codex`. If the Codex CLI is not on PATH, fall back to standard mode with a brief notification: "Codex CLI not found — using standard mode."
|
||||
- R9. No minimum version check for now. If a future CLI change breaks delegation, the invocation fails loudly and the fix is a single bash line update.
|
||||
|
||||
**Consent and Mode Selection**
|
||||
|
||||
- R10. First time delegation activates in a project, show a one-time consent flow that: (1) explains what delegation does and the security implications, (2) presents the sandbox mode choice with a recommendation, and (3) records the user's decisions. The sandbox modes are:
|
||||
- **yolo** (recommended): Maps to `--yolo` (`--dangerously-bypass-approvals-and-sandbox`). Full system access including network. Required for verification steps that run tests or install dependencies. Explain why this is recommended.
|
||||
- **full-auto**: Maps to `--full-auto`. Workspace-write sandbox, no network access. Tests/installs that need network will fail. Suitable for pure code-writing tasks without verification dependencies.
|
||||
- R11. On user acceptance, store `work_delegate_consent: true` and `work_delegate_sandbox: yolo` (or `full-auto`) in `.claude/compound-engineering.local.md`. Do not show the consent flow again for this project.
|
||||
- R12. On user decline, ask whether to disable codex delegation entirely. If yes, set `work_delegate: false` in local.md and proceed in standard mode.
|
||||
- R13. In headless mode, delegation proceeds only if `work_delegate_consent` is already `true` in local.md. If not set or `false`, fall back to standard mode silently. Headless runs never prompt for consent and never silently escalate to unsandboxed mode without prior interactive consent.
|
||||
|
||||
**Execution Mechanism**
|
||||
|
||||
- R14. Delegation uses concrete bash commands, not prose instructions. The exact invocation template:
|
||||
|
||||
```bash
|
||||
# Read sandbox mode from settings (default: yolo)
|
||||
if [ "$CODEX_SANDBOX_MODE" = "full-auto" ]; then
|
||||
SANDBOX_FLAG="--full-auto"
|
||||
else
|
||||
SANDBOX_FLAG="--yolo"
|
||||
fi
|
||||
|
||||
codex exec \
|
||||
$SANDBOX_FLAG \
|
||||
--output-schema .context/compound-engineering/codex-delegation/result-schema.json \
|
||||
-o .context/compound-engineering/codex-delegation/result-<unit-id>.json \
|
||||
- < .context/compound-engineering/codex-delegation/prompt-<unit-id>.md
|
||||
```
|
||||
|
||||
The agent executes this verbatim — no improvisation of CLI syntax.
|
||||
|
||||
- R15. Sandbox posture defaults to `yolo` (`--yolo`, shorthand for `--dangerously-bypass-approvals-and-sandbox`) but the user may choose `full-auto` during the consent flow (R10). The choice is stored in `work_delegate_sandbox` in local.md. `yolo` is recommended because `--full-auto` blocks network access, which is required for verification steps (running tests, installing dependencies). If `full-auto` is chosen and causes repeated verification failures, the circuit breaker (R18) handles fallback.
|
||||
|
||||
- R16. When delegation mode is active, ALL units execute serially — both delegated and locally-executed units. Git stash is a global stack; mixing parallel and serial execution on the same working tree causes stash entanglement. This means delegation mode and swarm mode (Agent Teams) are mutually exclusive. Before each delegated unit, the loop assumes a clean working tree (enforced by ce:work's Phase 1 setup and by mandatory commits after each successful unit). Snapshot the working tree via named stash: `git stash push --include-untracked -m "ce-codex-<unit-id>"`. On failure, rollback via `git checkout -- . && git clean -fd && git stash drop "$(git stash list | grep 'ce-codex-<unit-id>' | head -1 | cut -d: -f1)"`. On success, commit the changes, then drop the named stash.
|
||||
|
||||
- R17. The structured prompt template is written to a file at `.context/compound-engineering/codex-delegation/prompt-<unit-id>.md` rather than piped via stdin, to avoid ARG_MAX limits for large CURRENT PATTERNS sections. The template includes: TASK (goal from implementation unit), FILES TO MODIFY (file list), CURRENT PATTERNS (relevant code context), APPROACH (from implementation unit), CONSTRAINTS (no git commit, restrict modifications to files within the repository root, scoped changes, line limit, mandatory result reporting), and VERIFY (test/lint commands). Prompt files are cleaned up after each successful unit.
|
||||
|
||||
- R18. A consecutive failure counter tracks delegation failures. After 3 consecutive failures, the skill falls back to standard mode for remaining units with a notification.
|
||||
|
||||
- R19. Failure classification uses a multi-signal approach. `codex exec` returns exit code 0 even when the task fails — the exit code only reflects CLI infrastructure, not task success.
|
||||
|
||||
| Category | Signal | Action |
|
||||
|---|---|---|
|
||||
| **CLI failure** | Exit code != 0 | Hard failure — fall back to standard mode |
|
||||
| **Result absent** | Exit code 0, result JSON missing or malformed | Count as task failure |
|
||||
| **Task failure** | Exit code 0, result schema `status: "failed"` | Count toward circuit breaker, rollback |
|
||||
| **Task partial** | Exit code 0, result schema `status: "partial"` | Keep changes, report gaps to main agent |
|
||||
| **Verify failure** | Exit code 0, `status: "completed"`, VERIFY fails | Count toward circuit breaker, rollback |
|
||||
| **Success** | Exit code 0, `status: "completed"`, VERIFY passes | Commit, drop stash, continue |
|
||||
|
||||
- R20. A result schema file is written alongside the prompt file. Codex is instructed via `--output-schema` to produce structured JSON conforming to this schema. The `-o` flag writes the result to `result-<unit-id>.json`. The schema:
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"status": { "enum": ["completed", "partial", "failed"] },
|
||||
"files_modified": { "type": "array", "items": { "type": "string" } },
|
||||
"issues": { "type": "array", "items": { "type": "string" } },
|
||||
"summary": { "type": "string" }
|
||||
},
|
||||
"required": ["status", "files_modified", "issues", "summary"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
```
|
||||
|
||||
The prompt CONSTRAINTS section includes mandatory result reporting instructions telling Codex it MUST fill in the schema honestly: `status: "completed"` only if all changes were made, `"partial"` if incomplete, `"failed"` if no meaningful progress. Known limitation: `--output-schema` only works with `gpt-5` family models, not `gpt-5-codex` or `codex-` prefixed models (Codex CLI bug #4181). If the result JSON is absent or malformed, classify as task failure.
|
||||
|
||||
- R21. The prompt constraint tells Codex to restrict all modifications to files within the repository root. If Codex discovers mid-execution that it needs to modify files outside the repo root, it should complete what it can within the repo and report what it couldn't do via the result schema `issues` field. The main agent then handles the out-of-repo work in standard mode. Out-of-repo changes cannot be detected or rolled back by git stash — this is an accepted risk mitigated by the prompt constraint and per-unit pre-screening (R5).
|
||||
|
||||
**Settings in compound-engineering.local.md**
|
||||
|
||||
- R22. New YAML frontmatter keys in `.claude/compound-engineering.local.md`:
|
||||
- `work_delegate`: `codex`/`false` (default: `false`) — delegation target when enabled
|
||||
- `work_delegate_consent`: `true`/`false` — whether the user has completed the one-time consent flow
|
||||
- `work_delegate_sandbox`: `yolo`/`full-auto` (default: `yolo`) — sandbox posture for codex exec
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Codex successfully implements implementation units from ce:plan output across a variety of task types (new features, bug fixes, refactors)
|
||||
- CLI invocations are deterministic — no agent improvisation of shell syntax across runs
|
||||
- Delegation activates only when explicitly requested (argument or local.md), only with a plan file, and never when running inside Codex
|
||||
- Failed delegation rolls back cleanly via named git stash without corrupting tracked repository files
|
||||
- The result schema provides reliable signal for success/failure classification
|
||||
- Users who never enable delegation experience zero change in ce:work behavior
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- **Not a separate skill.** ce-work-beta is superseded. This modifies ce:work directly.
|
||||
- **No app-server integration.** We use bare `codex exec`, not the codex-companion.mjs app server or the codex plugin's rescue skill. The delegation pattern is fire-prompt -> wait -> inspect-result, which is exactly what `codex exec` provides.
|
||||
- **No ad-hoc delegation.** Delegation only applies to structured plan execution with a plan file. Bare prompts without plans always use standard mode.
|
||||
- **No minimum version gating.** Added later if a breaking CLI change actually occurs.
|
||||
- **No periodic re-consent.** One acceptance per project. Version-gated or calendar-based re-consent can be added later if needed.
|
||||
- **No converter changes.** The skill handles platform detection internally via environment variable checks.
|
||||
- **No out-of-repo detection.** Git stash cannot protect files outside the repo. Defense is prompt constraint + per-unit pre-screening, not post-execution validation.
|
||||
- **No timeout for v1.** Neither `codex exec` nor the most mature codex integration (osc-work) implements timeouts. Added later if users report hung processes.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Modify ce:work, not a separate skill**: Avoids skill proliferation. Users stay in their existing workflow. ce-work-beta's delegation section is superseded; its structural patterns (guards, circuit breaker) are ported.
|
||||
- **`delegate:codex` namespace, not `mode:codex`**: Existing `mode:` tokens describe interaction style (headless, autofix). Delegation describes execution target. Separate namespace avoids semantic overloading.
|
||||
- **Bare `codex exec` over app-server**: App server offers structured output and thread management, but requires fragile path discovery into another plugin's versioned install directory. `codex exec` is one line of bash, works identically in subagents, and does exactly what fire-and-wait delegation needs.
|
||||
- **User-selected sandbox mode (yolo default, full-auto option)**: yolo is recommended because `--full-auto` blocks network access needed for test/lint commands. But users who prefer sandboxed execution can choose `full-auto`, accepting that verification may fail. The circuit breaker handles repeated failures.
|
||||
- **One-time consent with mode selection**: Consent is about informed awareness, not ongoing compliance. The sandbox mode choice is part of the consent flow and persisted in local.md.
|
||||
- **Per-unit delegation eligibility, not all-or-nothing**: Default is to delegate all units, but the agent pre-screens units that need out-of-repo access or are trivially small. This avoids delegating work that can't succeed in the unsandboxed environment and reduces overhead for trivial changes.
|
||||
- **Prompt file over stdin**: Writing prompts to `.context/compound-engineering/codex-delegation/` avoids ARG_MAX limits, provides debugging artifacts on failure, and follows the repo's scratch space convention.
|
||||
- **Complete-and-report over error-and-rollback**: When Codex discovers it needs out-of-repo access mid-execution, it completes in-repo changes and reports what it couldn't do. Preserves useful work rather than wasting it.
|
||||
- **Plan-only delegation**: Ad-hoc prompts use standard mode. Delegation requires the structured plan decomposition to build effective prompts and provide meaningful implementation units.
|
||||
- **Serial execution for all units when delegation is active**: Git stash is a global stack. Mixing parallel and serial execution causes stash entanglement. When delegation mode is on, all units (including locally-executed ones) run serially. This makes delegation mode and swarm mode (Agent Teams) mutually exclusive — a deliberate tradeoff of parallelism for the ability to use Codex.
|
||||
- **`--output-schema` for result classification**: `codex exec` returns exit code 0 even on task failure. The structured result schema combined with VERIFY commands provides reliable success/failure signal. Prompt-enforced honest reporting plus cross-validation with VERIFY catches model misreporting.
|
||||
- **No timeout for v1**: `codex exec` has no built-in timeout, and the most mature integration (osc-work) doesn't implement one either. Added if users report hung processes.
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- Codex CLI `exec` subcommand with `--yolo`, `--full-auto`, `--output-schema`, `-o`, and `-m` flags remains stable
|
||||
- `--output-schema` works with `gpt-5` family models. Known bug #4181 breaks it for `gpt-5-codex` / `codex-` prefixed models — delegation should use `gpt-5` family models (e.g., `o4-mini`, `gpt-5.4`)
|
||||
- `$CODEX_SANDBOX` and `$CODEX_SESSION_ID` environment variables continue to be set when running inside Codex
|
||||
- `.claude/compound-engineering.local.md` YAML frontmatter reading/writing infrastructure must be built as part of this work — no existing skill currently reads or writes these keys. This is a prerequisite, not an assumption.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R17][Needs research] What is the optimal prompt template structure for maximizing Codex code quality? The printing-press skill provides one template; the codex plugin's prompting skill (`gpt-5-4-prompting`) may offer insights on how to structure prompts for Codex/GPT models specifically.
|
||||
- [Affects R14][Technical] Where exactly in ce:work's Phase 2 task execution loop does the delegation branch? Need to read the current task-worker dispatch logic to identify the cleanest insertion point.
|
||||
- [Affects R18][Technical] Should the circuit breaker (3 consecutive failures) reset per-unit or persist across the entire plan execution? Per-unit is more forgiving; per-plan is more conservative.
|
||||
- [Affects R22][Technical] How does the agent parse `.claude/compound-engineering.local.md` YAML frontmatter at runtime? Is there an existing utility or must the skill instruct the agent to parse it directly via bash?
|
||||
- [Affects R20][Needs testing] How reliably does `--output-schema` constrain Codex's final response? Need to test with representative implementation prompts to validate the result classification approach. Use `--ephemeral` flag during testing to avoid session file clutter (production invocations do not use `--ephemeral` — session persistence is valuable for debugging).
|
||||
- [Affects R20][Technical] Fallback behavior when `--output-schema` fails (wrong model family, malformed output): define the exact classification logic when the result JSON is absent.
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
date: 2026-04-01
|
||||
topic: cross-invocation-cluster-analysis
|
||||
---
|
||||
|
||||
# Cross-Invocation Cluster Analysis for resolve-pr-feedback
|
||||
|
||||
## Problem Frame
|
||||
|
||||
The resolve-pr-feedback skill's cluster analysis is gated on two signals: volume (3+ items) and verify-loop re-entry (2nd+ pass within the same invocation). The verify-loop signal is effectively dead — it requires new review threads to appear between push and verify, but automated reviewers take minutes while verify runs seconds after push. The timing gap makes this gate unreliable at best, and in the common case of automated reviewers, impossible.
|
||||
|
||||
This leaves volume as the only working gate. The skill misses the exact scenario clustering was designed for: a reviewer posts feedback about the same *class* of problem across multiple rounds, with each round containing only 1-2 threads. Individually, no round triggers the volume gate. But taken together, there's a clear recurring pattern — e.g., "three separate rounds of feedback all about missing convergence behavior in target writers." The skill should step back and investigate the problem class holistically rather than applying band-aids to each instance.
|
||||
|
||||
## Requirements
|
||||
|
||||
**Detection Signal**
|
||||
|
||||
- R1. Replace the verify-loop re-entry gate signal with a cross-invocation awareness signal. Before triaging, the skill checks whether it has previously resolved threads on this same PR. Its own prior reply comments are the evidence.
|
||||
- R2. If prior resolutions exist and new unresolved feedback has arrived since the last resolution, that constitutes the re-entry signal — even with just 1 new item. If no prior resolutions are found (first invocation), the cross-invocation signal does not fire and processing continues with the volume gate as the only cluster trigger.
|
||||
- R3. The volume gate (3+ items) remains unchanged as a parallel trigger. The two gates are OR'd: either one fires cluster analysis.
|
||||
|
||||
**Cost Control**
|
||||
|
||||
- R9. Cross-invocation detection must not add GraphQL API calls. The existing `get-pr-comments` query should be broadened to return both unresolved and resolved threads (with skill replies) in a single call. All cross-invocation analysis — detection, overlap check, clustering — works on data already in memory from that one call.
|
||||
- R10. Cross-invocation clustering is scoped to the last N resolution rounds (not all history). A "round" is the set of threads resolved in a single skill invocation. This bounds the data the skill processes regardless of PR history length. Planning should determine the right value of N; 2-3 rounds is likely sufficient since recurring patterns surface in recent history.
|
||||
- R11. When the cross-invocation signal fires but the volume gate does not, the skill runs a lightweight overlap check first: compare concern categories and file paths between new and prior threads using data already fetched. Promote to full clustering only if category or spatial overlap exists. If no overlap, skip clustering and process the new thread(s) individually.
|
||||
|
||||
**Clustering Input**
|
||||
|
||||
- R4. When the cross-invocation signal fires and overlap is confirmed (R11), cluster analysis considers both the new thread(s) AND previously-resolved threads from the last N rounds as input. This enables detecting that the same concern category keeps recurring across rounds.
|
||||
- R5. Previously-resolved threads are included in category assignment and spatial grouping alongside new threads, so clusters can span rounds.
|
||||
|
||||
**Resolver Behavior on Cross-Invocation Clusters**
|
||||
|
||||
- R6. When a cross-invocation cluster forms, the resolver agent assesses the prior fixes and applies one of three modes:
|
||||
- **Band-aid fixes** — prior fixes addressed symptoms, not root cause. Re-examine and potentially redo them as part of a holistic fix.
|
||||
- **Correct but incomplete** — prior fixes were right for their scope, but the recurring pattern reveals the same problem likely exists in untouched sibling code. Keep prior fixes, fix the new thread, and proactively investigate whether the pattern extends to code no reviewer has flagged yet. This is the highest-value mode — it's what catches "three rounds of the same concern category in different files means there are probably more files with the same issue."
|
||||
- **Sound and independent** — prior fixes were adequate and the new thread is genuinely unrelated despite clustering. Use prior context for awareness only.
|
||||
- R7. The cluster brief XML gains a `<prior-resolutions>` element listing previously-resolved thread IDs and their concern categories, with reply timestamps (createdAt) to establish ordering across rounds, so the resolver agent has the full cross-round picture.
|
||||
|
||||
**Within-Session Verify Loop**
|
||||
|
||||
- R8. The within-session verify loop (step 8: if new threads remain, repeat from step 2) continues to function as a workflow mechanism. Replies posted during earlier cycles within the same session count as prior resolutions for the cross-invocation signal, so the new gate naturally subsumes the old verify-loop re-entry gate.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- Recurring feedback about the same problem class across 2+ rounds triggers cluster analysis, even when each round has only 1-2 threads
|
||||
- A single new thread on a PR with prior resolutions in the same concern category produces a cluster brief that includes both the new and old threads
|
||||
- The resolver agent can distinguish three modes: "prior fixes were band-aids, redo holistically", "prior fixes were correct but incomplete, investigate sibling code", and "prior fixes were sound, this is independent"
|
||||
- Token cost is bounded: a PR with 15 prior resolution rounds costs no more for clustering than a PR with 3, and unrelated new feedback on a multi-round PR skips clustering entirely after the lightweight overlap check
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- No persistent state files or `.context/` storage — detection relies entirely on GitHub PR comment history
|
||||
- No changes to the volume gate threshold or the cluster spatial grouping rules
|
||||
- No changes to how the resolver agent handles standard (non-cluster) threads
|
||||
- The `get-pr-comments` script currently filters to unresolved threads only (`isResolved == false`). Per R9, this query is broadened to also return resolved threads — no new script, just a wider filter in the existing one
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Detection via own replies, not persistent state**: Prior resolutions are detected by checking for the skill's own reply comments on PR threads. This keeps the skill stateless and avoids `.context/` file management. The data is already authoritative (GitHub is the source of truth for what was resolved).
|
||||
- **Three-mode resolver assessment**: The agent distinguishes band-aid fixes (redo), correct-but-incomplete fixes (keep fixes, investigate sibling code), and sound-and-independent fixes (context only). The "correct but incomplete" mode is the highest-value case — it's what turns "three rounds of the same concern in different files" into proactive investigation of untouched code with the same pattern.
|
||||
- **Cross-invocation signal subsumes verify-loop signal**: Within-session cycles produce replies that count as prior resolutions, so the new gate handles both cross-session and within-session re-entry without needing a separate verify-loop signal.
|
||||
- **Bounded lookback, not full history**: Clustering only considers the last N resolution rounds. Recurring patterns surface in recent history — if the same concern category appeared in the last 2-3 rounds, that's the signal. Going back further adds cost without proportional value.
|
||||
- **Zero additional API calls**: Cross-invocation detection piggybacks on the existing `get-pr-comments` query by broadening the filter. All analysis — detection, overlap check, clustering — happens in-memory on data already fetched. No new GraphQL calls.
|
||||
- **Two-tier cost control**: The lightweight overlap check (R11) prevents unnecessary full clustering. Most multi-round PRs get unrelated feedback in later rounds; those skip clustering entirely after a cheap metadata comparison. Full clustering only runs when there's evidence it will find something.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R1][Technical] How should the skill identify its own prior replies? Options include checking the authenticated `gh` user, matching a reply-text pattern, or both. Planning should check what the existing `resolve-pr-thread` and `reply-to-pr-thread` scripts produce and what's easily queryable.
|
||||
- [Affects R4][Technical] How should previously-resolved threads be represented in the triage list alongside new threads? They need a status marker (e.g., `previously-resolved`) so clustering can include them while dispatch skips re-resolution of threads that don't cluster.
|
||||
- [Affects R9][Technical] What fields does the existing `get-pr-comments` GraphQL query return per thread? Planning should check whether the query already fetches enough data (file path, line range, comment body, author) to support both resolved and unresolved threads without changing the response shape, or whether fields need to be added.
|
||||
- [Affects R10][Technical] What is the right value of N for resolution round lookback? 2-3 is the starting hypothesis. Planning should consider typical PR review patterns and the marginal value of deeper lookback.
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
date: 2026-04-02
|
||||
topic: ce-slack-researcher-agent
|
||||
---
|
||||
|
||||
# Slack Analyst Agent
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Coding agents operating within compound-engineering workflows (ideate, plan, brainstorm) have no visibility into organizational knowledge that lives in Slack. Decisions, constraints, ongoing discussions, and context about projects are often undocumented anywhere except Slack conversations. When a developer is about to make a change, relevant Slack context -- a discussion about why something was designed a certain way, a decision to deprecate a feature, constraints mentioned by another team -- is invisible to the agent assisting them.
|
||||
|
||||
The official Slack plugin provides user-facing commands (`/slack:find-discussions`, `/slack:summarize-channel`), but these are standalone and manual. There is no research agent that compound-engineering workflows can dispatch programmatically to surface Slack context as part of their normal research phase.
|
||||
|
||||
## Requirements
|
||||
|
||||
**Agent Identity and Placement**
|
||||
|
||||
- R1. Create a research-category agent at `agents/research/ce-slack-researcher.md` following the established research agent pattern (frontmatter with name, description, model:inherit; examples block; phased execution).
|
||||
- R2. The agent's role is analytical: it searches Slack for context relevant to the task at hand and returns a concise, structured digest. It does not send messages, create canvases, or take any write actions in Slack.
|
||||
|
||||
---
|
||||
|
||||
**Precondition and Short-Circuit Design**
|
||||
|
||||
- R3. Two-level short-circuit to minimize token waste:
|
||||
- **Caller level:** Calling workflows check whether the Slack MCP server is connected before dispatching the agent. If unavailable, skip dispatch entirely. Detection should check for MCP availability (not specific tool names, which may change).
|
||||
- **Agent level:** The agent performs its own precondition check on entry. If Slack MCP tools are not accessible, return a short message ("Slack MCP not connected -- skipping Slack analysis") and exit immediately.
|
||||
- R4. The agent should also short-circuit if the caller provides no meaningful search context (e.g., an empty or overly generic topic). Return a message indicating insufficient context rather than running broad, low-value searches.
|
||||
|
||||
---
|
||||
|
||||
**Search Strategy**
|
||||
|
||||
- R5. Default behavior is search-first: run 2-3 targeted searches using `slack_search_public_and_private` based on keywords derived from the task topic. Search both public and private channels by default (user has already authed the Slack MCP).
|
||||
- R6. Read threads (`slack_read_thread`) only for high-relevance search hits -- not speculatively. Limit thread reads to avoid runaway token consumption (cap at ~3-5 thread reads per invocation).
|
||||
- R7. Accept an optional channel hint from the caller. When provided, also read recent history from the specified channel(s) using `slack_read_channel` with appropriate time bounds. Without a channel hint, do not read channel history -- search results are sufficient.
|
||||
- R8. Future consideration (not in scope): a user preference/setting for channels that should always be searched. Defer to a later iteration.
|
||||
|
||||
---
|
||||
|
||||
**Output Format**
|
||||
|
||||
- R9. Return a concise summary digest organized by topic/theme. Each finding should include:
|
||||
- The topic or theme
|
||||
- A brief summary of what was discussed/decided
|
||||
- Source attribution (channel name, approximate date, participants if notable)
|
||||
- Relevance to the current task
|
||||
- R10. When no relevant Slack context is found, return a short explicit statement ("No relevant Slack discussions found for [topic]") rather than generating filler.
|
||||
- R11. Keep output compact enough to be useful context without dominating the calling workflow's token budget. Target roughly 200-500 tokens for typical results.
|
||||
|
||||
---
|
||||
|
||||
**Workflow Integration**
|
||||
|
||||
- R12. Integrate into three calling workflows:
|
||||
- **ce-ideate** -- dispatch during Phase 1 (Codebase Scan), alongside learnings-researcher. Slack context enriches ideation by surfacing org discussions about the focus area.
|
||||
- **ce-plan** -- dispatch during the research/context-gathering phase. Slack context surfaces constraints, prior decisions, and ongoing discussions relevant to the implementation.
|
||||
- **ce-brainstorm** -- dispatch during Phase 1.1 (Existing Context Scan). Brainstorming especially benefits from knowing what the org has already discussed about the topic.
|
||||
- R13. In all calling workflows, dispatch the Slack analyst agent in parallel with other research agents (learnings-researcher, etc.) to avoid adding latency. Callers wait for all parallel agents to return before consolidating results (this is the existing pattern for parallel research dispatch). The Slack analyst's dispatch condition is MCP availability (R3). The agent itself handles the meaningful-context check (R4) internally.
|
||||
- R14. Callers should incorporate the Slack analyst's output into their existing context summary alongside other research results, not as a separate section.
|
||||
|
||||
---
|
||||
|
||||
**Dependency on External Plugin**
|
||||
|
||||
- R15. The Slack MCP server is owned by the official Slack plugin, not compound-engineering. The agent uses MCP tools that the Slack plugin configures. This creates a soft dependency: the agent is useful only when the Slack plugin is installed and authenticated, but compound-engineering must not require it.
|
||||
- R16. Do not bundle or reference the Slack plugin's `.mcp.json` or configuration from within compound-engineering. The agent relies solely on MCP tools being available at runtime.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- When Slack MCP is connected, the agent surfaces relevant org context that would not have been available from codebase analysis alone, enriching the output of ideate/plan/brainstorm workflows.
|
||||
- When Slack MCP is not connected, the agent adds zero token overhead (caller-level short-circuit prevents dispatch).
|
||||
- The agent completes within a reasonable time budget (~10-15 seconds) and returns compact output that doesn't bloat calling workflows.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- No write actions to Slack (no sending messages, no creating canvases).
|
||||
- No channel history reads unless the caller provides an explicit channel hint.
|
||||
- No user preference/settings system for default channels (deferred).
|
||||
- No replacement of existing Slack plugin commands -- this agent is complementary, not competitive.
|
||||
- No installation or configuration of the Slack MCP -- that remains the Slack plugin's responsibility.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Agent, not skill:** This is a sub-agent invoked programmatically by workflows, not a user-facing slash command. It lives in `agents/research/`.
|
||||
- **Public + private search by default:** The user already authed the Slack MCP, so searching private channels avoids missing the richest context.
|
||||
- **Search-first, reads on demand:** Avoids the token cost of speculatively reading channel history. Thread reads are limited to high-relevance hits.
|
||||
- **Concise digest output:** Callers are responsible for interpreting the output for their specific context. The agent returns useful summaries, not raw message dumps.
|
||||
- **MCP availability check, not tool-name check:** Callers check if the Slack MCP is connected, not for specific tool names (which may change in future Slack MCP versions).
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R3][Technical] How exactly should callers detect Slack MCP availability? Claude Code's tool list inspection, checking for any `slack_*` tool prefix, or another mechanism?
|
||||
- [Affects R5][Needs research] What is the optimal number of search queries per invocation to balance coverage vs. token cost? Start with 2-3 and tune based on real usage.
|
||||
- [Affects R12][Technical] What modifications are needed in ce-ideate, ce-plan, and ce-brainstorm skill files to add the conditional dispatch? Review each skill's research phase to find the right insertion point.
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,87 @@
|
||||
---
|
||||
date: 2026-04-05
|
||||
topic: universal-planning
|
||||
---
|
||||
|
||||
# Universal Planning: Non-Software Task Support for ce:plan and ce:brainstorm
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Users naturally reach for `/ce:plan` to plan any multi-step task — trip itineraries, study plans, content strategies, research workflows. Currently, the model self-gates and refuses non-software tasks because ce:plan's language is heavily software-centric ("implementation units", "test scenarios", "repo patterns"). This forces users back to unstructured prompting for non-software work, losing the structured thinking that makes ce:plan valuable.
|
||||
|
||||
The structured thinking behind ce:plan — breaking down ambiguity, researching context, sequencing steps, identifying dependencies — is domain-agnostic. The skill's value proposition should not be limited to software.
|
||||
|
||||
**Why a conditional path instead of just softening language:** Softening the self-gating language in SKILL.md would be cheaper and might stop the refusal. But the value of ce:plan for non-software tasks comes from the structured workflow — ambiguity assessment, research orchestration, quality-guided output, and a durable plan file. Without the non-software path, the model would attempt to follow software-specific phases (repo research, implementation units, test scenarios) on a non-software task, producing a worse result than a direct prompt. The conditional path lets non-software tasks benefit from structured thinking without fighting software-specific structure.
|
||||
|
||||
See: [GitHub issue #517](https://github.com/EveryInc/compound-engineering-plugin/issues/517)
|
||||
|
||||
## Requirements
|
||||
|
||||
**Skill Description and Trigger Language**
|
||||
|
||||
- R1. ce:plan's YAML `description` and trigger phrases are updated to include non-software planning. The model reads this description when deciding which skill to invoke — if triggers only mention software concepts, the internal detection logic never fires. Example: *"Create structured plans for any multi-step task — software features, research workflows, events, study plans, or any goal that benefits from structured breakdown."*
|
||||
|
||||
**Detection and Routing**
|
||||
|
||||
- R2. ce:plan detects whether a task is software-related or not early in Phase 0, before searching for requirements docs or launching software-specific research agents
|
||||
- R3. Detection error policy: false positives (software task routed to non-software path) are worse than false negatives (non-software task staying on software path), because a false positive skips repo research and produces a disconnected plan. When detection is ambiguous, ask the user rather than guessing. Default to software path when uncertain.
|
||||
- R4. ce:brainstorm: verify whether it actually self-gates on non-software tasks. If it doesn't (its description is already domain-agnostic), no changes needed — its existing Phase 4 handoff to ce:plan already works. If it does self-gate, soften the gating language so it stops refusing. ce:plan owns the non-software planning path; ce:brainstorm only needs to not block the flow.
|
||||
|
||||
**Non-Software Planning Path in ce:plan (Core — Phase 1)**
|
||||
|
||||
- R5. When a non-software task is detected, ce:plan skips Phases 0.2-0.5 and Phase 1 (all software-specific) and loads a reference file (`references/universal-planning.md`) containing the alternative workflow. Existing Phase 5.2 (Write Plan File) and Phase 5.4 (Handoff options) are reusable; Phase 5.3 (Confidence Check with software-specific agents) is not.
|
||||
- R6. The non-software path assesses ambiguity: is the request clear enough to plan directly, or does it need clarification first?
|
||||
- R7. When clarification is needed, the non-software path runs focused Q&A inline — up to 3 questions as a guideline, not a hard cap — targeting the most impactful clarifying questions. Stop when remaining ambiguity is acceptable to defer to plan execution.
|
||||
- R8. The plan output is guided by quality principles (what makes a great plan), not a prescribed template. The model decides the format based on the task domain
|
||||
|
||||
**Non-Software Planning Path (Extensions — Phase 2, after core validation)**
|
||||
|
||||
- R9. The non-software path can invoke web search directly (no new MCP integrations or research subsystems) when the task benefits from external context. The main skill collates findings inline.
|
||||
- R10. The non-software path can still interact with local files when the task involves them (e.g., "read these materials and create a study plan")
|
||||
|
||||
**Token Cost Management**
|
||||
|
||||
- R11. The non-software path lives entirely in reference files loaded conditionally via backtick paths. Main SKILL.md changes are minimal — detection stub only
|
||||
- R12. The software planning path remains completely unchanged — negligible token cost increase for software-only users (detection stub only)
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- `/ce:plan a 3 day trip to Disney World with 2 kids ages 11 and 13` produces a thoughtful, structured plan instead of refusing
|
||||
- `/ce:plan look at the materials in this folder and create a study plan` reads local files and produces a study plan
|
||||
- `/ce:brainstorm plan my team offsite` produces a structured plan (verify — may already work without changes)
|
||||
- `/ce:plan plan the database migration to support multi-tenancy` routes to the software path (boundary case — software despite "plan" and "migration")
|
||||
- `/ce:plan plan our team's migration to the new office` routes to the non-software path (boundary case — non-software despite "migration")
|
||||
- Software tasks continue to work identically — no regression
|
||||
- Non-software detection adds negligible tokens to the software path
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Not building domain-specific planning templates (travel, education, etc.) — the model adapts format to domain
|
||||
- Not changing the software planning path in ce:plan at all
|
||||
- Not adding non-software support to ce:work or other downstream skills — those remain software-focused
|
||||
- Not adding MCP integrations or domain-specific research tools — use existing web research capabilities
|
||||
- Pipeline mode (LFG/SLFG): non-software tasks are not supported. Detection should short-circuit the pipeline gracefully rather than producing a plan that ce:work cannot execute. The short-circuit contract (what ce:plan returns, how LFG's retry gate handles it) is deferred to planning.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **ce:plan owns universal planning, not ce:brainstorm**: The durable output is a plan file. Brainstorming Q&A is a means to an end, not a separate non-software workflow. ce:plan does its own focused Q&A when needed.
|
||||
- **No prescribed template for non-software outputs**: Impossible to anticipate all domains. Quality principles guide the model; format is emergent.
|
||||
- **Reference file extraction**: Non-software path in `references/universal-planning.md` keeps token costs down and avoids bloating the main skill for software users.
|
||||
- **Default to software when uncertain**: False positives (software → non-software) are costlier than false negatives (non-software → software). When ambiguous, ask the user.
|
||||
- **Non-software plan file location is user-chosen.** Before writing, prompt the user with options: (a) `docs/plans/` if it exists, (b) current working directory, (c) `/tmp`, or (d) a path they specify. Frontmatter omits software-specific fields (`type: feat|fix|refactor`). Filename convention (`YYYY-MM-DD-<descriptive-name>-plan.md`) applies regardless of location.
|
||||
- **Incremental delivery**: Core path (R5-R8) first — detection, ambiguity assessment, quality-guided output. Extensions (R9-R10) — research orchestration, local file interaction — added after core validation.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R2][Technical] What heuristics should the detection use? Likely a combination of: does the request reference code/repos/files in a software context, specific programming languages, software concepts? Needs to handle ambiguous cases like "plan a migration" (could be data migration or office migration). Error policy (R3) constrains the design: default to software, ask when uncertain.
|
||||
- [Affects R8][Technical] What output quality principles produce the best non-software plans? Define these directly during planning — principles like specificity, sequencing, resource identification, contingency planning — rather than running a separate research effort.
|
||||
- [Affects R9][Technical] Which research mechanisms work best for non-software tasks? WebSearch/WebFetch directly, or best-practices-researcher adapted for non-software topics? Defer until core path is validated.
|
||||
- [Affects R4][Technical] Does ce:brainstorm actually self-gate on non-software tasks? Verify before building detection there. Its description appears domain-agnostic — changes may be unnecessary. Note: even if it doesn't self-gate, its Phase 1.1 repo scan would waste tokens finding nothing on a non-software task. Decide whether that's acceptable or needs a skip.
|
||||
- [Affects R5][Technical] Non-software plan file location: prompt the user with options (docs/plans/ if it exists, CWD, /tmp, or custom path). Only show docs/plans/ option when the directory exists.
|
||||
- [Affects pipeline][Technical] LFG/SLFG short-circuit contract: does ce:plan write a stub file, return an error, or produce no file? LFG has a hard gate that retries if no plan file exists — the contract must satisfy or bypass that gate.
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
date: 2026-04-17
|
||||
topic: ce-release-notes-skill
|
||||
---
|
||||
|
||||
# `ce-release-notes` Skill
|
||||
|
||||
## Problem Frame
|
||||
|
||||
The `compound-engineering` plugin ships frequently — often multiple releases per week. Users who install the plugin via the marketplace can't easily keep up with what's changed: skill renames, new behaviors, retired commands, or relevant fixes. The release history exists publicly on GitHub (release-please-generated GitHub Releases at `EveryInc/compound-engineering-plugin`), but scrolling through release pages to answer "what happened to the deepen-plan skill?" is friction users won't bother with.
|
||||
|
||||
This skill provides a conversational interface over the plugin's GitHub Releases so a user can ask either "what's new?" or a specific question and get a grounded, version-cited answer without leaving Claude Code.
|
||||
|
||||
**Premise note:** The user-pain claim above is grounded in the rapid release cadence rather than in cited support asks or telemetry. We accept the residual risk that the skill may see low adoption if the conversational-lookup framing turns out to be a weaker need than discoverability or release-page bookmarking.
|
||||
|
||||
## Requirements
|
||||
|
||||
**Invocation and Modes**
|
||||
- R1. Skill is invoked via slash command `/ce:release-notes` (matching the `ce:` namespace convention used by sibling skills like `/ce:plan`, `/ce:brainstorm`). The skill directory is `plugins/compound-engineering/skills/ce-release-notes/`; the SKILL.md `name:` frontmatter field is `ce:release-notes` (colon form, not dash) — that is what produces the `/ce:release-notes` slash command. (Several existing `ce-` skills use `name: ce-x` and are not slash-invoked; this one needs the colon form to match R1.)
|
||||
- R2. Bare invocation (`/ce:release-notes`) returns a summary of recent releases.
|
||||
- R3. Argument invocation (`/ce:release-notes <question or topic>`) returns a direct answer to the user's question, grounded in the relevant release(s).
|
||||
- R4. **v1 is slash-only invocation.** The SKILL.md frontmatter sets `disable-model-invocation: true` so the skill only fires when the user explicitly types `/ce:release-notes`. Auto-invocation is deferred to a possible v2 once dogfooding shows users clearly want conversational triggering and a tested gating description has been validated against a prompt corpus.
|
||||
|
||||
**Data Source**
|
||||
- R5. Source of truth is the GitHub Releases API for `EveryInc/compound-engineering-plugin`. **Layered access strategy:** prefer the `gh` CLI when available (authenticated, consistent JSON output, better error messages, higher rate limits). Fall back to anonymous HTTPS against `https://api.github.com/repos/EveryInc/compound-engineering-plugin/releases` (or the equivalent paginated endpoint) when `gh` is missing or unauthenticated. The repo is public, so anonymous reads work and the 60 req/hr-per-IP unauth'd limit is more than enough for this skill's invocation frequency.
|
||||
- R6. Only releases tagged with the `compound-engineering-v*` prefix are considered. Sibling tags (`cli-v*`, `coding-tutor-v*`, `marketplace-v*`, `cursor-marketplace-v*`) are filtered out, even though `cli` and `compound-engineering` share version numbers via release-please's `linked-versions` plugin.
|
||||
- R7. No local caching, no fallback to `CHANGELOG.md` files. Always fetch live.
|
||||
- R8. Skill must fail gracefully with an actionable message when **both** access paths fail (e.g., no network, GitHub API outage, rate-limit exhaustion on the anonymous fallback). Missing `gh` alone is not a failure — the skill silently uses the anonymous fallback.
|
||||
|
||||
**Output — Summary Mode**
|
||||
- R9. Default window is the last 10 plugin releases.
|
||||
- R10. Per-release section format: version + publish date + the release-please-generated changelog body (already grouped by `Features`, `Bug Fixes`, etc.), trimmed minimally — release sizes vary, so do not impose a uniform highlight count.
|
||||
- R11. Each release section links to its GitHub release URL so users can read the full notes.
|
||||
|
||||
**Output — Query Mode**
|
||||
- R12. Search window is the last 20 plugin releases — fixed cap, no expansion. 20 releases is already a substantial corpus (multiple weeks of cadence). If no matching content is found within that window, report "not found" and surface the GitHub releases page link (per R14) so the user can search further manually.
|
||||
- R13. **When a confident match is found**, the answer is a direct narrative response that cites the specific release version(s) the answer is drawn from (e.g., "The `deepen-plan` skill was renamed to `ce-debug` in `v2.45.0`"). Include a link to the cited release. The release body itself is a terse one-line conventional-commit bullet per change with a linked PR number; for query-mode synthesis the skill should follow the linked PR(s) (e.g., `gh pr view <N>`) to ground the narrative in the rich PR description rather than only the commit subject. (Verified against `v2.65.0`–`v2.67.0` release bodies and PR #568.)
|
||||
- R14. **When no confident match is found** (after expanding the search window per R12) **or the answer is uncertain**, say so plainly rather than guessing — and surface a link to the GitHub releases page so the user can investigate further.
|
||||
|
||||
## Success Criteria
|
||||
- A user who installed the plugin via the marketplace can run `/ce:release-notes` and immediately see what's shipped recently in the compound-engineering plugin (not CLI noise, not other plugins).
|
||||
- A user can ask `/ce:release-notes what happened to deepen-plan?` and get a direct narrative answer with a version citation, without having to open any browser tab.
|
||||
- The skill works for users without `gh` installed (silent anonymous-API fallback) and produces a clear error only when both access paths fail.
|
||||
|
||||
## Scope Boundaries
|
||||
- **Out of scope:** Coverage of `cli`, `coding-tutor`, `marketplace`, or `cursor-marketplace` releases. Only `compound-engineering` plugin releases are surfaced.
|
||||
- **Out of scope:** "What's coming next" / unreleased changes. The skill does not peek at the open release-please PR. Only shipped releases are summarized.
|
||||
- **Out of scope:** Local caching, CHANGELOG.md parsing, or any source other than the GitHub Releases API.
|
||||
- **Out of scope:** Per-PR or per-commit drill-down *as a primary user-facing surface*. Query mode may follow PR links for context (per R13), but the skill does not browse arbitrary commits or expose PR-level navigation as a separate mode.
|
||||
- **Out of scope:** Customization flags for window size or output format in v1. Defaults are fixed; users can ask follow-up questions in chat to drill deeper.
|
||||
|
||||
## Key Decisions
|
||||
- **Plugin-only filter (excludes `cli-v*`):** Linked versions mean a `2.67.0` bump can contain CLI-only or plugin-only changes; surfacing both would dilute the user-facing signal. Users who care about plugin behavior should not have to mentally filter CLI noise.
|
||||
- **GitHub Releases over CHANGELOG.md:** GitHub Releases are authoritative for what shipped, are accessible without a repo checkout (most plugin users won't have one), and the release-please-generated body is already markdown-grouped and ready to display.
|
||||
- **Slash-only invocation in v1 (no auto-invoke):** No sibling `ce:*` skill currently auto-invokes. Making this the first one introduces a hard-to-validate gating problem (the skill description is the only lever, and the failure modes are silent — either firing on unrelated projects' "what's new?" prompts, or never firing for actual CE-shaped questions). Slash-only satisfies both stated user journeys (`/ce:release-notes` bare summary and `/ce:release-notes <question>`) without the gating risk. Auto-invoke is deferred to a possible v2 once dogfooding shows the conversational triggering is genuinely wanted and a tested gating description exists.
|
||||
- **Layered data access (`gh` preferred, anonymous public API fallback):** The repo is public, so anonymous reads work and the 60 req/hr unauth'd limit is far above this skill's invocation frequency. Layering means users without `gh` installed still get value rather than bouncing on an "install gh and retry" message. Prefer `gh` when present for cleaner error handling, consistent JSON output, and authenticated rate limits.
|
||||
- **No local caching:** `gh release list` is fast (~1s for metadata; bodies add some cost) and release queries are infrequent; caching adds carrying cost (invalidation, location in `.context/`) without meaningful payoff. Reversal cost is low — caching can be added later if real latency or frequency problems show up.
|
||||
- **Two-mode design instead of always-query:** A bare-invocation summary serves the casual "what have I missed?" use case, which is materially different from "what specifically happened to X?". One skill covers both with a clean argument convention.
|
||||
- **Distinct from the existing `changelog` skill:** The plugin already ships a `changelog` skill that produces witty daily/weekly changelog summaries of recent activity. That serves a different use case (narrative recap of work) than this skill's version-aware release-notes lookup against shipped GitHub Releases. The two are complementary, not redundant.
|
||||
|
||||
## Dependencies / Assumptions
|
||||
- Users have **either** the `gh` CLI (preferred path) **or** outbound HTTPS access to `api.github.com` (anonymous fallback path). Per R5, missing `gh` alone is not a failure.
|
||||
- The 60 req/hr anonymous limit is per source IP, not per user. Users on shared NAT egress (corporate networks, VPN exit nodes) could in principle exhaust the budget collectively even at low individual usage. We accept this as low-likelihood given the skill's invocation pattern; if it surfaces in practice, encourage `gh auth login` rather than adding caching.
|
||||
- The repo `EveryInc/compound-engineering-plugin` remains the canonical source. (If the plugin moves repos, the hardcoded repo reference in the skill must be updated.)
|
||||
- Release-please continues to use the `compound-engineering-v*` tag prefix and the conventional-commit-grouped release body format. A change to release-please configuration could break R6 or R10.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
- [Affects R10][Technical] Should the summary impose a maximum-length cap on individual release bodies (separate from R10's no-uniform-highlight-count rule), to prevent a single 30-bullet release from dominating the summary view? Decide based on real release sizes during implementation.
|
||||
- [Affects R8][Technical] Exact failure messages when both access paths fail (network down, GitHub outage, anonymous rate-limit hit). Ensure they're actionable (point the user to the GitHub releases URL as a manual fallback).
|
||||
- [Affects R5][Technical] Implementation choice for the anonymous fallback: shell out to `curl` + `jq`, or use a different HTTP client. Decide based on cross-platform portability requirements (note: AGENTS.md "Platform-Specific Variables in Skills" rules apply since this skill will be converted for Codex/Gemini/OpenCode).
|
||||
- [Affects R13, R14][Technical] Define the "confident match" criterion that gates R13 (direct narrative answer) vs. R14 (say-so-plainly). Options include keyword/substring match against release bodies, semantic match via embedding, or LLM judgment with an explicit confidence prompt. Decide during planning based on cost and accuracy tradeoffs.
|
||||
- [Affects R4][Needs research] If/when v2 auto-invoke is reconsidered, define the actual gate. Since v1 has no auto-invoke surface to observe, "dogfooding shows users want it" is unfalsifiable as written — the v2 trigger needs a concrete source of evidence (explicit user requests, opt-in beta flag with telemetry, or a stated time-box for revisiting).
|
||||
- [Affects R5][Technical] Should the repo reference (`EveryInc/compound-engineering-plugin`) be hardcoded in the skill, or derived from `.claude-plugin/plugin.json` (`homepage`/`repository` field) for portability? Hardcoding is simpler; derivation survives a future repo move without skill edits. Decide based on portability vs. complexity tradeoff during planning.
|
||||
- [Affects R10][Technical] Release-please body format drift handling: R10 assumes the `Features`/`Bug Fixes` markdown grouping. Decide whether to (a) accept silent degradation if release-please config changes, (b) parse defensively and fall back to raw rendering, or (c) detect drift and surface a warning. Low priority — release-please config has been stable.
|
||||
|
||||
## Next Steps
|
||||
- `/ce:plan docs/brainstorms/2026-04-17-ce-release-notes-skill-requirements.md` for structured implementation planning.
|
||||
@@ -0,0 +1,155 @@
|
||||
---
|
||||
date: 2026-04-17
|
||||
topic: ce-review-interactive-judgment
|
||||
---
|
||||
|
||||
# ce:review Interactive Judgment Loop
|
||||
|
||||
## Problem Frame
|
||||
|
||||
`ce:review`'s Interactive mode produces a report, auto-applies `safe_auto` fixes, and then asks a single bucket-level policy question covering every remaining `gated_auto` and `manual` finding as a group. The findings themselves are presented as a pipe-delimited table grouped by severity.
|
||||
|
||||
Two problems surface repeatedly:
|
||||
|
||||
1. **Judgment calls are hard to make.** When a finding needs human judgment, the table row rarely gives enough context to decide confidently. The user is asked to approve or defer a bucket of findings they haven't individually understood.
|
||||
2. **High-volume feedback is unreason-able.** A review producing 8-12 findings turns into a scrolling table the user can't engage with. There's no way to respond to individual items meaningfully — the only choice is "approve the whole bucket" or "defer the whole bucket."
|
||||
|
||||
The result is that Interactive mode mostly degrades into rubber-stamping or wholesale deferral. The "judgment" in `gated_auto` / `manual` routing is never actually exercised per-finding.
|
||||
|
||||
## Requirements
|
||||
|
||||
**Routing after `safe_auto` fixes**
|
||||
|
||||
- R1. After `safe_auto` fixes are applied, if any `gated_auto` or `manual` findings remain, Interactive mode presents a four-option routing question that replaces today's bucket-level policy question.
|
||||
- R2. When zero `gated_auto` / `manual` findings remain after `safe_auto`, the routing question is skipped. Interactive mode shows a brief completion summary (e.g., "All findings resolved — N `safe_auto` fixes applied.") before handing off to the final-next-steps flow.
|
||||
- R3. The routing question names the detected tracker inline (e.g., "File a Linear ticket per finding") only when detection is high-confidence — the tracker is explicitly named in `CLAUDE.md` / `AGENTS.md` or equivalent project documentation. When detection is lower-confidence, the label uses a generic form (e.g., "File an issue per finding") and the agent confirms the tracker with the user before executing any ticket creation.
|
||||
- R4. The four routing options are:
|
||||
- (A) `Review each finding one by one — accept the recommendation or choose another action`
|
||||
- (B) `LFG. Apply the agent's best-judgment action per finding`
|
||||
- (C) `File a [TRACKER] ticket per finding without applying fixes`
|
||||
- (D) `Report only — take no further action`
|
||||
- R5. Routing option C is a batch-defer shortcut: it files tickets for every pending `gated_auto` / `manual` finding without per-finding confirmation. The walk-through's own Defer option is per-finding; C skips that interactivity.
|
||||
|
||||
**Per-finding walk-through (routing option: Review)**
|
||||
|
||||
- R6. When the user picks the walk-through, findings are presented one at a time in severity order (P0 first). Each per-finding question opens with a position indicator (e.g., "Finding 3 of 8 (P1):") so the user can judge how many decisions remain.
|
||||
- R7. Each per-finding question includes: plain-English statement of what the bug does, severity, confidence, the proposed fix (diff or concrete action), and a short reasoning for why the fix is right (grounded in codebase patterns when available).
|
||||
- R8. Per-finding options:
|
||||
- `Apply the proposed fix`
|
||||
- `Defer — file a [TRACKER] ticket`
|
||||
- `Skip — don't apply, don't track`
|
||||
- `LFG the rest — apply the agent's best judgment to this and remaining findings`
|
||||
- R9. For findings with no concrete fix to apply (advisory-only), option A becomes `Acknowledge — mark as reviewed`. Defer, Skip, and LFG the rest remain unchanged.
|
||||
- R10. "Override" on a per-finding question means picking a different preset action (Defer or Skip in place of Apply); no inline freeform custom fix authoring. A user who wants a custom fix picks Skip and hand-edits outside the flow.
|
||||
|
||||
When exactly one `gated_auto` / `manual` finding remains, the walk-through's wording adapts for N=1 (e.g., "Review the finding" rather than "Review each finding one by one"), and `LFG the rest` is suppressed because no subsequent findings exist to bulk-handle.
|
||||
|
||||
**LFG path (routing option: LFG)**
|
||||
|
||||
- R11. LFG applies the per-finding action the agent would have recommended in the walk-through — Apply, Defer, or Skip. There is no separate confidence threshold; confidence already shapes what the agent recommends. The top-level LFG option scopes to every `gated_auto` / `manual` finding; the walk-through's `LFG the rest` (R8) scopes to the current finding and everything not yet decided. Both share the same per-finding mechanic and the same bulk preview (R13-R14).
|
||||
- R12. LFG (and `LFG the rest`) produces a single completion report after execution that must include, at minimum:
|
||||
- per-finding entries with: title, severity, action taken (Applied / Deferred / Skipped / Acknowledged), the tracker URL or in-session task reference for Deferred entries, and a one-line reason for Skipped entries grounded in the finding's confidence or content
|
||||
- summary counts by action
|
||||
- any failures called out explicitly (fix application failed, ticket creation failed)
|
||||
- the existing end-of-review verdict
|
||||
|
||||
**Bulk action preview**
|
||||
|
||||
- R13. Before executing any bulk action — top-level LFG (routing option B), top-level File tickets (routing option C), or walk-through `LFG the rest` (R8) — Interactive mode presents a compact plan preview and asks the user to confirm with `Proceed` or back out with `Cancel`. Two options. No per-item decisions in the preview; per-item decisioning is the walk-through's role.
|
||||
- R14. The preview content groups findings by the action the agent intends to take (e.g., `Applying (N):`, `Filing [TRACKER] tickets (N):`, `Skipping (N):`, `Acknowledging (N):`). Each finding gets one line under its bucket, written as a compressed form of the framing-quality bar (R22-R25) — observable behavior over code structure, no function or variable names unless needed to locate the issue. For walk-through `LFG the rest`, the preview scopes to remaining findings only and notes how many are already decided (e.g., "LFG plan — 5 remaining findings (3 already decided)").
|
||||
|
||||
**Recommendation tie-breaking**
|
||||
|
||||
- R15. When merged findings carry conflicting recommendations across contributing reviewers (e.g., one reviewer says Apply, another says Defer), synthesis picks the most conservative action using the order `Skip > Defer > Apply` so that LFG and walk-through behavior are deterministic and auditable post-hoc.
|
||||
|
||||
**Defer behavior and tracker detection**
|
||||
|
||||
- R16. Defer actions (from the walk-through, from the LFG path, or from routing option C) file a ticket in the project's tracker.
|
||||
- R17. The SKILL.md instruction for tracker detection is minimal: the agent determines the project's tracker from whatever documentation is obvious (primarily `CLAUDE.md` / `AGENTS.md`), without an enumerated checklist of files to read.
|
||||
- R18. When tracker detection is uncertain, the agent prefers durable external trackers over in-session-only primitives and communicates both the fallback behavior and the durability trade-off to the user before executing any Defer action.
|
||||
- R19. If a Defer action fails at ticket-creation time (API error, auth expiry, rate limit, malformed body), the agent surfaces the failure inline and offers: retry, fall back to the next available sink, or convert the finding to Skip with the error recorded in the completion report. Silent failure is not acceptable.
|
||||
- R20. When no external tracker is detectable and no harness task-tracking primitive is available on the current platform (e.g., CI contexts, converted targets without task binding), Defer is not offered as a menu option. The routing question and walk-through omit Defer paths and the agent tells the user why.
|
||||
- R21. The internal `.context/compound-engineering/todos/` system is **not** part of the fallback chain. It is on a deprecation path and must not be extended by this work.
|
||||
|
||||
**Framing quality (cross-cutting)**
|
||||
|
||||
- R22. Every user-facing surface that describes a finding — per-finding walk-through questions, LFG completion reports, and ticket bodies filed by Defer actions — explains the problem and fix in plain English that a reader can understand without opening the file.
|
||||
- R23. The framing leads with the *observable behavior* of the bug (what a user, attacker, or operator sees), not the code structure. Function and variable names appear only when the reader needs them to locate the issue.
|
||||
- R24. The framing explains *why the fix works*, not just what it changes. When a similar pattern exists elsewhere in the codebase, reference it so the recommendation is grounded.
|
||||
- R25. The framing is tight: approximately two to four sentences plus the minimum code needed to ground it. Longer framings are a regression.
|
||||
|
||||
*Illustrative pair — weak vs. strong framing for the same finding:*
|
||||
|
||||
> **Weak (code-citation style):**
|
||||
> *orders_controller.rb:42 — missing authorization check. Add `current_user.owns?(account)` guard before query.*
|
||||
>
|
||||
> **Strong (framed for a human):**
|
||||
> *Any signed-in user can read another user's orders by pasting the target account ID into the URL. The controller looks up the account and returns its orders without verifying the current user owns it. Adding a one-line ownership guard before the lookup matches the pattern already used in the shipments controller for the same attack.*
|
||||
|
||||
- R26. R22-R25 depend on reviewer personas producing framing-suitable `why_it_matters` and `evidence` fields. If the planning-phase sample shows existing persona outputs do not meet this bar, persona prompt upgrades (or a synthesis-time rewrite pass) land with or before this work.
|
||||
|
||||
**Mode boundaries**
|
||||
|
||||
- R27. Only Interactive mode changes behavior. Autofix, Report-only, and Headless modes are unchanged.
|
||||
- R28. The existing post-review "final next steps" flow (push fixes / create PR / exit) runs only when one or more fixes were applied to the working tree. It is skipped after routing option C (File tickets per finding) and option D (Report only), and skipped when LFG or the walk-through completes without any Apply action.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- A user facing a review with one high-stakes finding can decide confidently about the fix without rereading the file.
|
||||
- A user facing a review with 8+ findings has a clear path to either engage per-item or trust the agent's judgment in one keystroke.
|
||||
- A user who starts the walk-through but runs out of attention can bail mid-flow into a bulk action without losing the findings still ahead of them.
|
||||
- Deferred findings land in the team's actual tracker (not a `.context/` file that gets forgotten).
|
||||
- LFG runs feel honest: the completion report makes clear what was applied and why, so a user can audit the agent's judgment post-hoc.
|
||||
- For reviews with three or more `gated_auto` / `manual` findings, Review is picked at a meaningful share of the time — LFG alone is not disproportionately the default, so the intervention actually shifts engagement upward rather than renaming the rubber-stamp.
|
||||
- A first-time user of Interactive mode understands which routing options cause external side effects (fixes applied to the working tree, tickets filed in an external tracker) before choosing, without needing external docs.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- No new `ce:fix` skill. All changes live inside `ce:review`.
|
||||
- No changes to the findings schema, persona agents, merge/dedup pipeline, or autofix-mode residual-todo creation in this work.
|
||||
- No inline freeform fix authoring in the walk-through. The walk-through is a decision loop, not a pair-programming surface.
|
||||
- The "approve the fix's intent but write a variant" case is explicitly unsupported in v1. Users in that situation pick Skip and hand-edit outside the flow; if they want the variant tracked, they file a ticket manually.
|
||||
- No changes to Autofix, Report-only, or Headless mode behavior.
|
||||
- The pre-menu findings table format (pipe-delimited, severity-grouped) is intentionally unchanged. The walk-through is the engagement surface for high-volume feedback; the table only needs to be scannable enough to reach the routing menu. Restructuring the table format is a separate follow-up if it proves necessary.
|
||||
- Phasing out the internal `.context/compound-engineering/todos/` system and the `/todo-create`, `/todo-triage`, `/todo-resolve` skills is acknowledged as the long-term direction but is not scoped into this redesign. A separate follow-up covers that cleanup.
|
||||
- The current bucket-level policy question wording (`Review and approve specific gated fixes` / `Leave as residual work` / `Report only`) is removed and replaced by the four-option routing question. No backward-compatibility shim.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Expand Interactive mode, no new skill.** Review and fix stay colocated; the review artifact, routing metadata, and fixer subagent are already wired up. A separate `ce:fix` skill would split state and add reintegration cost without clear benefit.
|
||||
- **Four-option routing upfront, not an escape hatch buried inside the walk-through.** LFG and tracker-deferral are legitimate primary intents for many reviews, not fallbacks. Offering them as peers to the walk-through is honest about how users actually want to engage.
|
||||
- **LFG = auto-accept recommendations, not a separate confidence policy.** Keeps the mental model simple. Confidence is already baked into whether the agent recommends Apply, Defer, or Skip for a given finding.
|
||||
- **Tracker detection is reasoning-based, not rote.** Agents are smart enough to read the obvious documentation. An enumerated checklist of files in SKILL.md is pure rationale-discipline tax and caps the agent at the sources we happened to list.
|
||||
- **Harness task tracking is the last-resort fallback, not internal todos.** Aligns with the deprecation direction for the internal todo system. Honest about the fact that in-session tasks don't survive past the session.
|
||||
- **Override in the walk-through = pick a different preset action.** No freeform custom fixes. Keeps the interaction a decision loop and avoids turning it into a pair-programming transcript. Users who want custom fixes Skip and hand-edit.
|
||||
- **Internal-todos deprecation ships a durability regression for some users.** A subset of users today treat `.context/compound-engineering/todos/` as persistent defer storage; removing it from the fallback chain means those users lose cross-session durability for Defer actions until they either document a tracker in `CLAUDE.md` / `AGENTS.md` or the broader phase-out lands. The trade is acknowledged and deliberate, not a silent regression; the mitigation is the separate phase-out cleanup referenced in Scope Boundaries.
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- The cross-platform blocking question tool (`AskUserQuestion` / `request_user_input` / `ask_user`) caps at 4 options. All menu designs respect this.
|
||||
- Option labels across every menu in the flow (routing question, per-finding question, Stop-asking follow-up) must be self-contained, use third-person voice for the agent, and front-load a distinguishing word so they survive truncation in harnesses that hide description text.
|
||||
- The walk-through writes per-finding decisions to the run artifact (e.g., `.context/compound-engineering/ce-review/<run-id>/walkthrough-state.json`) after each decision, so partial progress is inspectable post-hoc. Formal cross-session resumption is out of scope.
|
||||
- Findings already carry enough detail (title, severity, confidence, file, line, autofix_class, suggested_fix, why_it_matters, evidence) to support the framing requirements. If some reviewers don't reliably produce plain-English `why_it_matters`, the framing quality bar may require prompt upgrades to those personas — flagged below as a question for planning.
|
||||
- The existing per-run artifact directory (`.context/compound-engineering/ce-review/<run-id>/`) and the fixer subagent flow remain the underlying mechanics for applying fixes.
|
||||
- The merged finding set produced by the existing Stage 5 merge pipeline carries only merge-tier fields; detail-tier fields (`why_it_matters`, `evidence`) live in the per-agent artifact files on disk. The per-finding walk-through enriches each merged finding by reading the contributing reviewer's artifact file at `.context/compound-engineering/ce-review/<run-id>/{reviewer}.json`, using the same `file + line_bucket(line, +/-3) + normalize(title)` matching that headless mode already uses. When no artifact match exists (merge-synthesized finding, or failed artifact write), the walk-through degrades to title plus `suggested_fix` and notes the gap.
|
||||
- The four-option routing design is built to the cross-platform question tool's 4-option cap. A future fifth primary routing intent would require replacing an existing option, chaining a follow-up question, or pressuring the platform cap — the design does not provide pressure relief for this case.
|
||||
- Autofix mode continues to write residual actionable work to `.context/compound-engineering/todos/` in this redesign, while Interactive-LFG and Defer actions route to external trackers per R16-R21. This temporary divergence is acknowledged — aligning autofix mode's residual sink with the new tracker routing is separate cleanup work tracked in the follow-up referenced in Scope Boundaries.
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Resolve Before Planning
|
||||
|
||||
None. All product decisions are made.
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects Problem Frame][Needs research] Sample recent `.context/compound-engineering/ce-review/<run-id>/` run artifacts to confirm the rubber-stamping / wholesale-deferral failure mode the Problem Frame asserts. If the dominant failure is something else (users disengage before the bucket question, report itself is unreadable), the four-option routing may not be the right intervention.
|
||||
- [Affects R22-R26][Technical] Do reviewer personas reliably produce plain-English `why_it_matters` today, or does the framing bar require prompt upgrades and/or a synthesis-time rewrite pass? Planning should inspect a sample of recent review artifacts to decide before committing to R22-R25 as achievable without persona changes.
|
||||
- [Affects R18][Technical] The concrete sequencing of the fallback chain on each target platform (e.g., GitHub Issues via `gh` vs harness task tracking, how to detect `gh` availability cheaply) is intentionally left out of the requirements so detection stays principle-based. Planning resolves the specific sequencing and detection heuristics per target environment.
|
||||
- [Affects R18][Technical] If no documented tracker is found and `gh` is unavailable on the current platform, should the fallback to harness task tracking happen silently or should the agent confirm once per session? Default expectation: confirm once so users are not surprised by in-session-only behavior.
|
||||
- [Affects R6][Technical] Whether the walk-through presents findings strictly in severity order (current default) or groups them by file first and then severity within each file. File-grouping may feel more coherent when many findings touch the same file, but it interacts with `Stop asking` semantics (a file-grouped bulk-accept applies to different findings than a severity-first bulk-accept).
|
||||
- [Affects R7][Needs validation] Whether surfacing reviewer persona names in each per-finding question (e.g., `julik-frontend-races-reviewer`) helps user judgment or is noise. If validation shows noise, omit reviewer attribution from R7's required content or replace with a short category label.
|
||||
|
||||
## Next Steps
|
||||
|
||||
`-> /ce:plan` for structured implementation planning
|
||||
@@ -0,0 +1,157 @@
|
||||
---
|
||||
date: 2026-04-18
|
||||
topic: ce-doc-review-autofix-and-interaction
|
||||
---
|
||||
|
||||
# ce-doc-review Autofix and Interaction Overhaul
|
||||
|
||||
## Problem Frame
|
||||
|
||||
`ce-doc-review` consistently produces painful reviews. It surfaces too many findings as "requires judgment" when one reasonable fix exists, nitpicks on low-confidence items, and hands the user a wall of prose with only two terminal options — "refine and re-review" or "review complete." The interaction model lags behind what `ce-code-review` now offers (per PR #590): per-finding walk-through, LFG, bulk preview, tracker defer, and a recommendation-stable routing question.
|
||||
|
||||
A real-world review of a plan document produced **14 findings all routed to "needs judgment"** — including five P3 findings at 0.55–0.68 confidence, three concrete mechanical fixes that a competent implementer would arrive at independently, and one subjective filename-symmetry observation that didn't need a decision at all. The user had to parse 14 prose blocks, pick answers, and then was forced into a re-review regardless of how little the edits actually changed.
|
||||
|
||||
The gaps are structural and line up with four observable failure modes:
|
||||
|
||||
1. **Classification is binary and coarse.** `autofix_class` is `auto` or `present`. There is no `gated_auto` tier (concrete fix, minor sign-off) and no `advisory` tier (report-only FYI). Everything that isn't "one clear correct fix with zero judgment" becomes `present`, which conflates high-stakes strategic decisions with small mechanical follow-ups.
|
||||
2. **Confidence gate is flat and too low.** A single 0.50 threshold across all severities lets borderline P3s through. `ce-code-review` moved to 0.60 with P0-only survival at 0.50+.
|
||||
3. **"Reasonable alternative" test is permissive.** Persona reviewers list `(a) / (b) / (c)` fix options where (b) and (c) are strawmen ("accept the regression," "document in release notes," "do nothing"). The classification rule reads those as multiple reasonable fixes and routes the finding to `present`, when in fact only (a) is a real option.
|
||||
4. **Subagent framing and interaction model are pre-PR-590.** No observable-behavior-first framing guidance, no walk-through, no bulk preview, no per-severity confidence calibration, no post-fix "apply and proceed" exit — every path that addresses findings forces a re-review, even when the user is done.
|
||||
|
||||
## Requirements
|
||||
|
||||
**Classification tiers**
|
||||
|
||||
- R1. `autofix_class` expands from two values to four: `auto`, `gated_auto`, `advisory`, `present`. Values preserve the existing "is there one correct fix" axis but add (a) a tier for concrete fixes that touch document scope / meaning and should be user-confirmed (`gated_auto`), and (b) a tier for report-only observations with no decision to make (`advisory`).
|
||||
- R2. `auto` findings are applied silently, same as today. The promotion rules in the synthesis pipeline (current steps 3.6 and 3.7) are sharpened per R4 below and carry the new strictness forward.
|
||||
- R3. `gated_auto` findings carry a concrete `suggested_fix` and a user-confirmation requirement. They enter the per-finding walk-through (R13) with `Apply the proposed fix` marked `(recommended)`. They are the default tier for "concrete fix exists, but it changes what the document says in a way the author should sign off on" (e.g., adding a backward-compatibility read-fallback, requiring two units land in one commit, substituting a framework-native API for a hand-rolled one).
|
||||
- R4. `advisory` findings are report-only. They surface in a compact FYI block in the final output and do not enter the walk-through or any bulk action. Subjective observations ("filename asymmetry — could go either way"), drift notes without actionable fixes, and low-stakes calibration gaps live here.
|
||||
- R5. `present` findings remain for genuinely strategic / scope / prioritization decisions where multiple reasonable approaches exist and the right choice depends on context the reviewer doesn't have.
|
||||
|
||||
**Classification rule sharpening**
|
||||
|
||||
- R6. The subagent-template classification rule adds teeth: "a 'do nothing / accept the defect' option is not a real alternative — it's the failure state the finding describes." If the only listed alternatives to the primary fix are strawmen, the finding is `auto` (or `gated_auto` if confirmation is warranted), not `present`. This applies equally to "document in release notes," "accept drift," and other deferral framings that sidestep the actual problem.
|
||||
- R7. Auto-promotion patterns already scattered in prose (steps 3.6 and 3.7) are consolidated into an explicit promotion rule set, covering:
|
||||
- Factually incorrect behavior where the correct behavior is derivable from context or the codebase
|
||||
- Missing standard security / reliability controls with established implementations (HTTPS, fallback-with-deprecation-warning, input sanitization, checksum verification, private IP rejection, etc.)
|
||||
- Codebase-pattern-resolved fixes that cite a concrete existing pattern
|
||||
- Framework-native-API substitutions when a hand-rolled implementation duplicates first-class framework behavior (e.g., cobra's `Deprecated` field)
|
||||
- Completeness additions mechanically implied by the document's own explicit decisions
|
||||
- R8. The subagent template includes a framing-guidance block (ported from the `ce-code-review` shared template): observable-behavior-first phrasing, why-the-fix-works grounding, 2-4 sentence budget, required-field reminder, positive/negative example pair. One file change, applied universally across all seven personas.
|
||||
|
||||
**Per-severity confidence gates**
|
||||
|
||||
- R9. The single 0.50 confidence gate is replaced with per-severity gates:
|
||||
- P0: survive at 0.50+
|
||||
- P1: survive at 0.60+
|
||||
- P2: survive at 0.65+
|
||||
- P3: survive at 0.75+
|
||||
- R10. The residual-concern promotion step (current step 3.4) is dropped. Cross-persona agreement instead boosts the confidence of findings that already survived the gate (by +0.10, capped at 1.0), mirroring `ce-code-review` stage 5 step 4. Residual concerns surface in Coverage only.
|
||||
- R11. `advisory` findings are exempt from the confidence gate — they are report-only and can't generate false-positive work even at lower confidence. This is the safety valve for observations the reviewer wants on record but doesn't want to escalate.
|
||||
|
||||
**Interaction model (post-fix routing)**
|
||||
|
||||
- R12. After `auto` fixes are applied and before any user interaction, Interactive mode presents a four-option routing question that mirrors `ce-code-review`'s post-PR-590 design:
|
||||
- (A) `Review each finding one by one — accept the recommendation or choose another action`
|
||||
- (B) `LFG. Apply the agent's best-judgment action per finding`
|
||||
- (C) `Append findings to the doc's Open Questions section and proceed` (ce-doc-review analogue of ce-code-review's "file a tracker ticket" — for docs, "defer" means appending the findings to a `## Deferred / Open Questions` section within the document itself, not an external system)
|
||||
- (D) `Report only — take no further action`
|
||||
If zero `gated_auto` / `present` findings remain after the `auto` pass, the routing question is skipped and the flow falls directly into the terminal question (R19).
|
||||
- R13. Routing option A enters a per-finding walk-through, presented one finding at a time in severity order (P0 first). Each per-finding question carries: position indicator (`Finding N of M`), severity, confidence, a plain-English statement of the problem, the proposed edit, and a short reasoning grounded in the document's own content or the codebase. Options: `Apply the proposed fix` / `Defer — append to the doc's Open Questions section` / `Skip — don't apply, don't append` / `LFG the rest — apply the agent's best judgment to this and remaining findings`. Advisory-only findings substitute `Acknowledge — mark as reviewed` for Apply.
|
||||
- R14. Routing option B and walk-through `LFG the rest` execute the agent's per-finding recommended action across the selected scope (all pending findings for B, remaining-undecided for walk-through). The recommendation for each finding is determined deterministically by R16.
|
||||
- R15. Before any bulk action executes (routing B, routing C, walk-through `LFG the rest`), a compact plan preview renders findings grouped by intended action (`Applying (N):`, `Appending to Open Questions (N):`, `Skipping (N):`, `Acknowledging (N):`) with a one-line summary per finding. Exactly two responses: `Proceed` or `Cancel`. Cancel from walk-through `LFG the rest` returns the user to the current finding, not to the routing question.
|
||||
|
||||
**Recommendation tie-breaking**
|
||||
|
||||
- R16. When merged findings carry conflicting recommendations across contributing personas (one says Apply, another says Defer), synthesis picks the most conservative using `Skip > Defer > Apply > Acknowledge`, so walk-through recommendations and LFG behavior are deterministic across re-runs.
|
||||
|
||||
**Terminal "next step" question (the re-review fix)**
|
||||
|
||||
- R17. The current Phase 5 binary question (`Refine — re-review` / `Review complete`) conflates "apply fixes" with "re-review" into a single option. This is replaced by a three-option terminal question that separates the two axes:
|
||||
- (A) `Apply decisions and proceed to <next stage>` — for requirements docs, hand off to `ce-plan`; for plan docs, hand off to `ce-work`. Default / recommended when fixes were applied or decisions were made.
|
||||
- (B) `Apply decisions and re-review` — opt-in re-review when the user believes the edits warrant another pass.
|
||||
- (C) `Exit without further action` — user wants to stop for now.
|
||||
When zero actionable findings remain (everything was `auto` or `advisory`), option B is omitted — re-review is not useful when there's nothing to re-examine.
|
||||
- R18. The terminal question is distinct from the mid-flow routing question (R12). The routing question chooses *how* to engage with findings; the terminal question chooses *what to do next* once engagement is complete. The two are asked separately, not merged.
|
||||
- R19. The zero-findings degenerate case (no `gated_auto` / `present` findings after the `auto` pass) skips the routing question entirely and proceeds directly to the terminal question with option B suppressed.
|
||||
|
||||
**In-doc deferral (Defer analogue)**
|
||||
|
||||
- R20. Document-review's `Defer` action appends the deferred finding to a `## Deferred / Open Questions` section at the end of the document under review. If the heading does not exist, it is created on first defer within a review. Multiple deferred findings from a single review accumulate under a single timestamped subsection (e.g., `### From 2026-04-18 review`) to keep sequential reviews distinguishable. This replaces `ce-code-review`'s tracker-ticket mechanic with a document-native analogue: deferred findings stay attached to the document they came from.
|
||||
- R21. The appended entry for each deferred finding includes: title, severity, reviewer attribution, confidence, and the `why_it_matters` framing — enough context that a reader returning to the doc later can understand the concern without re-running the review. The entry does not include `suggested_fix` or `evidence` — those live in the review run artifact and can be looked up if needed.
|
||||
- R22. When the append fails (document is read-only, path issue, write failure), the agent surfaces the failure inline and offers: retry, fall back to recording the deferral in the completion report only, or convert the finding to Skip. Silent failure is not acceptable.
|
||||
|
||||
**Framing quality in reviewer output**
|
||||
|
||||
- R23. Every user-facing surface that describes a finding — walk-through questions, LFG completion reports, Open Questions entries — explains the problem and fix in plain English. The framing leads with the *observable consequence* of the issue (what an implementer, reader, or downstream caller sees), not the document's structural phrasing.
|
||||
- R24. The framing explains *why the fix works*, not just what it changes. When a pattern exists elsewhere in the document or codebase, reference it so the recommendation is grounded.
|
||||
- R25. The framing is tight — approximately two to four sentences. Longer framings are a regression.
|
||||
|
||||
**Cross-cutting**
|
||||
|
||||
- R26. Tool-loading pre-flight mirrors `ce-code-review`: on Claude Code, `AskUserQuestion` is pre-loaded once at the start of Interactive mode via `ToolSearch` (`select:AskUserQuestion`), not lazily per-question. The numbered-list text fallback applies only when `ToolSearch` explicitly returns no match or the tool call errors.
|
||||
- R27. Headless mode behavior is preserved. `mode:headless` continues to apply `auto` fixes silently and return all other findings as structured text to the caller. The caller owns routing. New tiers (`gated_auto`, `advisory`) must appear distinctly in headless output so callers can route them appropriately.
|
||||
|
||||
**Multi-round decision memory**
|
||||
|
||||
- R28. Every review round after the first passes a cumulative decision primer to every persona, carrying forward all prior rounds' decisions in the current interactive session: rejected findings (Skipped / Deferred from any prior round) with title, evidence quote, and rejection reason; plus Applied findings from any prior round with title and section reference. Personas still receive the full current document as their primary input. No diff is passed — fixed findings self-suppress because their evidence no longer exists, regressions surface as normal findings on the current doc, and rejected findings are handled by the suppression rule in R29.
|
||||
- R29. Personas must not re-raise a finding whose title and evidence pattern-match a finding rejected in any prior round, unless the current document state makes the concern materially different. The orchestrator drops any finding that would violate this rule and records the drop in Coverage.
|
||||
- R30. For each prior-round Applied finding, synthesis confirms the fix landed by checking that the specific issue the finding described no longer appears in the referenced section. If a persona re-surfaces the same finding at the same location, synthesis flags it as "fix did not land" in the final report rather than treating it as a new finding.
|
||||
|
||||
**Institutional memory (learnings-researcher integration)**
|
||||
|
||||
- R31. `ce-doc-review` dispatches `research:ce-learnings-researcher` as an always-on agent, in parallel with coherence-reviewer and feasibility-reviewer. The agent owns its own fast-exit behavior when `docs/solutions/` is empty or absent — no activation-gating in the orchestrator.
|
||||
- R32. The orchestrator produces a compressed search seed during Phase 1's classify-and-select step: document type, 3-5 topic keywords extracted from the doc, named entities (tools, frameworks, patterns explicitly named), and the doc's top-level decision points. Learnings-researcher receives the search seed plus the document path, not the full document content. It searches `docs/solutions/` by frontmatter metadata first, then selectively reads matching solution bodies.
|
||||
- R33. Learnings-researcher returns, per match: the solution doc's path, a one-line relevance reason, and the specific claim in the doc under review that the past solution relates to. Full solution content is loaded on demand by other personas or the orchestrator if the match is promoted into a finding. Results are capped at a small N (default 5) most relevant matches — past-solution volume is not the goal; directly applicable grounding is.
|
||||
- R34. Learnings-researcher output surfaces in a dedicated "Past Solutions" section of the review output. Entries default to `advisory` tier (report-only grounding) unless a past solution directly contradicts a specific claim in the document under review, in which case they promote to `gated_auto` or `present` with the past solution's path as evidence.
|
||||
- R35. Learnings-researcher content does not participate in confidence-gating (R9) or cross-persona dedup (existing step 3.3). Its role is to add institutional memory, not to compete with persona findings for user attention.
|
||||
|
||||
**learnings-researcher agent rewrite (bundled)**
|
||||
|
||||
- R36. Rewrite `research:ce-learnings-researcher` to treat the `docs/solutions/` corpus as domain-agnostic institutional knowledge. Code bugs are one genre among several, alongside skill-design patterns, workflow learnings, developer-experience discoveries, integration gotchas, and anything else captured by `ce-compound` and its refresh counterpart. The agent's primary function is "find applicable past learnings given a work context," not "find past bugs given a feature description."
|
||||
- R37. The agent accepts a structured `<work-context>` input from callers: a short description of what the caller is working on or considering, a list of key concepts / decisions / domains / components extracted from the caller's work, and an optional domain hint when one applies cleanly (e.g., `skill-design`, `workflow`, `code-implementation`). No mode flag is required — the context shape adapts to the calling skill without the agent branching on caller identity.
|
||||
- R38. The hardcoded category-to-directory table is replaced with a dynamic probe of `docs/solutions/` to discover available subdirectories at runtime. Category narrowing uses the discovered set. The agent no longer assumes which subdirectories exist in a given repo.
|
||||
- R39. Keyword extraction handles decision-and-approach-shape content alongside symptom-and-component-shape content. The extraction taxonomy expands from the current four dimensions (Module names, Technical terms, Problem indicators, Component types) to include Concepts, Decisions, Approaches, and Domains. No input shape is privileged over another; the caller's context determines which dimensions carry weight.
|
||||
- R40. Output framing drops code-bug-biased phrasing ("gotchas to avoid during implementation," "prevent repeated mistakes" framed narrowly around bugs) in favor of neutral institutional-memory framing ("applicable past learnings," "related decisions and their outcomes"). The pointer + one-line-relevance + key-insight summary format carries across all input genres.
|
||||
- R41. Read `docs/solutions/patterns/critical-patterns.md` only when it exists. When absent, the agent proceeds without it — this file is a per-repo convention, not a protocol requirement.
|
||||
- R42. The agent's Integration Points section documents invocation by `/ce-plan`, `/ce-code-review`, `ce-doc-review`, and any other skill benefiting from institutional memory. Remove the framing that implies planning-time is the agent's primary home.
|
||||
|
||||
**Frontmatter enum expansion (bundled)**
|
||||
|
||||
- R43. Expand the `ce-compound` frontmatter `problem_type` enum to add non-bug genre values: `architecture_pattern`, `design_pattern`, `tooling_decision`, `convention`. Document `best_practice` as the fallback for entries not covered by any narrower value, not the default. Migrate the 8 existing `best_practice` entries that fit a narrower value (3 architecture patterns, 3 design patterns, 1 tooling decision, 1 remaining as best_practice), and resolve the one `correctness-gap` schema violation (`workflow/todo-status-lifecycle.md`) into a valid enum value. Update `ce-compound` and `ce-compound-refresh` so they steer authors toward narrower values when the new categories apply.
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Not introducing a document-native tracker integration (e.g., Linear / Jira / GitHub Issues). Document-review's Defer analogue is an in-doc `## Deferred / Open Questions` section. If users later want tracker integration for doc findings, that's a follow-up proposal.
|
||||
- Not changing persona selection logic. The seven personas and the activation signals for conditional ones stay as-is. The persona markdown files themselves change only to absorb the subagent-template framing-guidance block.
|
||||
- Not changing headless mode's structural contract with callers (`ce-brainstorm`, `ce-plan`). Headless continues to apply `auto` fixes silently and return a structured text envelope. Callers must be updated to handle the new `gated_auto` and `advisory` tiers but the envelope shape stays.
|
||||
- Not adding a `requires_verification` field or an in-skill fixer subagent. Document edits happen inline during the walk-through; there is no batch-fixer analogue to `ce-code-review`'s Step 3 fixer because document fixes are trivially confined in scope (single-file markdown edits).
|
||||
- Not addressing iteration-limit guidance. The existing "after 2 refinement passes, recommend completion" heuristic stays.
|
||||
- Not persisting decision primers across interactive sessions. The cumulative decision list (R28) lives in-memory across rounds within a single invocation. A new invocation of `ce-doc-review` on the same doc starts fresh with no carried memory, even if prior-session decisions were Applied to the document. Mirrors `ce-code-review` walk-through state rules.
|
||||
- Not building a fully new frontmatter schema. R43 adds non-bug enum values but does not redesign the schema dimensions (no split into `learning_category` + `problem_type`, no new required fields). The existing authoring flow stays the same; only the set of valid `problem_type` values grows.
|
||||
|
||||
## Design Decisions Worth Calling Out
|
||||
|
||||
- **Three new tiers, not two.** A minimal refactor could add only `gated_auto` and keep `advisory` collapsed into `present`. But real-world evidence shows FYI-grade findings (subjective observations, low-stakes drift notes) drive significant noise, and folding them into `present` forces user decisions on things that don't warrant any decision. Adding `advisory` as a distinct tier is cheap (one enum value + one output block) and materially reduces decision fatigue.
|
||||
- **Strawman-aware classification rule in the subagent template, not in synthesis.** Moving the rule to synthesis means persona reviewers still emit inflated alternative lists and the orchestrator retroactively collapses them. Moving it to the subagent template changes what reviewers produce at the source, so the evidence and framing travel together correctly.
|
||||
- **Per-severity confidence gates, not a flat 0.60.** A flat 0.60 would still let 0.60–0.68 P3 nits through (three of them in the attached real-world example). Severity-aware gates recognize that a P3 finding at 0.65 is noise in a way a P1 at 0.65 is not, because P3 impact is low enough that the expected value of a borderline call doesn't justify the user's attention.
|
||||
- **Separate terminal question from routing question.** The current skill conflates "engage with findings" and "exit the review" into one question with two poorly-aligned options. Splitting them gives the user explicit control over whether re-review happens — the most common user frustration surfaced in the bug report that prompted this work.
|
||||
- **In-doc Open Questions section, not a sibling follow-up note or external tracker, as Defer analogue.** Documents don't have the same "handoff to a different system" shape that code findings do. A sibling markdown note would fragment context; an external tracker would add platform complexity with no upside for document review. Appending deferred findings to a `## Deferred / Open Questions` section inside the document itself keeps deferred concerns attached to the artifact they came from, is naturally discoverable by anyone reading the doc, and requires no new infrastructure. The trade-off is that deferred findings visibly mutate the doc — but that is the point: "I want to remember this but not act now" is exactly what an Open Questions section expresses in a planning doc.
|
||||
- **Port framing-guidance once via the shared subagent template.** Matches how `ce-code-review` shipped the same fix in PR #590. One file change, applied universally. Per-persona edits would inflate scope to seven files; a synthesis-time rewrite pass would add per-review model cost and paper over the root cause in the persona output itself.
|
||||
- **Classification-rule sharpening and promotion-pattern consolidation ship together with the tier expansion.** Shipping the tiers without the sharpened rule would leave the classifier behavior unchanged and just add new tier labels nothing routes to. Shipping the rule without the tiers has no tier to promote findings into.
|
||||
- **Keep the existing persona markdown files mostly unchanged.** The framing-guidance block lives in the shared subagent template that wraps every persona dispatch; the personas themselves retain their confidence calibration, suppress conditions, and domain focus. This keeps the persona-level failure-mode catalogs stable while upgrading the shared framing bar.
|
||||
- **No diff passed to the multi-round decision primer.** Fixed findings self-suppress because their evidence is gone from the current doc; regressions surface as normal findings; rejected findings are handled by the suppression rule (R29). A diff would be signal amplification, not a correctness requirement, and would add prompt weight without changing what the agent can do.
|
||||
- **learnings-researcher rewrite bundled, not split.** The review-time use case has no consumer without ce-doc-review, so splitting into a precursor PR would ship a dormant feature. Bundling keeps the change coherent and easier to review as one unit. The agent rewrite (R36–R42) and the frontmatter enum expansion (R43) also benefit `/ce-plan`'s existing usage, so the scope investment pays off beyond ce-doc-review.
|
||||
- **Generalize learnings-researcher rather than patch with a mode flag.** The original proposal was a minimal `review-time` mode flag grafted onto the agent. But the real issue is that the agent's taxonomy, categories, and output framing are code-bug-shaped even when invoked by non-review callers — the plugin already captures non-code learnings via `ce-compound` / `ce-compound-refresh`, and the agent should treat them as first-class. Rewriting for domain-agnostic institutional knowledge is a bigger change but removes the drift, rather than accumulating special cases.
|
||||
- **Expand `problem_type` rather than introduce a new orthogonal dimension.** A cleaner design might split current `problem_type` into separate `learning_category` (genre) and `problem_type` (bug-shape detail) fields. But that requires migrating every existing entry and teaching authors to pick both. Expanding the existing enum with non-bug values absorbs the `best_practice` overflow with minimal schema churn and keeps the authoring flow stable.
|
||||
|
||||
## Calibration Against Real-World Example
|
||||
|
||||
The attached review output (14 findings, all `present`) re-classifies under the proposed rules as:
|
||||
|
||||
- **4 `auto`** (silently applied, no user interaction): missing fallback-with-deprecation-warning (industry-standard pattern), public-repo grep step (single action), deployment-coupling-commit guarantee (mechanical), cobra's native `Deprecated` field (framework-native substitution).
|
||||
- **1 `advisory`** (FYI line): filename asymmetry — genuinely ambiguous, no wrong answer.
|
||||
- **4 `present`** (walk-through): historical-docs rule, alias-compatibility breaking-change, escape-hatch scope decision, Unit merging decision.
|
||||
- **5 dropped** by per-severity gates: five P3-P2 findings at 0.55–0.68 confidence.
|
||||
|
||||
Net: the user sees **4 decisions**, not 14. The walk-through's `LFG the rest` escape further bounds fatigue — after the user calibrates on the agent's recommendations, they can bail and accept the rest.
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
date: 2026-04-22
|
||||
topic: demo-reel-local-save
|
||||
---
|
||||
|
||||
# Demo Reel: Local Evidence Save
|
||||
|
||||
## Problem Frame
|
||||
|
||||
When `ce-demo-reel` captures evidence (GIFs, screenshots, terminal recordings), the local artifacts are deleted after uploading to catbox.moe. Users who want to keep evidence locally — for offline access, committing to the repo, or archival — have no way to do so without manually copying files from the temp directory before cleanup runs.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
**Destination choice**
|
||||
- R1. After capture completes, ask the user whether to upload to catbox (existing behavior) or save locally.
|
||||
- R2. The question must present the captured artifact(s) and clearly describe both options.
|
||||
|
||||
**Local save behavior**
|
||||
- R3. When the user chooses local save, copy the final artifact(s) (GIF, PNG, or recording) to a stable OS-temp path (`$TMPDIR/compound-engineering/ce-demo-reel/`). Do not upload to catbox.
|
||||
- R4. Create the destination directory if it does not exist.
|
||||
- R5. Use a descriptive filename that includes the branch name or PR identifier and a timestamp to avoid collisions across runs.
|
||||
- R6. After saving, display the local file path(s) to the user for easy reference.
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- A user running `ce-demo-reel` can keep captured evidence on disk without manual intervention.
|
||||
- The saved artifacts are discoverable in a predictable, stable OS-temp location.
|
||||
|
||||
---
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Catbox upload logic itself is unchanged — only the routing (local vs. upload) is new.
|
||||
- No automatic git-add or commit of saved artifacts.
|
||||
- No configurable save path — `$TMPDIR/compound-engineering/ce-demo-reel/` is the fixed default for now.
|
||||
- No retroactive save of previously captured evidence.
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Local save as an alternative to upload, not an addition**: The user chooses one destination per capture — either catbox or local. This keeps the flow simple and avoids redundant artifacts.
|
||||
- **OS-temp as the local target**: Uses `$TMPDIR/compound-engineering/ce-demo-reel/` per the repo's cross-invocation scratch-space convention. Stable prefix makes files findable without polluting the repo tree.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce-plan` for structured implementation planning, or proceed directly to implementation given the small scope.
|
||||
@@ -0,0 +1,149 @@
|
||||
---
|
||||
date: 2026-04-24
|
||||
topic: surface-scope-earlier
|
||||
---
|
||||
|
||||
# Surface Scope Earlier in ce-brainstorm and ce-plan
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Issue #676 (jrdncstr) reports that CE works well for greenfield/low-stakes work but becomes a burden in brownfield codebases: brainstorms and plans reach 300+ lines, artifacts are excessively defensive, rewrites persist, and PRs stay at 1000+ lines regardless of steering. He suggested a `--pragmatic` flag.
|
||||
|
||||
The surface suggestion (a mode or flag) is the wrong fix. **Scope under-visibility is the upstream cause; artifact density and PR diff size are downstream symptoms.** Both ce-brainstorm and ce-plan synthesize user input + agent inference into an interpretation, but the user doesn't see that synthesis until the doc lands. The user agrees to many individual things in dialogue but never sees the whole; the agent makes substantial inferences (especially in ce-plan solo invocation, where Phase 0.4 bootstrap is brief by design) and then writes against an unverified scope. Surprise at write-time means rework, and the rework looks like artifact bloat downstream.
|
||||
|
||||
**Working hypothesis:** fix the cause — surface the synthesis to the user before doc-write — and the symptoms abate. If they don't, density-control tools (calibrated exemplars, brevity passes for defensive sections) become a follow-up. Shipping them now alongside the cause fix would entangle attribution (which mechanism worked?), add maintenance surface for value that may not be needed, and chase symptoms before testing whether the cause fix dissolves them.
|
||||
|
||||
The fix lives in templates and phase additions — no new mode, no flag, no user-facing classification question. Scope tiers stay as-is.
|
||||
|
||||
Related: [GitHub Issue #676](https://github.com/EveryInc/compound-engineering-plugin/issues/676)
|
||||
|
||||
---
|
||||
|
||||
## Actors
|
||||
|
||||
- A1. **ce-brainstorm agent**: generates requirements documents. Currently runs extensive pre-write dialogue but never surfaces a whole-scope synthesis before doc-write.
|
||||
- A2. **ce-plan agent**: generates implementation plans. Currently runs minimal interview in solo invocation (Phase 0.4 "keep it brief") and never surfaces synthesized scope before research or plan-write.
|
||||
- A3. **End-user developer**: pays the cognitive-debt cost when artifacts over-invest, the rework cost when scope was misinterpreted, and the review cost when PRs over-reach.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
### R1. ce-brainstorm synthesis summary
|
||||
|
||||
Before Phase 3 (write requirements doc), ce-brainstorm surfaces a synthesis summary to the user. Fires for **all tiers** including Lightweight — the value is partly synthesis confirmation and partly a transition checkpoint ("about to write a doc") that gives the user permission to proceed or redirect.
|
||||
|
||||
Structure:
|
||||
- **Stated** — what the user said directly (in prompt, prior conversation, dialogue answers, approach selection)
|
||||
- **Inferred** — what the agent assumed to fill gaps (scope boundaries the user never explicitly named, success criteria extrapolated from intent)
|
||||
- **Out of scope** — deliberately excluded items (adjacent work, refactors, nice-to-haves)
|
||||
|
||||
Length: Lightweight gets one paragraph plus brief lists; Standard/Deep get a few paragraphs with explicit lists. Open prose prompt invites feedback: *"Does this match your intent? Tell me what to add, remove, redirect, or that I got wrong — or just confirm to proceed."*
|
||||
|
||||
User can rebut even when the synthesis accurately reflects their stated answers (they may change their mind, surface new context, correct unstated assumptions). Soft-cut fires on **circularity** (same item revised twice), not iteration count — new-item revisions across rounds proceed without limit.
|
||||
|
||||
Always embedded as the first section of the requirements doc. **Headless mode** (pipeline / `disable-model-invocation` context): skip the prompt and embed the synthesis with the **Inferred list omitted** — pipelines consume without human review, so propagating un-validated agent inferences as authoritative content is unsafe.
|
||||
|
||||
### R2. ce-plan synthesis summary, invocation-context-aware
|
||||
|
||||
Same Stated/Inferred/Out structure, prose, soft-cut, always-embed, and headless behavior as R1. Two timing variants:
|
||||
|
||||
- **Solo invocation** (no upstream brainstorm doc): fires **after Phase 0.4 bootstrap, before Phase 1 research begins**. Catches scope misinterpretation before sub-agent dispatch is spent. Synthesis covers full breadth: problem frame, intended behavior, success criteria, in/out scope. The "Inferred" list is especially load-bearing here — Phase 0.4 makes substantial inferences from a brief interview.
|
||||
- **Brainstorm-sourced invocation**: fires **after Phase 1 research, before Phase 5.2 plan-write**. Brainstorm doc + R1 already validated WHAT. Synthesis focuses on plan-time decisions the brainstorm didn't make: which files/modules to touch (and not), which patterns extended vs. introduced new, test scope (which existing-but-untested code is in/out), and tangential refactor scope.
|
||||
|
||||
State-machine guards (explicit in SKILL.md, not implicit):
|
||||
- Skip on Phase 0.1 fast paths (resume existing plan, deepen-intent) — synthesis is pre-write, doesn't apply when doc exists
|
||||
- Skip when Phase 0.4 routes out (ce-debug, ce-work, universal-planning) — agent left planning workflow
|
||||
- Solo variant skips when Phase 0.2 found a brainstorm doc (defers to brainstorm-sourced variant)
|
||||
|
||||
Self-redirect support: if user surfaces "this is bigger than I thought, let me brainstorm first" or similar, agent stops, suggests the alternative skill, offers to load it in-session. No "do you want to brainstorm first?" question fires upfront — that would add friction in the common case.
|
||||
|
||||
Graceful fallback: if origin brainstorm doc lacks the R1 synthesis section (older brainstorms, hand-written ones), R2 brainstorm-sourced runs as normal — its content is independent of origin synthesis presence.
|
||||
|
||||
### R3. Anti-expansion clause in ce-plan
|
||||
|
||||
Both tangential refactors and scope expansions go to a deferred-items list, not the active diff. Cleanup spotted in touched files → deferred. "While we're here, we could also..." → deferred. Adjacent improvements → deferred. Reinforces R2 by setting the default the synthesis surfaces.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Examples
|
||||
|
||||
- AE1. **Covers R1.** Given a brainstorm task, ce-brainstorm surfaces a synthesis (Stated / Inferred / Out) before doc-write. The user can confirm, add, remove, redirect, or change their mind — even when the synthesis accurately reflects what they said in dialogue. The confirmed synthesis is embedded as the first section of the requirements doc. In headless mode, the synthesis embeds without the Inferred list and without prompting.
|
||||
- AE2. **Covers R2 (solo).** Given a /ce-plan invocation with no upstream brainstorm doc, after Phase 0.4 bootstrap and before Phase 1 research begins, the agent surfaces a full-breadth synthesis with explicit "Inferred" list. The user can correct ("actually I want the whole password reset feature, not just the link"), and research runs against the corrected scope.
|
||||
- AE3. **Covers R2 (brainstorm-sourced).** Given a /ce-plan invocation with a matching brainstorm doc, after Phase 1 research and before plan-write, the agent surfaces a plan-time-focused synthesis (which files will/won't be touched, which patterns extended, test scope, refactor scope). Brainstorm-validated WHAT is assumed and not re-stated.
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
**Directly validated outcomes** (this iteration tests these):
|
||||
- ce-brainstorm and ce-plan both surface scope synthesis before doc-write. Users have a clear opportunity to correct inferences, redirect, or confirm.
|
||||
- Solo ce-plan invocations specifically catch scope errors before research is spent.
|
||||
- Headless mode embeds synthesis (without Inferred) so a human PR reviewer can see what scope was auto-interpreted.
|
||||
- Greenfield protection: in-repo validation on this plugin's own current work shows no regression.
|
||||
|
||||
**Expected downstream effects** (consequences of upstream cause-fix; not directly enforced or validated):
|
||||
- PR diff size resolves toward what the confirmed scope actually requires.
|
||||
- Rewrite frequency decreases because tangential refactors land in deferred items (R3) rather than the active diff.
|
||||
- Token spend on misdirected research decreases because solo ce-plan invocations catch scope errors before sub-agent dispatch.
|
||||
- Artifact density (defensive Outstanding Questions, placeholder template-tail sections) becomes proportional to confirmed-scope size — speculative, but a sufficient post-rollout signal to determine whether density-control tools (deferred — see Scope Boundaries) need to ship later.
|
||||
|
||||
If these downstream effects do not materialize after Phase A ships, the diagnosis was wrong — that's a real signal, not a partial win. Treat post-rollout PR-size telemetry on jrdncstr's repo (or a comparable case) as the actual validation of the causal claim.
|
||||
|
||||
---
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- Not adding a new mode, flag, command, or user-facing classification question
|
||||
- Not changing existing Lightweight/Standard/Deep tier classification
|
||||
- Not adding diff-size budgets or PR-size gates (Goodhart concerns)
|
||||
- Not modifying ce-work or its handoff
|
||||
- Not duplicating ce-brainstorm dialogue inside ce-plan's solo synthesis (R2 solo is a synthesis checkpoint, not a brainstorm-style interview)
|
||||
- Not touching auto-deepening (Phase 5.3) — preserved as load-bearing depth
|
||||
- Not introducing automated validation for headless-mode embedded synthesis (human PR reviewer is the safety net; documented limitation)
|
||||
- Not extending ce-doc-review to validate synthesis sections
|
||||
|
||||
**Depth-calibration mechanisms deferred to follow-up:** an earlier draft of this brainstorm proposed calibrated tier exemplars, targeted brevity passes for defensive sections in ce-brainstorm, and brevity passes for plan template-tail sections. These are density-control tools — they target *output density* directly. Under the working hypothesis that scope under-visibility is the upstream cause, density should follow naturally from disciplined scope; shipping density-control tools alongside the cause fix would entangle attribution, add maintenance surface, and chase symptoms before testing whether the cause fix dissolves them. **Revisit only if post-rollout signals show density problems persist after this iteration ships.**
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Working hypothesis: scope under-visibility is the upstream cause; density is downstream.** Post-rollout signals are the actual validation. If real-user feedback surfaces density problems persisting despite synthesis discipline, density-control tools become a follow-up.
|
||||
- **Two distinct synthesis-summary mechanisms (R1, R2), not one shared one.** ce-brainstorm has substantial pre-write dialogue; its summary is shorter and serves as synthesis confirmation + transition checkpoint. ce-plan has minimal pre-write interview in solo mode; its summary fires earlier (pre-research) and is more elaborate. Same Stated/Inferred/Out structure, different timing and shape per skill.
|
||||
- **No "do you want to brainstorm first?" fork in ce-plan.** Explicit forks add friction to the common case. The synthesis lets users self-redirect when they recognize they need brainstorming.
|
||||
- **Solo ce-plan synthesis fires pre-research, not pre-write.** Pre-research catches scope errors when correction is cheap (no sub-agent dispatch spent).
|
||||
- **Brainstorm-sourced ce-plan synthesis fires pre-write, not pre-research.** Brainstorm validates WHAT; plan-time decisions emerge during research, so pre-write catches them.
|
||||
- **Stated/Inferred/Out is the load-bearing structure.** Neutral about input richness (works for one-line prompts and rich prior conversation alike); forces honesty about how much was assumed vs. agreed.
|
||||
- **Open prose, not AskUserQuestion.** Cite Interaction Rule 5(a) inline in SKILL.md to prevent future "fix" back to a menu — option sets would leak the agent's framing of valid corrections.
|
||||
- **Headless mode omits the "Inferred" list.** Pipelines consume without human review; propagating un-validated inferences as authoritative is unsafe.
|
||||
- **Soft-cut fires on circularity, not iteration count.** Revising different aspects of a wrong synthesis is exactly what the mechanism should support.
|
||||
- **Always embed synthesis as first section of doc.** Self-describing artifact for human PR reviewers; no auto-validation in headless (accepted limitation).
|
||||
- **Phased delivery: Phase A (ce-brainstorm) before Phase B (ce-plan).** Validates the simpler synthesis mechanism in the smaller surface first.
|
||||
- **Rejected: diff budgets** (Goodhart failure mode).
|
||||
- **Deferred: depth-calibration mechanisms** (calibrated exemplars + brevity passes). Revisit only if post-rollout signals show density problems persist.
|
||||
|
||||
---
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- Assumes ce-brainstorm Phase 2→3 boundary and ce-plan Phase 0.6→1.1 boundary and pre-Phase-5.2 boundary can accommodate new synthesis-summary phases without restructuring. Needs codebase verification during planning.
|
||||
- Assumes skill-isolation rules continue to forbid cross-skill references. Synthesis-summary template content will be duplicated between ce-brainstorm and ce-plan reference directories.
|
||||
- Assumes users will engage with the synthesis summary rather than skip past it. If users routinely confirm without reading, the mechanism degrades to invisible scope drift. Worth structuring the prompt to invite scanning ("look at the Inferred list — did I assume anything wrong?").
|
||||
- ce-plan Phase 0.3 (origin-doc carry-forward) must handle a brainstorm doc whose first section is the new synthesis. Verify pre-Phase-A; if incompatible, the relevant fix lands in Phase A alongside R1.
|
||||
|
||||
---
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- [Affects R1, R2][Technical] Exact wording of the synthesis-summary prompt template. Per learning #9 (`pass-paths-not-content-to-subagents.md`), phrasing matters more than meta-rules. Author during implementation; iterate if early manual validation shows drift.
|
||||
- [Affects R1, R2][Technical] Whether `synthesis-summary.md` content lives as one file per skill (with both solo and brainstorm-sourced variants in the ce-plan version) or split. Default: one file per skill, two clearly-labeled sections in ce-plan's version.
|
||||
- [Affects R2][Technical] Whether the solo-mode prompt uses a blocking question tool or chat-output-with-natural-interrupt. Tradeoff: blocking is more reliable but adds friction; natural interrupt is lower friction but easier to skip past. Decide during planning.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
-> `/ce-plan` for implementation planning.
|
||||
@@ -0,0 +1,277 @@
|
||||
---
|
||||
date: 2026-05-19
|
||||
topic: vscode-copilot-agent-tool-access
|
||||
---
|
||||
|
||||
# VS Code Copilot Agent Tool Access for CE Plugin
|
||||
|
||||
## Problem Frame
|
||||
|
||||
When the Compound Engineering plugin is installed in VS Code via "Chat: Install Plugin from Source", CE subagents (reviewers, researchers, etc.) cannot read workspace files. Invoking `ce-correctness-reviewer` produces:
|
||||
|
||||
```
|
||||
ACCESS_FAILED No filesystem read tool is available in this session to read README.md
|
||||
```
|
||||
|
||||
Meanwhile, built-in subagents like `Explore` succeed in the same session, proving the VS Code Copilot host does provide workspace access to subagents — but only when tools are properly declared in the agent's frontmatter.
|
||||
|
||||
The root cause is a gap in the converter pipeline:
|
||||
|
||||
1. Claude agent `.agent.md` files declare tools (`tools: Read, Grep, Glob, Bash`), but the parser never captures them.
|
||||
2. The Copilot converter intentionally drops tools, emitting agents without a `tools` field.
|
||||
3. VS Code Copilot interprets a missing `tools` field as "no tools granted" for custom plugin agents (contrary to the converter's original assumption that omitting means defaults).
|
||||
|
||||
This renders all CE subagents inert under Copilot — they can reason but cannot inspect code.
|
||||
|
||||
---
|
||||
|
||||
## Actors
|
||||
|
||||
- A1. **Developer using CE in VS Code Copilot**: Invokes CE skills and agents expecting them to read/search/execute against the workspace.
|
||||
- A2. **CE converter pipeline**: Parses Claude plugin source, converts agents/skills to Copilot-compatible format, and writes output files.
|
||||
- A3. **VS Code Copilot host**: Loads plugin agent definitions, grants tools based on frontmatter declarations, and dispatches subagents.
|
||||
|
||||
---
|
||||
|
||||
## Key Flows
|
||||
|
||||
- F1. **Subagent tool access (broken path)**
|
||||
- **Trigger:** User invokes a CE skill that dispatches a reviewer/researcher subagent.
|
||||
- **Actors:** A1, A3
|
||||
- **Steps:**
|
||||
1. User invokes `/compound-engineering:ce-code-review`
|
||||
2. Skill dispatches `ce-correctness-reviewer` as subagent
|
||||
3. VS Code Copilot loads the agent definition, finds no `tools` field
|
||||
4. Subagent receives no filesystem tools
|
||||
5. Subagent fails to read any files
|
||||
- **Outcome:** Review fails with tool-access error.
|
||||
- **Covered by:** R1, R2, R3
|
||||
|
||||
- F2. **Subagent tool access (fixed path)**
|
||||
- **Trigger:** Same as F1, after fix is applied.
|
||||
- **Actors:** A1, A2, A3
|
||||
- **Steps:**
|
||||
1. Parser captures `tools` from Claude agent frontmatter
|
||||
2. Converter maps Claude tools to Copilot aliases (`read`, `search`, `execute`, etc.)
|
||||
3. Emitted `.agent.md` includes `tools: [read, search, execute]`
|
||||
4. VS Code Copilot grants declared tools to subagent
|
||||
5. Subagent reads workspace files successfully
|
||||
- **Outcome:** CE reviewers and researchers operate with full workspace access.
|
||||
- **Covered by:** R1, R2, R3, R4
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
**Parser: Capture agent tools**
|
||||
|
||||
- R1. The Claude parser (`src/parsers/claude.ts` `loadAgents`) must parse the `tools` field from agent frontmatter and populate it on the `ClaudeAgent` type.
|
||||
- R2. The `ClaudeAgent` type (`src/types/claude.ts`) must include an optional `tools?: string[]` field.
|
||||
|
||||
**Converter: Map tools to Copilot aliases**
|
||||
|
||||
- R3. The Copilot converter (`src/converters/claude-to-copilot.ts`) must map Claude tool names to VS Code Copilot tool aliases and emit a `tools` array in agent frontmatter. Mapping:
|
||||
- `Read` → `read`
|
||||
- `Grep`, `Glob` → `search`
|
||||
- `Glob` → `search` (deduplicated with Grep)
|
||||
- `Bash` → `execute`
|
||||
- `Write`, `Edit`, `Patch`, `MultiEdit` → `edit`
|
||||
- `WebFetch`, `WebSearch` → `web`
|
||||
- `TodoRead`, `TodoWrite` → `todo`
|
||||
- `Task` → `agent`
|
||||
- MCP tool references (e.g., `mcp__context7__*`) → omitted (not mappable to Copilot built-in aliases)
|
||||
- R4. Output deduplication: the emitted `tools` array must contain unique values only (e.g., `Grep` + `Glob` both map to `search`, emit `search` once).
|
||||
- R5. If no tools are declared on the source agent, the converter must omit the `tools` field (preserving current behavior for agents that genuinely have no tool declarations).
|
||||
|
||||
**Copilot type: Support tools field**
|
||||
|
||||
- R6. The `CopilotAgent` type should support tools metadata so the converter's output is type-safe. This may be achieved by adding a field to the type or by ensuring the frontmatter serialization path handles it.
|
||||
|
||||
**Tests**
|
||||
|
||||
- R7. Update `tests/copilot-converter.test.ts` to assert that agents with declared tools produce correct Copilot `tools` arrays.
|
||||
- R8. Add test cases for: deduplication, unknown/unmappable tools (omitted gracefully), agents with no tools (field omitted), agents with web/MCP tools.
|
||||
- R9. Add or update parser tests to verify `tools` is captured from agent frontmatter.
|
||||
|
||||
**No install target required for plugin-from-source**
|
||||
|
||||
- R10. The fix must work when VS Code loads the plugin directly from the repo via "Chat: Install Plugin from Source" — meaning the plugin-native `.agent.md` files must carry the correct Copilot `tools` frontmatter, OR the conversion happens at install time. Determine which path applies (see Outstanding Questions).
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Examples
|
||||
|
||||
- AE1. **Covers R1, R2, R3, R4.** Given a CE agent file with `tools: Read, Grep, Glob, Bash`, when the plugin is parsed and converted to Copilot format, the output `.agent.md` frontmatter includes `tools: [read, search, execute]` (search appears once despite two source entries).
|
||||
|
||||
- AE2. **Covers R3, R5.** Given a CE agent file with no `tools` field, when converted to Copilot format, the output `.agent.md` frontmatter does NOT include a `tools` key.
|
||||
|
||||
- AE3. **Covers R3.** Given a CE agent with `tools: Read, Grep, Glob, Bash, WebFetch, WebSearch, mcp__context7__*`, when converted, the output is `tools: [read, search, execute, web]` (MCP reference omitted, web deduplicated).
|
||||
|
||||
- AE4. **Covers R1, R3, R7.** Given the `ce-correctness-reviewer` agent is installed in VS Code Copilot, when it is dispatched as a subagent, it can successfully read `README.md` from the workspace.
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- CE reviewer and researcher subagents can read, search, and execute in the workspace when invoked through VS Code Copilot.
|
||||
- The smoke test (invoke `ce-correctness-reviewer`, ask it to read `README.md`) returns file content instead of `ACCESS_FAILED`.
|
||||
- No regression: agents without declared tools continue to work as before (tools field omitted).
|
||||
- Existing non-Copilot targets (OpenCode, Codex, Pi, Gemini, Kiro) are unaffected.
|
||||
|
||||
---
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
- **Not in scope: Changing VS Code Copilot host behavior.** We work within the host's documented tool-declaration mechanism.
|
||||
- **Not in scope: Changing `/compound-engineering:ce-*` namespacing.** This is VS Code Copilot host behavior for installed plugins. Document it but do not attempt to override.
|
||||
- **Not in scope: `.compound-engineering/config.local.yaml` as a tool-access fix.** That config controls CE preferences (Codex delegation, etc.), not Copilot tool grants.
|
||||
- **Not in scope: Adding a full `copilot` install target to `src/targets/index.ts`.** The immediate fix is making the converter emit tools. A dedicated install target may be added later.
|
||||
- **Not in scope: Changing how the plugin is distributed/installed.** The fix must work with the existing "Install Plugin from Source" workflow.
|
||||
- **Deferred: Copilot skill `tools` field.** Skills (SKILL.md) may also benefit from tool declarations, but the immediate failure is in subagents. Skill tool access can be addressed separately if needed.
|
||||
- **Deferred: Registering a `copilot` target in `src/targets/index.ts`.** This would enable `bun convert --to copilot` as a first-class workflow but is not required for the plugin-from-source fix.
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **Map tools explicitly rather than emitting all tools unconditionally.** An explicit mapping ensures CE agents get precisely the capabilities they declare, matching the principle of least privilege. Emitting `tools: [read, search, execute, edit, web, todo, agent]` on every agent would work but grants unnecessary capabilities.
|
||||
- **Omit unmappable tools (MCP references) rather than erroring.** MCP tools are platform-specific and have no Copilot built-in equivalent. Silently dropping them with a warning is the safe default.
|
||||
- **Parse tools as a flat string array.** Claude agent frontmatter declares tools as a comma-separated line (`tools: Read, Grep, Glob, Bash`). Parse by splitting on commas and trimming whitespace.
|
||||
|
||||
---
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- **VS Code honors `tools` in plugin agent files.** Confirmed: the docs explicitly state custom agents use the `tools` frontmatter field to declare available tools.
|
||||
- **"Install Plugin from Source" reads raw agent files.** Confirmed: VS Code clones the repo and loads files directly. No conversion step occurs. The fix must modify source files or the plugin format.
|
||||
- **VS Code Claude format detection uses file extension.** The docs state Claude agents are "plain `.md` files" in `.claude/agents`. The CE plugin uses `.agent.md` — this likely causes format mis-detection. Needs empirical verification.
|
||||
- **Tool set names are stable.** The `read`, `search`, `execute`, `edit`, `web`, `agent`, `todos`, `vscode`, `browser` tool sets are documented as of May 2026.
|
||||
- **Claude Code may or may not accept Copilot-native tool format.** If we change tools to `tools: [read, search, execute]`, Claude Code behavior needs testing. This is the key cross-platform compatibility question.
|
||||
|
||||
---
|
||||
|
||||
## Research Findings (2026-05-20)
|
||||
|
||||
### Q1: How does "Install Plugin from Source" load agents?
|
||||
|
||||
**Answer: VS Code reads raw agent files directly from the cloned repo. There is no conversion step.**
|
||||
|
||||
Evidence from [VS Code Agent Plugins docs](https://code.visualstudio.com/docs/copilot/customization/agent-plugins):
|
||||
- "Run Chat: Install Plugin From Source from the Command Palette. Enter a Git repository URL and VS Code clones and installs the plugin."
|
||||
- Cached at: `%APPDATA%\Code\agentPlugins\github.com\{org}\{repo}` (Windows)
|
||||
- VS Code auto-detects plugin format by checking: `.plugin/plugin.json` → `plugin.json` (root) → `.github/plugin/plugin.json` → `.claude-plugin/plugin.json`
|
||||
- The CE plugin has `.claude-plugin/plugin.json`, so VS Code identifies it as **Claude format**
|
||||
|
||||
**Critical implication:** Fixing only the converter is INSUFFICIENT. The raw plugin files must carry tool declarations that VS Code can interpret correctly.
|
||||
|
||||
### Q2: Does VS Code map Claude tool names automatically?
|
||||
|
||||
**Answer: YES — documented, but likely broken for this specific case.**
|
||||
|
||||
From the [Custom Agents docs](https://code.visualstudio.com/docs/copilot/customization/custom-agents), Claude agent format section:
|
||||
> "VS Code maps Claude-specific tool names to the corresponding VS Code tools. Both the VS Code `.agent.md` format (with YAML arrays for tools) and the Claude format (with comma-separated strings) are supported."
|
||||
|
||||
However, the same docs state:
|
||||
> "Agent files in the `.claude/agents` folder use **plain `.md` files**"
|
||||
|
||||
The CE plugin agents use `.agent.md` extension (`ce-correctness-reviewer.agent.md`), NOT plain `.md`. VS Code's Claude format detection for agent files appears to depend on the file extension:
|
||||
- `.md` in `.claude/agents/` → Claude format (comma-separated tools string, auto-mapped)
|
||||
- `.agent.md` → Copilot format (YAML array of VS Code tool names)
|
||||
|
||||
**Likely root cause:** The CE agent files have Copilot file extension (`.agent.md`) but Claude-style frontmatter (`tools: Read, Grep, Glob, Bash`). VS Code parses them as Copilot-format agents and looks for VS Code tool names like `Read`, `Grep` — which don't exist. Unrecognized tools are silently ignored, leaving the agent with **zero tools**.
|
||||
|
||||
### Q3: Canonical VS Code tool set names
|
||||
|
||||
From the [VS Code cheat sheet](https://code.visualstudio.com/docs/copilot/reference/copilot-vscode-features), built-in tool sets:
|
||||
|
||||
| Tool Set | Individual Tools |
|
||||
|----------|-----------------|
|
||||
| `agent` | `agent/runSubagent` |
|
||||
| `browser` | (experimental, multiple) |
|
||||
| `edit` | `edit/createDirectory`, `edit/createFile`, `edit/editFiles`, `edit/editNotebook` |
|
||||
| `execute` | `execute/runInTerminal`, `execute/getTerminalOutput`, `execute/createAndRunTask`, `execute/runNotebookCell`, `execute/testFailure` |
|
||||
| `read` | `read/readFile`, `read/problems`, `read/getNotebookSummary`, `read/readNotebookCellOutput`, `read/terminalLastCommand`, `read/terminalSelection` |
|
||||
| `search` | `search/changes`, `search/codebase`, `search/fileSearch`, `search/listDirectory`, `search/textSearch`, `search/usages` |
|
||||
| `todos` | (todo list tool) |
|
||||
| `vscode` | `vscode/askQuestions`, `vscode/extensions`, `vscode/runCommand`, `vscode/VSCodeAPI` |
|
||||
| `web` | `web/fetch` |
|
||||
|
||||
Custom agent `tools` field accepts: tool set names (e.g. `read`), individual tool names (e.g. `read/readFile`), MCP tool names, or `*` for all.
|
||||
|
||||
### Q4: Does `tools: []` differ from omitting `tools`?
|
||||
|
||||
**Answer: Not explicitly documented.** Based on the error behavior ("No filesystem read tool is available"), an agent with unrecognized tools behaves the same as one with no tools — it gets nothing. The distinction between explicit empty array and omission is academic for this fix since the real issue is the format mismatch.
|
||||
|
||||
### Q5: Subagent tool inheritance
|
||||
|
||||
From the docs, subagents:
|
||||
- Run as isolated instances with their own agent definition
|
||||
- The **parent** agent needs `agent` in its tools list and the subagent in its `agents` field
|
||||
- The **subagent** uses its own `tools` declaration
|
||||
- Built-in `Explore` succeeds because it's a built-in agent with proper tool access
|
||||
|
||||
This confirms the issue is in how the subagent's own tools are parsed, not in inheritance.
|
||||
|
||||
---
|
||||
|
||||
## Revised Problem Analysis
|
||||
|
||||
The root cause is a **format mismatch**, not a missing converter feature:
|
||||
|
||||
1. CE agent files use `.agent.md` extension (Copilot format indicator)
|
||||
2. CE agent files contain Claude-style frontmatter: `tools: Read, Grep, Glob, Bash` (comma-separated string)
|
||||
3. VS Code sees `.agent.md` → applies Copilot-format parsing → looks for VS Code tool names
|
||||
4. `Read`, `Grep`, `Glob`, `Bash` are not valid VS Code tool names → silently dropped
|
||||
5. Agent ends up with zero tools → "No filesystem read tool is available"
|
||||
|
||||
**The built-in `Explore` agent works because it's VS Code's own agent with proper Copilot-native tool declarations.**
|
||||
|
||||
---
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
### Resolve Before Planning
|
||||
|
||||
- **[Affects fix strategy][Needs testing]** Does renaming CE agents to plain `.md` (and keeping `tools: Read, Grep, Glob, Bash`) trigger VS Code's Claude-to-Copilot tool mapping? If yes, the fix is just a file extension rename. If no, we must also change the tool declarations to Copilot-native format.
|
||||
- **[Affects fix strategy][Needs testing]** If we keep `.agent.md` extension but change `tools` to Copilot-native format (`tools: [read, search, execute]`), does Claude Code still function correctly? Claude's docs say `tools` is a comma-separated string — does Claude also accept YAML arrays?
|
||||
|
||||
### Deferred to Planning
|
||||
|
||||
- **[Affects R6][Technical]** Should `CopilotAgent` type carry a `tools?: string[]` field, or is it sufficient for the converter to inject tools into the frontmatter string without type-level modeling?
|
||||
- **[Affects scope][Technical]** Should the parser change also benefit other converter targets (Codex, Gemini, etc.), or is tool mapping currently handled differently for those targets?
|
||||
- **[Affects upstream][Decision]** Should this be reported as a VS Code bug (Claude format mapping not applied to `.agent.md` files in Claude-format plugins)?
|
||||
|
||||
---
|
||||
|
||||
## Validation Plan
|
||||
|
||||
After implementation, verify the fix end-to-end:
|
||||
|
||||
1. **Build/convert the plugin** (if a build step is required).
|
||||
2. **Install in VS Code** via "Chat: Install Plugin from Source" pointing at the fork repo.
|
||||
3. **Confirm plugin loaded:** Check VS Code's extension/plugin list shows compound-engineering from the fork.
|
||||
4. **Smoke test — subagent file read:**
|
||||
- Invoke `/compound-engineering:ce-correctness-reviewer` (or dispatch it from a skill)
|
||||
- Ask it to read `README.md` and report the first heading
|
||||
- Expected: returns content (e.g., `# fantastic-chainsaw` or whatever the repo's H1 is)
|
||||
- Failure: `ACCESS_FAILED No filesystem read tool`
|
||||
5. **Comparative test — built-in agent:**
|
||||
- Invoke built-in `Explore` with the same request
|
||||
- Expected: succeeds (baseline proof the host provides tools)
|
||||
6. **Full flow test — code review:**
|
||||
- Invoke `/compound-engineering:ce-code-review` on a small diff
|
||||
- Verify reviewer subagents produce findings referencing actual file content
|
||||
7. **Regression — no-tools agent:**
|
||||
- If any CE agent legitimately has no `tools` field, verify it still loads without error
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
Two quick empirical tests will determine the fix strategy:
|
||||
|
||||
1. **Test A (file extension):** Rename one CE agent to `.md` (e.g., `ce-correctness-reviewer.md`), keep Claude-style `tools: Read, Grep, Glob, Bash`. Install plugin, invoke as subagent. If it works → fix is renaming all agent files.
|
||||
|
||||
2. **Test B (tool format):** Keep `.agent.md` extension, change `tools` to `tools: [read, search, execute, edit]` (Copilot-native YAML array). Install plugin, invoke as subagent. If it works → fix is converting tool declarations to Copilot format.
|
||||
|
||||
After one test succeeds → `/ce-plan` for full implementation across all 49 agent files, converter updates, and test changes.
|
||||
@@ -0,0 +1,177 @@
|
||||
---
|
||||
title: "ce-plan approach altitude — plan-for-a-plan as a first-class shape"
|
||||
date: 2026-06-04
|
||||
topic: ce-plan-approach-altitude
|
||||
---
|
||||
|
||||
# ce-plan approach altitude — plan-for-a-plan as a first-class shape
|
||||
|
||||
## Summary
|
||||
|
||||
Give `ce-plan` a deliberate "approach altitude": when a problem is hard, answer it one level up first — produce a grounded *plan for how the deliverable will be made* — before committing to the deliverable itself. Entered explicitly ("plan for a plan") or, rarely, offered proactively. The approach-plan lands in chat (file-optional, deepenable); at a checkpoint the user runs it now or later. Execution of a non-code deliverable routes to a lightweight `ce-work` carve-out (or any agent, since the plan stays portable); code execution stays `ce-work`'s normal path. `ce-plan` never executes — it stays a planning/knowledge-structuring skill.
|
||||
|
||||
---
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Users have started asking `ce-plan` for an *intermediate* plan — a plan for how the agent will approach a hard problem — and then trying to execute it. The canonical case (the "Margolis" request):
|
||||
|
||||
> "Make a plan for the plan. I'm about to hand you two things: a book as a PDF, and the two-hour transcript of the meeting I just had with the author. I want a thoughtful plan for how my business problem, that conversation, and the lessons in the book come together into something I can actually use. Do not write that document now. Writing it is the work. Right now I only want the plan for how you'll read the book, mine the transcript, and produce a great document."
|
||||
|
||||
This is a way to get **certainty and structure on something hard** rather than zero-shotting a fragile final deliverable. It fails today, and tracing the exact prompt shows why:
|
||||
|
||||
- `ce-plan`'s non-software path forces a binary — **plan-seeking** (save a plan) or **answer-seeking** (deliver an answer, discard the scaffold). The request is neither: it wants an *approach plan now*, then the *real deliverable later*, as a deliberate two-step. The classifier is as likely to start synthesizing the document, or to answer-seek toward it, as to hold at the approach. The user had to spend three sentences forcing the hold ("Do not write that document now…") — that fight is the symptom of a missing shape.
|
||||
- **The second phase is homeless.** Even with a perfect approach-plan, "now go do it" has nowhere to land — `ce-work` is code-only, and the non-software handoff offers no execution. The `ce-plan → ce-work` chain users invented breaks because `ce-work` can't do the right thing with that kind of plan, not because two skills is wrong.
|
||||
- **The approach-plan wouldn't be grounded.** A good plan for mining a *specific* transcript against a *specific* book requires looking at them. Today's research is repo/web-flavored, with no "ingest the user's heavy inputs to shape the approach" step — so the output is generic methodology, not a plan worth approving.
|
||||
|
||||
The same pattern generalizes beyond knowledge work: *"before you write the implementation plan, plan how you'll investigate the codebase."* The executor (human or agent) is irrelevant — a portable plan reads the same either way. What's missing is the altitude, the hold, and a home for execution.
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- **The real boundary is code vs. knowledge-work, not plan vs. execute.** `ce-plan` already executes knowledge work — the answer-seeking disposition reads sources, analyzes, and delivers a produced result. Producing a synthesis document is that same act with a bigger output. So planning, answering, and synthesizing a deliverable are all `ce-plan`'s knowledge-work remit; only **code** needs `ce-work`'s lifecycle. Drawing the line at code keeps the sacred boundary (no code, no execution-time discovery) fully intact while letting non-code deliverables flow.
|
||||
|
||||
- **General capability, high-precision / low-recall trigger.** The capability is domain-general, but the *proactive* offer fires rarely. Because the explicit path is always available as a safety net, the errors are asymmetric: a missed offer is cheap (the user just asks), a wrong offer is a nag. The named enemy is the new-hammer failure mode — every `ce-plan` turn opening with "want me to plan the approach first?" When borderline, stay silent.
|
||||
|
||||
- **Execution stays out of `ce-plan`; `ce-work` gains a non-code carve-out.** Rather than make the planning skill execute (which feels wrong) or force a document-production plan through `ce-work`'s code lifecycle (which would mangle it), `ce-work` gets a minimal non-code branch. `ce-work` stays "the execution skill" regardless of domain; `ce-plan` stays planning.
|
||||
|
||||
- **Light recon, two-stage grounding.** A cheap heuristic (request shape + input metadata) decides whether to *offer*; light recon (skim/sample, not deep-read) happens only after the user accepts. This makes the approach-plan specific enough to judge without paying the deliverable's cost up front.
|
||||
|
||||
- **Separate but coordinated, not a refactor of existing mechanics.** Three in-chat "approach" surfaces already exist — answer-seeking's plan-of-attack, the Phase 0.7 scoping synthesis, and the deepening pass. Approach-altitude is built as its own surface with firing rules drawn so it never overlaps them, rather than unifying them into one concept or destabilizing skills that already work. The cost moves to boundary-drawing: the rules for when approach-altitude fires vs. when an existing mechanic fires must be crisp enough that it never reads as a confusable fourth thing.
|
||||
|
||||
---
|
||||
|
||||
## Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
Req[Request to ce-plan] --> Explicit{Explicit<br/>'plan the approach'?}
|
||||
Explicit -->|yes| Recon[Light recon of inputs]
|
||||
Explicit -->|no| Gate{Method-uncertainty AND<br/>cost-of-getting-it-wrong<br/>both high?}
|
||||
Gate -->|no| Normal[Plan / do normally]
|
||||
Gate -->|yes| Offer[Offer approach-plan:<br/>one dismissible line,<br/>names the signal that fired]
|
||||
Offer -->|declined| Normal
|
||||
Offer -->|accepted| Recon
|
||||
Recon --> Approach[Approach-plan in chat<br/>file-optional, deepenable]
|
||||
Approach --> Checkpoint{Do it now / save for later}
|
||||
Checkpoint -->|later| Save[Save portable plan]
|
||||
Checkpoint -->|now, code| CW[ce-work normal path]
|
||||
Checkpoint -->|now, non-code| Carve[ce-work non-code carve-out<br/>or any agent]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Actors
|
||||
|
||||
- A1. **User** — issues the request, accepts/declines a proactive offer, decides at the checkpoint.
|
||||
- A2. **`ce-plan`** — recognizes or offers the approach altitude, does light recon, produces the approach-plan, routes execution. Never writes code or executes a non-code deliverable itself.
|
||||
- A3. **`ce-work` (and its non-code carve-out)** — executes the deliverable. Any agent can substitute, since the plan is portable.
|
||||
|
||||
---
|
||||
|
||||
## Key Flows
|
||||
|
||||
- F1. **Explicit approach-plan**
|
||||
- **Trigger:** User asks for the approach ("plan for a plan", "plan how you'll do it", "don't do it yet").
|
||||
- **Steps:** `ce-plan` does light recon of provided inputs → produces a grounded approach-plan in chat → checkpoint → routes execution per choice.
|
||||
- **Covered by:** R1, R5, R7, R8.
|
||||
|
||||
- F2. **Proactive offer**
|
||||
- **Trigger:** Plain request with no approach language, where method-uncertainty AND cost-of-getting-it-wrong are both high.
|
||||
- **Steps:** Cheap heuristic fires → `ce-plan` offers once, dismissibly, naming the signal → if accepted, continue as F1 from recon onward; if declined, plan/do normally.
|
||||
- **Covered by:** R2, R3, R6.
|
||||
|
||||
- F3. **Execution routing**
|
||||
- **Trigger:** User chooses "do it now" at the checkpoint.
|
||||
- **Steps:** Code deliverable → `ce-plan` produces the implementation plan and hands code to `ce-work`'s normal path. Non-code deliverable → routes to `ce-work`'s non-code carve-out (skip the code lifecycle) or to any agent given the portable plan.
|
||||
- **Covered by:** R9, R10, R11, R12.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
**Recognition and triggering**
|
||||
|
||||
- R1. `ce-plan` recognizes an explicit request for an approach-plan and always honors it, ungated by the proactive heuristic — it holds at the approach and does not start the deliverable.
|
||||
- R2. `ce-plan` proactively offers an approach-plan only when method-uncertainty AND cost-of-getting-it-wrong are both high; when either is low, it stays silent and plans/does normally.
|
||||
- R3. A proactive offer is a single lightweight, dismissible line that names the specific signal that fired (the "why it helps"); it is never a blocking ceremony.
|
||||
- R4. The capability is domain-general — available for software and knowledge-work requests alike, and indifferent to whether a human or the agent will execute.
|
||||
- R16. Approach-altitude is a distinct surface from the existing in-chat approach mechanics (answer-seeking's plan-of-attack, the scoping synthesis, the deepening pass); its firing rules are drawn so it never overlaps or duplicates them.
|
||||
|
||||
**Approach-plan production**
|
||||
|
||||
- R5. Before producing the approach-plan, the agent does light recon of provided inputs (skim/sample), grounding the approach in specifics; full ingestion is deferred to execution.
|
||||
- R6. The offer/no-offer decision is a cheap heuristic over request shape and input metadata; recon cost is paid only after the user accepts.
|
||||
- R7. The approach-plan is delivered chat-first and is file-optional; the user can choose to persist it and deepen it.
|
||||
|
||||
**Checkpoint and execution routing**
|
||||
|
||||
- R8. After the approach-plan, the user decides at a checkpoint: execute now, or save for later.
|
||||
- R9. Code execution stays on `ce-work`'s normal path; `ce-plan` never writes code.
|
||||
- R10. Non-code deliverable execution routes to `ce-work`'s non-code carve-out, or to any agent given the portable plan.
|
||||
- R11. `ce-plan` itself does not execute the deliverable; it produces the approach-plan and hands off.
|
||||
|
||||
**`ce-work` non-code carve-out**
|
||||
|
||||
- R12. `ce-work`'s input triage recognizes a non-code plan (no implementation units / files / test scenarios, or an explicit signal) and routes to a branch that skips the code lifecycle (no branch/worktree, no Test Discovery, no commit/PR/CI).
|
||||
- R13. The carve-out executes the production plan — read sources, synthesize, produce and save the deliverable, and report where it landed.
|
||||
- R14. The carve-out is a minority-case branch alongside the code path, not a co-equal mode, and must not disturb the code path.
|
||||
|
||||
**Portability**
|
||||
|
||||
- R15. The approach-plan / production-plan stays agent-agnostic — no `ce-work`-specific choreography baked in — so handing it to any agent to execute works without `ce-work`.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Examples
|
||||
|
||||
- AE1. **Covers R1.** Given "plan for a plan" or "don't write it yet — plan the approach", `ce-plan` produces an approach-plan and does not begin the deliverable, regardless of the proactive heuristic.
|
||||
- AE2. **Covers R2, R3.** Given a plain request whose method is clear — even a large one — `ce-plan` does not offer an approach-plan; it proceeds to plan/do normally.
|
||||
- AE3. **Covers R2, R3, R6.** Given a plain request with heavy disparate inputs and a vague outcome ("something I can actually use"), `ce-plan` offers once, dismissibly, naming the signal; if declined, it proceeds normally without re-asking.
|
||||
- AE4. **Covers R9, R10.** Given approval to execute a *software* approach-plan, `ce-plan` produces the implementation plan and hands code to `ce-work`. Given a *knowledge-work* approach-plan, execution routes to the `ce-work` carve-out (or any agent).
|
||||
- AE5. **Covers R12, R13.** Given the `ce-work` carve-out receives a non-code plan, it skips branch/test/commit/CI and instead reads the sources, synthesizes, and writes the deliverable.
|
||||
|
||||
---
|
||||
|
||||
## Scope Boundaries
|
||||
|
||||
**Deferred for later**
|
||||
|
||||
- A full non-software `ce-work` mode. The carve-out is intentionally minimal; building a co-equal knowledge-work execution engine is out of scope.
|
||||
- Git/save behavior of the produced deliverable (commit vs. plain write, save location) — settle during planning.
|
||||
- Renaming `ce-plan` to reflect that it can produce non-plan output. The naming oddness is accepted for now; the answer-seeking disposition already lives with it.
|
||||
|
||||
**Outside this capability's identity**
|
||||
|
||||
- `ce-plan` writing or running code. Code is always `ce-work`. The approach altitude never crosses into code execution.
|
||||
- Auto-executing the deliverable without the checkpoint. The hold is the point of the feature.
|
||||
|
||||
---
|
||||
|
||||
## Dependencies / Assumptions
|
||||
|
||||
- Builds on the existing answer-seeking disposition in `plugins/compound-engineering/skills/ce-plan/references/universal-planning.md` — the precedent that `ce-plan` can execute knowledge work and produce a result, not just a plan.
|
||||
- Light recon assumes provided inputs are available at approach-plan time. If inputs arrive later, recon degrades gracefully to propose-from-request (less grounded, flagged as such).
|
||||
- The approach-altitude decision must sit above `ce-plan`'s software/non-software split so the capability is domain-general rather than trapped in the universal (non-software) path.
|
||||
|
||||
---
|
||||
|
||||
## Outstanding Questions
|
||||
|
||||
**Deferred to planning**
|
||||
|
||||
- The crisp firing boundaries that keep approach-altitude from overlapping the three existing in-chat approach mechanics — the rules for when it fires vs. when answer-seeking's plan-of-attack, the scoping synthesis, or the deepening pass fires. The reconciliation decision is made (separate but coordinated); the boundary-drawing is a planning-time design task that needs grounded reading of how each existing mechanic actually triggers.
|
||||
- How `ce-plan` signals "non-code plan" to the `ce-work` carve-out: plan metadata, absence of implementation units, or an explicit flag.
|
||||
- The explicit-trigger phrase set, and how recognition stays robust without inflating the proactive offer's firing rate.
|
||||
- Exactly how light "light recon" is per input type (PDF, transcript, codebase), and how it is bounded so the checkpoint stays cheap.
|
||||
|
||||
---
|
||||
|
||||
## Sources / Research
|
||||
|
||||
- `plugins/compound-engineering/skills/ce-plan/SKILL.md` — the planning/execution boundary ("does not implement code… belongs in `ce-work`"), Core Principle 6 (keep the plan portable), and the Phase 0.7 solo-mode scoping synthesis (an existing approach-checkpoint in spirit).
|
||||
- `plugins/compound-engineering/skills/ce-plan/references/universal-planning.md` — the plan-seeking vs. answer-seeking dispositions; the answer-seeking flow that already executes knowledge work and delivers a produced result.
|
||||
- `plugins/compound-engineering/skills/ce-work/SKILL.md` — code-only Phase 0 triage (files to change, test files), Phase 1 branch/worktree setup, task lists built from implementation units; confirms `ce-work` would mishandle a non-code plan today.
|
||||
- `plugins/compound-engineering/skills/ce-work/references/shipping-workflow.md` — the commit → PR → CI lifecycle the carve-out must skip.
|
||||
- Motivating example: the "Margolis" request (book PDF + two-hour transcript → synthesis document), used as the canonical knowledge-work case throughout.
|
||||
@@ -0,0 +1,198 @@
|
||||
---
|
||||
title: "Agentless plugin surface reduction"
|
||||
date: 2026-06-19
|
||||
topic: agentless-plugin-surface-reduction
|
||||
---
|
||||
|
||||
# Agentless Plugin Surface Reduction
|
||||
|
||||
## Summary
|
||||
|
||||
Move Compound Engineering away from standalone plugin agents and toward skill-local subagent prompt assets. Public skills should represent user-facing jobs, not convenience wrappers around specialist agents. Surviving skills that need subagents will dispatch generic subagents using prompt definitions stored inside that skill's own directory, such as `references/agents/*.md` or `references/personas/*.md`.
|
||||
|
||||
The goal is maximum deletion: remove redundant public skills, delete `plugins/compound-engineering/agents/`, and make the Codex plugin fully self-contained through native skills rather than a hybrid native-skill plus Bun-installed custom-agent setup.
|
||||
|
||||
## Problem Frame
|
||||
|
||||
Compound Engineering currently defines specialist behavior in two forms:
|
||||
|
||||
- standalone agent files under `plugins/compound-engineering/agents/`
|
||||
- skill-local prompt/persona files under skill `references/` directories
|
||||
|
||||
The standalone-agent model creates portability and install friction. Codex native plugins load the CE skills, but standalone agent registration has required a second Bun install path that converts Claude Markdown agents to Codex TOML agents. That split is brittle, profile-sensitive, and prevents the Codex plugin from behaving as a normal self-contained plugin.
|
||||
|
||||
Other plugin ecosystems, including Superpowers, use a cleaner pattern: the skill owns the workflow and carries prompt templates for subagents in its own directory. That gives the skill custom subagent behavior without requiring a platform-level custom-agent registry.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Delete standalone CE agents.** Remove `plugins/compound-engineering/agents/` after moving still-needed behavior into surviving skills as local prompt assets.
|
||||
- **Delete redundant public skills.** Public skills must map to real user jobs. Skills that only make an agent easy to call should be removed.
|
||||
- **Prefer skill-local duplication over shared agent infrastructure.** Duplicating prompt text across a few skills is acceptable when it removes cross-platform agent registration and makes each skill self-contained.
|
||||
- **Do not preserve compatibility wrappers.** This migration intentionally uses maximum deletion. Removed skills and agents go into legacy cleanup registries; no deprecated skill stubs or one-release grace wrappers.
|
||||
- **Keep generic converter support where it serves non-CE plugins.** The CLI may still convert agents for other plugin payloads, but CE-specific Codex installation should no longer require generated custom agents.
|
||||
|
||||
## Skill Decisions
|
||||
|
||||
### Delete
|
||||
|
||||
| Skill | Disposition |
|
||||
| --- | --- |
|
||||
| `ce-agent-native-audit` | Delete; fold useful checklist material into `ce-agent-native-architecture` or `ce-code-review` only if needed. |
|
||||
| `ce-clean-gone-branches` | Delete. |
|
||||
| `ce-dhh-rails-style` | Delete. |
|
||||
| `ce-frontend-design` | Delete; salvage durable frontend rules into `ce-work`, `ce-work-beta`, and `ce-polish` where relevant. |
|
||||
| `ce-gemini-imagegen` | Delete. |
|
||||
| `ce-release-notes` | Delete. |
|
||||
| `ce-report-bug` | Delete. |
|
||||
| `ce-sessions` | Delete as a public product surface; fold discovery/extraction scripts and historian synthesis into `ce-compound` only for compounding/documentation workflows. |
|
||||
| `ce-slack-research` | Delete; fold Slack research prompts into `ce-brainstorm`, `ce-ideate`, and `ce-plan`. |
|
||||
| `ce-update` | Delete. |
|
||||
|
||||
### Keep
|
||||
|
||||
| Skill | Notes |
|
||||
| --- | --- |
|
||||
| `ce-agent-native-architecture` | Keep as a domain guide for agent-native systems. |
|
||||
| `ce-brainstorm` | Keep; localize Slack research prompt. |
|
||||
| `ce-code-review` | Keep; localize code-review personas. |
|
||||
| `ce-commit` | Keep. |
|
||||
| `ce-commit-push-pr` | Keep. |
|
||||
| `ce-compound` | Keep; absorb session-history workflow. |
|
||||
| `ce-compound-refresh` | Keep. |
|
||||
| `ce-debug` | Keep. |
|
||||
| `ce-demo-reel` | Keep. |
|
||||
| `ce-doc-review` | Keep; localize document-review personas. |
|
||||
| `ce-dogfood-beta` | Keep. |
|
||||
| `ce-ideate` | Keep; localize research prompts. |
|
||||
| `ce-optimize` | Keep; localize research prompts. |
|
||||
| `ce-plan` | Keep; localize research and deepening prompts. |
|
||||
| `ce-polish` | Keep; absorb frontend-design rules where useful. |
|
||||
| `ce-product-pulse` | Keep. |
|
||||
| `ce-promote` | Keep. |
|
||||
| `ce-proof` | Keep. |
|
||||
| `ce-resolve-pr-feedback` | Keep; localize PR comment resolver prompt. |
|
||||
| `ce-riffrec-feedback-analysis` | Keep. |
|
||||
| `ce-setup` | Keep. |
|
||||
| `ce-simplify-code` | Keep. |
|
||||
| `ce-strategy` | Keep. |
|
||||
| `ce-test-browser` | Keep. |
|
||||
| `ce-test-xcode` | Keep. |
|
||||
| `ce-work` | Keep; localize Figma/design-sync prompt if still needed. |
|
||||
| `ce-work-beta` | Keep. |
|
||||
| `ce-worktree` | Keep. |
|
||||
| `lfg` | Keep. |
|
||||
|
||||
## Agent Decisions
|
||||
|
||||
### Preserve as Skill-Local Prompt Assets
|
||||
|
||||
| Agent | Destination |
|
||||
| --- | --- |
|
||||
| `ce-adversarial-document-reviewer` | `ce-doc-review/references/personas/` |
|
||||
| `ce-coherence-reviewer` | `ce-doc-review/references/personas/` |
|
||||
| `ce-design-lens-reviewer` | `ce-doc-review/references/personas/` |
|
||||
| `ce-feasibility-reviewer` | `ce-doc-review/references/personas/` |
|
||||
| `ce-product-lens-reviewer` | `ce-doc-review/references/personas/` |
|
||||
| `ce-scope-guardian-reviewer` | `ce-doc-review/references/personas/` |
|
||||
| `ce-security-lens-reviewer` | `ce-doc-review/references/personas/` |
|
||||
| `ce-adversarial-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-agent-native-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-api-contract-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-correctness-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-julik-frontend-races-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-maintainability-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-performance-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-previous-comments-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-project-standards-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-reliability-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-security-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-swift-ios-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-testing-reviewer` | `ce-code-review/references/personas/` |
|
||||
| `ce-pr-comment-resolver` | `ce-resolve-pr-feedback/references/agents/` |
|
||||
| `ce-figma-design-sync` | `ce-work/references/agents/` and `ce-work-beta/references/agents/` |
|
||||
| `ce-session-historian` | `ce-compound/references/agents/`; move supporting scripts from `ce-sessions`. |
|
||||
| `ce-slack-researcher` | Workflow-tuned local copies in `ce-brainstorm`, `ce-ideate`, and `ce-plan`. |
|
||||
| `ce-learnings-researcher` | Local copies in `ce-code-review`, `ce-ideate`, `ce-optimize`, and `ce-plan`; slim per workflow if practical. |
|
||||
| `ce-repo-research-analyst` | Local copies in `ce-plan` and `ce-optimize`. |
|
||||
| `ce-web-researcher` | Local copies in `ce-plan` and `ce-ideate`. |
|
||||
| `ce-best-practices-researcher` | Local copies in `ce-plan` and `ce-compound`, or merge with framework-docs prompt where it improves clarity. |
|
||||
| `ce-framework-docs-researcher` | Local copies in `ce-plan` and `ce-compound`, or merge with best-practices prompt where it improves clarity. |
|
||||
| `ce-data-migration-reviewer` | Local copies in `ce-code-review` and `ce-plan`. |
|
||||
| `ce-deployment-verification-agent` | Local copies in `ce-code-review` and `ce-plan`. |
|
||||
| `ce-data-integrity-guardian` | Local copies in `ce-plan` and `ce-compound`, or merge into nearby data prompts if redundant. |
|
||||
| `ce-security-sentinel` | Local copies in `ce-plan` and `ce-compound`, or merge into nearby security prompts if redundant. |
|
||||
| `ce-performance-oracle` | Local copies in `ce-plan` and `ce-compound`, or merge into nearby performance prompts if redundant. |
|
||||
| `ce-pattern-recognition-specialist` | Local copies in `ce-plan` and `ce-compound`, or fold into repo research/maintainability prompts if redundant. |
|
||||
| `ce-spec-flow-analyzer` | `ce-plan/references/agents/` |
|
||||
| `ce-architecture-strategist` | `ce-plan/references/agents/` |
|
||||
| `ce-git-history-analyzer` | `ce-plan/references/agents/` |
|
||||
| `ce-issue-intelligence-analyst` | `ce-ideate/references/agents/` |
|
||||
|
||||
### Delete
|
||||
|
||||
| Agent | Reason |
|
||||
| --- | --- |
|
||||
| `ce-ankane-readme-writer` | No runtime consumer. |
|
||||
| `ce-design-implementation-reviewer` | No runtime consumer. |
|
||||
| `ce-design-iterator` | Only consumed by deleted `ce-frontend-design`; salvage useful frontend rules elsewhere if needed. |
|
||||
| `ce-code-simplicity-reviewer` | Delete as a standalone agent; preserve code simplification as an important capability owned by `ce-simplify-code`. |
|
||||
|
||||
## Requirements
|
||||
|
||||
- R1. No surviving CE skill may dispatch a standalone `ce-*` agent by name.
|
||||
- R2. Every surviving subagent dispatch must identify a skill-local prompt asset or inline local prompt block as the source of specialist behavior.
|
||||
- R3. No `plugins/compound-engineering/agents/` directory remains after migration.
|
||||
- R4. Deleted skills and agents are added to both legacy cleanup registries:
|
||||
- `src/utils/legacy-cleanup.ts`
|
||||
- `src/data/plugin-legacy-artifacts.ts`
|
||||
- R5. Plugin README and docs catalogs no longer advertise standalone agents.
|
||||
- R6. Codex docs no longer require `bunx @every-env/compound-plugin install compound-engineering --to codex` for CE custom agents once CE no longer ships standalone agents.
|
||||
- R7. CE-specific converter tests are updated so current CE output is skills-only for Codex. Generic non-CE agent conversion coverage may remain.
|
||||
- R8. `bun run release:validate` passes after manifest, README, and marketplace metadata are updated.
|
||||
- R9. Behavioral validation for modified skills uses the repo's skill-validation path rather than stale in-session plugin agent dispatch.
|
||||
- R10. Install documentation is overhauled everywhere it appears, including plugin READMEs, marketplace/catalog README content, and tracked Markdown docs, so users no longer see obsolete custom-agent install guidance.
|
||||
- R11. `bun test` passes after converter, writer, cleanup, and plugin inventory changes.
|
||||
- R12. The implementation run is tracked under a single `/goal` objective tied to this requirements doc so long-running work can resume without losing the source of truth.
|
||||
- R13. High-risk surviving skills produce skill-eval receipts after migration, comparing revised behavior against a pre-migration snapshot where possible.
|
||||
- R14. Static validation proves there are no surviving CE standalone-agent dependencies: no `plugins/compound-engineering/agents/` directory, no surviving skill dispatches a standalone `ce-*` agent by name, and deleted artifacts are present in both cleanup registries.
|
||||
|
||||
## Validation Strategy
|
||||
|
||||
Use `/goal` for continuity, not as the proof mechanism. The goal should reference this requirements doc and remain active until implementation, docs, cleanup registries, automated tests, release validation, and eval receipts are complete.
|
||||
|
||||
Skill-level evals should use `skill-eval` with `old_skill` baselines for revised skills. Snapshot each target skill before edits when possible, then run the same prompts against the migrated skill and the snapshot. Store artifacts under `/tmp/skill-eval/<skill-name>/<run-id>/` with prompts, transcripts, outputs, grading, benchmark summaries, and review notes.
|
||||
|
||||
Minimum eval coverage:
|
||||
|
||||
| Skill | Eval purpose |
|
||||
| --- | --- |
|
||||
| `ce-code-review` | Prove localized reviewer personas still produce schema-valid, confidence-gated findings on a small diff. |
|
||||
| `ce-doc-review` | Prove localized document-review personas still classify requirements/plan documents, synthesize findings, and apply safe-auto fixes correctly. |
|
||||
| `ce-plan` | Prove localized research/deepening prompts still produce a structured plan from a requirements doc without standalone agent dispatch. |
|
||||
| `ce-compound` | Prove folded session-history support works only inside the compounding workflow and no standalone `ce-sessions` path is required. |
|
||||
| `ce-brainstorm` | Prove localized Slack research prompt can be invoked as workflow context without the deleted `ce-slack-research` skill. |
|
||||
| `ce-ideate` | Prove localized research prompts still ground ideas and preserve the expected artifact shape. |
|
||||
| `ce-optimize` | Prove localized repo/learnings research prompts still support optimization setup without standalone agents. |
|
||||
| `ce-resolve-pr-feedback` | Prove the localized PR comment resolver prompt still evaluates review feedback and emits actionable resolution guidance. |
|
||||
| `ce-work` / `ce-work-beta` | Prove any localized Figma/design-sync prompt remains accessible where the work skills need it; explicitly state the stable/beta sync decision. |
|
||||
| `ce-simplify-code` | Prove code simplification remains available even though `ce-code-simplicity-reviewer` is deleted as a standalone agent. |
|
||||
|
||||
Automated validation should include:
|
||||
|
||||
- `bun test`
|
||||
- `bun run release:validate`
|
||||
- targeted static scans for removed skill names, removed agent names, stale install instructions, and forbidden standalone `ce-*` agent dispatches
|
||||
- native Codex plugin smoke validation when the local Codex plugin install path is available; otherwise record the unavailable command/tooling as a validation gap
|
||||
|
||||
## Open Implementation Notes
|
||||
|
||||
- Prefer moving agent bodies mechanically first, then slimming/merging prompts only where the consuming skill's context clearly warrants it.
|
||||
- For shared prompts, duplication must happen inside each consuming skill directory. Do not create cross-skill shared files.
|
||||
- The implementation plan should sequence this migration by workflow cluster to keep reviewable diffs:
|
||||
1. Delete no-consumer agents and wrapper skills.
|
||||
2. Localize `ce-code-review` and `ce-doc-review` personas.
|
||||
3. Localize planning/research prompts.
|
||||
4. Fold sessions into `ce-compound`.
|
||||
5. Localize remaining one-off prompts.
|
||||
6. Remove CE standalone agent install assumptions from Codex docs, READMEs, Markdown guides, and tests.
|
||||
7. Run static checks, automated tests, release validation, and skill-eval receipts for the high-risk surviving skills.
|
||||
Reference in New Issue
Block a user