main
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Seth Hobson
|
608c3840ca |
feat: native plugin-install for Codex/Cursor/Gemini + CLAUDE.md→AGENTS.md symlink
Add lean, native plugin-install entry points so each harness's own plugin manager can install this marketplace (mirroring obra/superpowers) — committing only small JSON registries, not duplicated skill/agent content trees. - Codex: committed marketplace registry (.agents/plugins/marketplace.json) + per-plugin manifests (plugins/*/.codex-plugin/plugin.json). Entries point at source ./plugins/<name>; Codex reads SKILL.md directly. Transformed .codex/skills|agents stay gitignored. - Cursor: commit the existing .cursor-plugin/ marketplace + .cursor/rules/ (these already point at source plugins/). - Gemini: gemini-extension.json already committed (contextFileName: AGENTS.md); transformed trees stay gitignored (install via clone + make generate). - OpenCode: unchanged — install via `make install-opencode` (transformed tree stays gitignored). - CLAUDE.md is now a symlink to AGENTS.md; Claude-Code addenda moved to docs/harnesses.md. - CI: new step fails if `make generate-all` drifts from the committed registries. Net new committed: ~720 KB of manifests (no skill/agent duplication). Adds round-trip tests for the new registries + the symlink. Docs updated across README, docs/harnesses.md, ARCHITECTURE.md, CONTRIBUTING.md, GEMINI.md, docs/authoring.md, and the PR template. |
||
|
Seth Hobson
|
8df77ecd46 |
Fix duplicate agent name collisions (#554)
* fix: namespace duplicate agent names * fix: format duplicate-name regression test * fix: align command phase output references * docs: clarify agent naming formula |
||
|
Seth Hobson
|
be57c0b2e3 |
feat: multi-harness plugin marketplace (Codex, Cursor, OpenCode, Gemini) (#541)
* feat(adapters): multi-harness framework + harness_portability eval dimension
Turn this Claude Code plugin marketplace into a generic agentic-harness
marketplace. Adapters under tools/adapters/ emit harness-native artifacts
for OpenAI Codex CLI, Cursor, OpenCode, and Gemini CLI from a single
Markdown source. Source-of-truth stays under plugins/ — Claude Code is
unchanged.
Framework (tools/adapters/):
- base.py — PluginSource parser, HarnessAdapter ABC, write/mirror helpers
(path-traversal guard, UTF-8-safe), inline-list + block-list + block-scalar
YAML-ish parser, _utf8_safe_cut, _split_inline_list, _normalize_author
- capabilities.py — per-harness capability matrix, TOOL_NAME_MAPS,
MODEL_ALIASES, resolve_model() with explicit warnings
- codex.py — emits .codex/{skills,agents}/ + AGENTS.md (≤150-line
table-of-contents). Fence-aware body splitter, _utf8_safe_cut for
multibyte safety, _yaml_scalar with reserved-word + special-char quoting.
Skill/command name collision detection (and second-order __cmd fallback).
- cursor.py — emits .cursor-plugin/{plugin,marketplace}.json + curated
.cursor/rules/*.mdc. _validate_mdc_frontmatter handles YAML block scalars
(no false positives on colons in description body). _normalize_author
handles dict, npm-style strings, and author lists.
- opencode.py — transpiles agents to .opencode/agents/<id>.md with
mode:subagent + permission: deny-everything-else block (skill/task always
allowed as base capabilities — Claude's implicit defaults).
- gemini.py — emits native skills/, agents/, and commands/ at extension
root (April 2026 spec). Tool-allowlist remapped via TOOL_NAME_MAPS.
CLI + tooling:
- tools/generate.py — unified `make generate HARNESS=<x> [PLUGIN=<y>]`,
with --clean (containment-guarded; case-insensitive on Darwin/Win32),
--prune (orphan removal across all per-harness output trees), --strict
(warnings fail), per-plugin error aggregation, refuses --clean --plugin
(would silently wipe other plugins' artifacts).
- tools/validate_generated.py — structural validation across all four
harness outputs. Codex 8KB cap → error. _extract_permission_block
correctly handles nested permission keys (column-0 only).
- tools/doc_gardener.py — recurring drift detection per OpenAI harness-
engineering principle. STALE_ARTIFACT (info), DEAD_LINK (error),
MARKETPLACE_ORPHAN (error), SKILL_OVER_CODEX_CAP (warning), grouped
output sorted by severity.
plugin-eval (extends existing framework):
- New harness_portability dimension (6% weight, rebalanced from existing
static sub-scores). Surfaces non-portable patterns with concrete
remediation hints: SKILL_OVER_CODEX_CAP, CLAUDE_TOOL_REFS,
CLAUDE_TOOL_PROSE, AGENT_NAME_COLLISION, BARE_MODEL_ALIAS.
- _CAMEL_TOOL_PATTERN requires Claude-tool context (no false positives
on Rust's `Task` etc.). _TOOL_PROSE_PATTERN case-sensitive on tool
names, case-insensitive on the leading article.
- Findings do NOT also feed anti_pattern_penalty (no double-counting).
Documentation:
- Top-level guides: CODEX.md, CURSOR.md, OPENCODE.md (≤150 lines each,
table-of-contents pattern per OpenAI harness-engineering post)
- docs/harnesses.md — capability matrix, graceful-degradation table,
generated output paths
- docs/authoring.md — portable-content style guide (tools, models,
collision rules, fence-respect)
- docs/round-trip-results.md — real-CLI verification recipes (OpenCode
discovers 193 subagents, Gemini extensions validate passes, Codex
TOMLs all parse)
- CONTRIBUTING.md — new file pointing at docs/authoring.md
- README.md — rewritten for multi-harness (145 lines, was 460)
- CLAUDE.md — trimmed to 60-line table-of-contents
- GEMINI.md — trimmed from 1500 to 500 tokens (3× over budget previously)
Tests: 181 passing (103 plugin-eval + 78 tools/tests). Real-CLI round-trip
verified for OpenCode, Gemini, and Codex (TOML parses).
Replaces tools/generate_gemini_commands.py with the unified CLI.
* refactor(skills): extract detail to references/details.md (~75 skills)
Apply Anthropic's canonical SKILL.md progressive-disclosure pattern across
the marketplace: SKILL.md body becomes a navigation tier (trigger phrasing
+ quick start), detailed templates and worked examples move to
references/details.md (loaded on demand by the agent).
Motivation: OpenAI Codex CLI hard-truncates skills at 8 KB. Before this
change, ~90 skills exceeded that cap and would silently break on Codex.
The progressive-disclosure pattern is also Anthropic's documented
recommendation for token efficiency — Claude Code reads references/ files
on demand when the body navigation says to.
What's extracted, by pattern:
- Pass 1 (## Templates section): 19 skills — full template libraries
moved to references/details.md
- Pass 2 (## Implementation Patterns / ## Advanced Patterns): 13 skills
- Pass 3 (everything between nav-tier and wrap-tier headings): 53 skills
- Conservative re-extraction for 8 skills that got over-reduced — kept
~6-7 KB inline (most of the quick-start tier) plus references/ overflow
What stays inline (SKILL.md navigation tier):
- description: frontmatter (triggering — unchanged for all skills)
- ## When to Use This Skill / ## Core Concepts / ## Quick Start
- ## Best Practices / ## Troubleshooting / ## See Also wrap-ups
- A pointer note ("see references/details.md") so the agent knows where
to look for detail
What goes to references/details.md (detail tier, on-demand load):
- ## Templates (full code template libraries)
- ## Implementation Patterns / ## Advanced Patterns (deep examples)
- Mid-skill walkthroughs that exceed the inline budget
Also in this commit:
- plugins/brand-landingpage description trimmed from 958→543 chars
(preserves trigger phrasing, drops verbose example-quote list)
Net effect:
- SKILL_OVER_CODEX_CAP findings: 90 → 10 (88% reduction)
- All triggers unchanged — discovery behavior identical across harnesses
- 75 new references/details.md files with the extracted content
- Same depth of guidance, loaded progressively
Remaining 10 oversized skills are complex multi-section docs (e.g.
postgresql, code-review-excellence, evaluation-methodology) that need
per-skill manual judgment — flagged by `make garden` for future work.
* chore: bump all plugin versions (multi-harness release)
Patch-bump every local plugin (81) in both .claude-plugin/marketplace.json
entries and each plugins/<name>/.claude-plugin/plugin.json. Minor-bump the
top-level marketplace metadata.version (1.6.0 → 1.7.0) to signal the
multi-harness adapter framework addition.
The external git-subdir entry (qa-orchestra) is unaffected — its version
is governed by its upstream repo.
* fix(opencode): preserve explicit tools:[] + word-boundary subtask match
Addresses two Codex review findings on PR #541.
## P1 — `tools: []` silently upgraded to permissive (privilege escalation)
Before: `_build_permission_block` returned `{}` for any empty list, which
omits the `permission:` block entirely from the emitted agent. An author
who explicitly wrote `tools: []` to lock down an advisory-only agent got
an UNRESTRICTED agent in OpenCode. Affected agent in this tree:
`plugins/arm-cortex-microcontrollers/agents/arm-cortex-expert.md`.
Fix: `_build_permission_block` now takes a `has_tools_field` flag so the
caller can distinguish "tools: key missing" (Claude default permissive)
from "tools: []" (explicit lock-down). The lock-down case emits a
deny-everything block that allows ONLY the base capabilities (skill, task)
that Claude Code always grants implicitly. Verified against the real
arm-cortex-expert agent — now emits read/edit/write/bash/grep/glob/list:
deny, task/skill: allow.
## P2 — `"agent" in cmd.body.lower()` false-positives on substrings
Before: a command body containing `PerformanceReviewAgent` (class name
in a code snippet) or `useragent` triggered `subtask: true`, changing
runtime behavior based on incidental text.
Fix: switch to a compiled word-boundary regex `\b(agent|subagent)s?\b`
(case-insensitive). Tests confirm the substring `PerformanceReviewAgent`
no longer fires, while a real "spawn a subagent" sentence still does.
## Tests
3 new regression tests in tools/tests/test_adapters.py:
- `test_explicit_empty_tools_yields_locked_permission_block` (P1)
- `test_missing_tools_field_yields_no_permission_block` (P1 boundary)
- `test_subtask_inference_word_boundary` (P2)
184 total tests pass (was 181). OpenCode round-trip still discovers all
193 subagents; arm-cortex-expert agent is now properly locked down.
* test: behavioral verification + CI gates for multi-harness pipeline
Adds three layers of automated verification that pure-Python parser tests
miss, plus the CI jobs that turn them into hard gates. Catches the kinds
of issues that previously only surfaced when a real user installed the
marketplace and tried to use it.
## test_real_world.py — real-source structural tests
Runs against the actual `plugins/` tree (not synthetic fixtures). Catches
issues that only appear on real content:
- every marketplace entry resolves to a plugins/<name>/ dir
- every local plugin dir appears in marketplace.json
- marketplace.json version == per-plugin plugin.json version (catches drift)
- every plugin loads via load_plugin() without error
- no plugin name contains `__` (adapter namespace separator)
- every agent has name + description; every skill has a trigger phrase
(same regex plugin_eval's MISSING_TRIGGER check uses)
- no agent name collides with Codex built-ins
- every refactored skill (with `references/details.md`) has:
- meaningful detail content (>=500 B in details.md)
- a pointer to references/ in the SKILL.md body
- a navigation-tier heading preserved (When to Use, Overview, etc.)
- body >= 600 B (not a stub)
- every plugin.json has name + version matching the dir
This test pass found and fixed three real defects before commit:
- ship-mate/skills/scan: description had no trigger phrase ("Use when…")
- reverse-engineering/skills/memory-forensics: nav-tier section lost
during extraction
- reverse-engineering/skills/binary-analysis-patterns: same
All three are now fixed (preserved trigger phrasing, added When-to-Use
sections back to the skills my extraction over-trimmed).
## test_round_trip.py — generate→parse→verify
CI runs this AFTER `make generate-all`. Catches generation-time regressions:
- OpenCode/Codex/Gemini agent counts match source agent count (no skips)
- every Codex SKILL.md under 8 KB (the cap that would silently truncate)
- every Codex agent TOML has required fields + valid sandbox_mode
- every OpenCode agent has mode in {primary,subagent,all} and
provider-prefixed model
- locked agents (source `tools: []`) emit proper deny-everything permission
block with skill/task allow (regression guard for PR-541 P1)
- every Gemini @{path} injection resolves to a real source file
- every Gemini command TOML has prompt + {{args}} placeholder
- every context file (CLAUDE.md, AGENTS.md, GEMINI.md, etc.) within
150-line cap
- Cursor marketplace + per-plugin manifests cover all local plugins
- .cursor/rules/*.mdc only use the 3 documented frontmatter keys
## test_cli_smoke.py — real-CLI subprocess tests
Invokes the actual harness binaries (OpenCode, Gemini, Codex, Claude Code)
against the generated artifacts. Catches CLI-level issues pure-Python
parsing can't see: schema-loader drift, plugin-discovery bugs, version
incompatibilities.
- `opencode agent list` — must succeed AND discover every source agent
(currently 191 + 2 OpenCode built-ins)
- `gemini extensions validate <repo>` — must return success
- `codex doctor` — must report healthy install
- every Codex agent TOML must parse with stdlib `tomllib`
- `claude --version` — sanity check the Claude Code CLI loads
- marketplace.json must have owner + metadata.version for Claude Code's loader
Per-CLI tests skip gracefully when the binary isn't on PATH, so local
devs only exercise what they have installed. CI installs OpenCode +
Gemini and turns those skips into hard gates.
## Makefile + CI
- `make test` — full pytest suite (plugin-eval + tools/tests/)
- `make smoke-test` — generates if needed, then runs real-CLI smoke tests
- `.github/workflows/validate.yml` extended with:
- `tools-tests` job — runs pytest tools/tests/
- `multi-harness-generate` job — `make generate-all && make validate
STRICT=1 && make garden`, uploads generated artifacts on every run
- `cli-smoke-test` job — installs OpenCode + Gemini, runs test_cli_smoke.py
## Test counts
- Before: 184 tests
- After: 386 tests (parameterized real-source tests over all 82 plugins)
- All passing locally on OpenCode 1.15.7 + Gemini 0.42.0 + Codex 0.133.0
+ Claude Code 2.1.148
|
||
|
Seth Hobson
|
4820385a31 |
chore: modernize all plugins to new format with per-plugin plugin.json
Add .claude-plugin/plugin.json to all 67 remaining plugins and simplify marketplace.json entries by removing redundant fields (keywords, strict, commands, agents, skills, repository) that are now auto-discovered. Bump marketplace version to 1.5.0. |
||
|
Ruyut
|
918a770990 | fix: add missing ')' in winston File transport (#426) | ||
|
Seth Hobson
|
56848874a2 | style: format all files with prettier | ||
|
Seth Hobson
|
c7ad381360 |
feat: implement three-tier model strategy with Opus 4.5 (#139)
* feat: implement three-tier model strategy with Opus 4.5 This implements a strategic model selection approach based on agent complexity and use case, addressing Issue #136. Three-Tier Strategy: - Tier 1 (opus): 17 critical agents for architecture, security, code review - Tier 2 (inherit): 21 complex agents where users choose their model - Tier 3 (sonnet): 63 routine development agents (unchanged) - Tier 4 (haiku): 47 fast operational agents (unchanged) Why Opus 4.5 for Tier 1: - 80.9% on SWE-bench (industry-leading for code) - 65% fewer tokens for long-horizon tasks - Superior reasoning for architectural decisions Changes: - Update architect-review, cloud-architect, kubernetes-architect, database-architect, security-auditor, code-reviewer to opus - Update backend-architect, performance-engineer, ai-engineer, prompt-engineer, ml-engineer, mlops-engineer, data-scientist, blockchain-developer, quant-analyst, risk-manager, sql-pro, database-optimizer to inherit - Update README with three-tier model documentation Relates to #136 * feat: comprehensive model tier redistribution for Opus 4.5 This commit implements a strategic rebalancing of agent model assignments, significantly increasing the use of Opus 4.5 for critical coding tasks while ensuring Sonnet is used more than Haiku for support tasks. Final Distribution (153 total agent files): - Tier 1 Opus: 42 agents (27.5%) - All production coding + critical architecture - Tier 2 Inherit: 42 agents (27.5%) - Complex tasks, user-choosable - Tier 3 Sonnet: 38 agents (24.8%) - Support tasks needing intelligence - Tier 4 Haiku: 31 agents (20.3%) - Simple operational tasks Key Changes: Tier 1 (Opus) - Production Coding + Critical Review: - ALL code-reviewers (6 total): Ensures highest quality code review across all contexts (comprehensive, git PR, code docs, codebase cleanup, refactoring, TDD) - All major language pros (7): python, golang, rust, typescript, cpp, java, c - Framework specialists (6): django (2), fastapi (2), graphql-architect (2) - Complex specialists (6): terraform-specialist (3), tdd-orchestrator (2), data-engineer - Blockchain: blockchain-developer (smart contracts are critical) - Game dev (2): unity-developer, minecraft-bukkit-pro - Architecture (existing): architect-review, cloud-architect, kubernetes-architect, hybrid-cloud-architect, database-architect, security-auditor Tier 2 (Inherit) - User Flexibility: - Secondary languages (6): javascript, scala, csharp, ruby, php, elixir - All frontend/mobile (8): frontend-developer (4), mobile-developer (2), flutter-expert, ios-developer - Specialized (6): observability-engineer (2), temporal-python-pro, arm-cortex-expert, context-manager (2), database-optimizer (2) - AI/ML, backend-architect, performance-engineer, quant/risk (existing) Tier 3 (Sonnet) - Intelligent Support: - Documentation (4): docs-architect (2), tutorial-engineer (2) - Testing (2): test-automator (2) - Developer experience (3): dx-optimizer (2), business-analyst - Modernization (4): legacy-modernizer (3), database-admin - Other support agents (existing) Tier 4 (Haiku) - Simple Operations: - SEO/Marketing (10): All SEO agents, content, search - Deployment (4): deployment-engineer (4 instances) - Debugging (5): debugger (2), error-detective (3) - DevOps (3): devops-troubleshooter (3) - Other simple operational tasks Rationale: - Opus 4.5 achieves 80.9% on SWE-bench with 65% fewer tokens on complex tasks - Production code deserves the best model: all language pros now on Opus - All code review uses Opus for maximum quality and security - Sonnet > Haiku (38 vs 31) ensures better intelligence for support tasks - Inherit tier gives users cost control for frontend, mobile, and specialized tasks Related: #136, #132 * feat: upgrade final 13 agents from Haiku to Sonnet Based on research into Haiku 4.5 vs Sonnet 4.5 capabilities, upgraded agents requiring deep analytical intelligence from Haiku to Sonnet. Research Findings: - Haiku 4.5: 73.3% SWE-bench, 3-5x faster, 1/3 cost, sub-200ms responses - Best for Haiku: Real-time apps, data extraction, templates, high-volume ops - Best for Sonnet: Complex reasoning, root cause analysis, strategic planning Agents Upgraded (13 total): - Debugging (5): debugger (2), error-detective (3) - Complex root cause analysis - DevOps (3): devops-troubleshooter (3) - System diagnostics & troubleshooting - Network (2): network-engineer (2) - Complex network analysis & optimization - API Documentation (2): api-documenter (2) - Deep API understanding required - Payments (1): payment-integration - Critical financial integration Final Distribution (153 total): - Tier 1 Opus: 42 agents (27.5%) - Production coding + critical architecture - Tier 2 Inherit: 42 agents (27.5%) - Complex tasks, user-choosable - Tier 3 Sonnet: 51 agents (33.3%) - Support tasks needing intelligence - Tier 4 Haiku: 18 agents (11.8%) - Fast operational tasks only Haiku Now Reserved For: - SEO/Marketing (8): Pattern matching, data extraction, content templates - Deployment (4): Operational execution tasks - Simple Docs (3): reference-builder, mermaid-expert, c4-code - Sales/Support (2): High-volume, template-based interactions - Search (1): Knowledge retrieval Sonnet > Haiku as requested (51 vs 18) Sources: - https://www.creolestudios.com/claude-haiku-4-5-vs-sonnet-4-5-comparison/ - https://www.anthropic.com/news/claude-haiku-4-5 - https://caylent.com/blog/claude-haiku-4-5-deep-dive-cost-capabilities-and-the-multi-agent-opportunity Related: #136 * docs: add cost considerations and clarify inherit behavior Addresses PR feedback: - Added comprehensive cost comparison for all model tiers - Documented how 'inherit' model works (uses session default, falls back to Sonnet) - Explained cost optimization strategies - Clarified when Opus token efficiency offsets higher rate This helps users make informed decisions about model selection and cost control. |
||
|
Seth Hobson
|
8346c1f2f7 |
refactor: migrate to model-agnostic Sonnet/Haiku architecture
- Migrate all 48 Opus agents to Sonnet - Optimize 35 execution-focused agents for Haiku - Update README with hybrid orchestration patterns - Simplify model configuration to use agnostic aliases Final distribution: 97 Sonnet / 47 Haiku agents |
||
|
Seth Hobson
|
20d4472a3b |
Restructure marketplace for isolated plugin architecture
- Organize 62 plugins into isolated directories under plugins/
- Consolidate tools and workflows into commands/ following Anthropic conventions
- Update marketplace.json with isolated source paths for each plugin
- Revise README to reflect plugin-based structure and token efficiency
- Remove shared resource directories (agents/, tools/, workflows/)
Each plugin now contains only its specific agents and commands, enabling
granular installation and minimal token usage. Installing a single plugin
loads only its resources rather than the entire marketplace.
Structure: plugins/{plugin-name}/{agents/,commands/}
|