chore: import upstream snapshot with attribution
This commit is contained in:
@@ -0,0 +1,75 @@
|
||||
# Code of Conduct
|
||||
|
||||
## Our Pledge
|
||||
|
||||
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
|
||||
|
||||
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
|
||||
|
||||
## Our Standards
|
||||
|
||||
### Acceptable Behavior
|
||||
|
||||
- Using welcoming and inclusive language
|
||||
- Being respectful of differing viewpoints and experiences
|
||||
- Gracefully accepting constructive criticism
|
||||
- Focusing on what is best for the community
|
||||
- Showing empathy towards other community members
|
||||
- Contributing constructively to discussions about AI agents and development
|
||||
|
||||
### Unacceptable Behavior
|
||||
|
||||
The following behaviors are considered harassment and are unacceptable:
|
||||
|
||||
- **Hate speech**: The use of abusive or threatening speech that expresses prejudice against a particular group, especially on the basis of race, religion, gender, sexual orientation, or other characteristics
|
||||
- **Discriminatory language**: Slurs, offensive comments, or language targeting protected characteristics
|
||||
- **Personal attacks**: Insulting, demeaning, or hostile comments directed at individuals
|
||||
- **Harassment**: Deliberate intimidation, stalking, following, or threatening
|
||||
- **Doxxing**: Publishing private information without consent
|
||||
- **Spam**: Excessive off-topic content, promotional material, or repetitive posts
|
||||
- **Automated abuse**: Using scripts, bots, or AI tools to bulk-create issues, PRs, or comments
|
||||
- **LLM-generated spam**: Submitting AI-generated content en masse without meaningful human curation or relevance
|
||||
- **Trolling**: Deliberately inflammatory or disruptive behavior
|
||||
- **Sexual harassment**: Unwelcome sexual attention or advances
|
||||
|
||||
## Enforcement
|
||||
|
||||
### Reporting
|
||||
|
||||
If you experience or witness unacceptable behavior, please report it by:
|
||||
|
||||
- Creating an issue with the `moderation` label
|
||||
- Contacting the repository maintainers directly
|
||||
- Using GitHub's built-in reporting mechanisms
|
||||
|
||||
### Consequences
|
||||
|
||||
Community leaders will follow these guidelines in determining consequences:
|
||||
|
||||
1. **Warning**: First offense or minor violation
|
||||
2. **Temporary restriction**: Temporary loss of interaction privileges
|
||||
3. **Permanent ban**: Severe or repeated violations
|
||||
|
||||
### Enforcement Actions
|
||||
|
||||
- **Immediate removal**: Hate speech, threats, or doxxing will result in immediate content removal and user blocking
|
||||
- **Automated spam**: Bulk automated submissions will result in immediate content removal and permanent account blocking without warning
|
||||
- **Issue/PR closure**: Inappropriate content will be closed and locked
|
||||
- **Account blocking**: Repeat offenders will be blocked from the repository
|
||||
|
||||
## Scope
|
||||
|
||||
This Code of Conduct applies within all community spaces, including:
|
||||
|
||||
- Issues and pull requests
|
||||
- Discussions and comments
|
||||
- Wiki and documentation
|
||||
- External representations of the project
|
||||
|
||||
## Attribution
|
||||
|
||||
This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/), version 2.1.
|
||||
|
||||
## Contact
|
||||
|
||||
Questions about the Code of Conduct can be directed to the repository maintainers through GitHub issues or discussions.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Contributing
|
||||
|
||||
The canonical contributing guide lives at the repo root:
|
||||
**[CONTRIBUTING.md](../CONTRIBUTING.md)**.
|
||||
|
||||
This file exists only because GitHub will look for `.github/CONTRIBUTING.md`
|
||||
when surfacing the contributing prompt to new issue / PR authors. The root
|
||||
file is the source of truth.
|
||||
|
||||
For the multi-harness style guide (Claude Code · OpenAI Codex CLI · Cursor ·
|
||||
OpenCode · Gemini CLI), see
|
||||
[`docs/authoring.md`](../docs/authoring.md).
|
||||
@@ -0,0 +1 @@
|
||||
github: wshobson
|
||||
@@ -0,0 +1,114 @@
|
||||
---
|
||||
name: Bug Report
|
||||
description: Report a bug in a plugin, agent, skill, command, or harness adapter
|
||||
title: "[BUG] "
|
||||
labels: ["bug"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for reporting a bug! Please fill out this template
|
||||
to help us understand and resolve the issue.
|
||||
|
||||
**Important**: This template is for technical bugs only.
|
||||
For questions, discussions, or general feedback, please use
|
||||
GitHub Discussions instead.
|
||||
|
||||
- type: checkboxes
|
||||
id: preliminary-checks
|
||||
attributes:
|
||||
label: Preliminary Checks
|
||||
description: Please confirm you have completed these steps
|
||||
options:
|
||||
- label: >-
|
||||
I have read the [Code of
|
||||
Conduct](https://github.com/wshobson/agents/blob/main/.github/CODE_OF_CONDUCT.md)
|
||||
required: true
|
||||
- label: >-
|
||||
I have searched existing issues to ensure this is not a duplicate
|
||||
required: true
|
||||
- label: This report contains only technical information about a bug
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: harness
|
||||
attributes:
|
||||
label: Harness
|
||||
description: Which harness exhibits the bug? Pick "Multiple" if cross-cutting.
|
||||
options:
|
||||
- Claude Code
|
||||
- OpenAI Codex CLI
|
||||
- Cursor
|
||||
- OpenCode
|
||||
- Gemini CLI
|
||||
- Multiple harnesses
|
||||
- Adapter framework / tooling (not a specific harness)
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: harness-version
|
||||
attributes:
|
||||
label: Harness version
|
||||
description: Output of `<cli> --version` (e.g., `claude --version`, `opencode --version`).
|
||||
placeholder: e.g., Claude Code 2.1.148, OpenCode 1.15.7
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: input
|
||||
id: affected-component
|
||||
attributes:
|
||||
label: Affected plugin / agent / skill / command
|
||||
description: >-
|
||||
Which marketplace component is broken? Use the source path
|
||||
(`plugins/<name>/skills/<n>/SKILL.md`) or the namespaced name
|
||||
(`backend-development__backend-architect`).
|
||||
placeholder: e.g., python-development, plugins/agent-teams/skills/parallel-debugging
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: bug-description
|
||||
attributes:
|
||||
label: Bug description
|
||||
description: A clear and concise description of what the bug is.
|
||||
placeholder: Describe the unexpected behavior...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: reproduction-steps
|
||||
attributes:
|
||||
label: Steps to reproduce
|
||||
description: Reproduction recipe — include the exact commands you ran.
|
||||
placeholder: |
|
||||
1. `/plugin install python-development` (or equivalent for your harness)
|
||||
2. Trigger the skill / invoke the agent...
|
||||
3. Observe...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: expected-behavior
|
||||
attributes:
|
||||
label: Expected behavior
|
||||
description: What you expected to happen.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: actual-behavior
|
||||
attributes:
|
||||
label: Actual behavior
|
||||
description: What actually happened. Paste any error output verbatim.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: >-
|
||||
Any other context, screenshots, logs, output from `make validate` /
|
||||
`make garden`, or environment details.
|
||||
placeholder: Add any other context about the problem here...
|
||||
@@ -0,0 +1,28 @@
|
||||
---
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: GitHub Discussions
|
||||
url: https://github.com/wshobson/agents/discussions
|
||||
about: >-
|
||||
Questions, brainstorming, and general discussion about plugins,
|
||||
agents, skills, and the multi-harness adapter framework.
|
||||
- name: AGENTS.md (canonical context)
|
||||
url: https://github.com/wshobson/agents/blob/main/AGENTS.md
|
||||
about: >-
|
||||
Start here — table of contents for the whole repo, read natively by
|
||||
Codex / Cursor / OpenCode and imported by Claude Code / Gemini CLI.
|
||||
- name: Authoring guide (cross-harness style guide)
|
||||
url: https://github.com/wshobson/agents/blob/main/docs/authoring.md
|
||||
about: >-
|
||||
Read before adding new plugins or components — covers portability
|
||||
across Claude Code, Codex CLI, Cursor, OpenCode, and Gemini CLI.
|
||||
- name: Per-harness setup
|
||||
url: https://github.com/wshobson/agents/blob/main/docs/harnesses.md
|
||||
about: >-
|
||||
Capability matrix and setup guides for each supported harness.
|
||||
- name: Contributing guide
|
||||
url: https://github.com/wshobson/agents/blob/main/CONTRIBUTING.md
|
||||
about: How to contribute — branching, PRs, tests, quality gates.
|
||||
- name: Community guidelines (Code of Conduct)
|
||||
url: https://github.com/wshobson/agents/blob/main/.github/CODE_OF_CONDUCT.md
|
||||
about: Community standards for all interactions.
|
||||
@@ -0,0 +1,106 @@
|
||||
---
|
||||
name: Feature Request
|
||||
description: >-
|
||||
Suggest an improvement to a plugin / agent / skill / command, the adapter
|
||||
framework, or harness support
|
||||
title: "[FEATURE] "
|
||||
labels: ["enhancement"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for suggesting a feature! Please provide detailed
|
||||
information to help us understand your request.
|
||||
|
||||
**Note**: For new plugins or new agents, use the "New Component
|
||||
Proposal" template instead.
|
||||
|
||||
- type: checkboxes
|
||||
id: preliminary-checks
|
||||
attributes:
|
||||
label: Preliminary Checks
|
||||
description: Please confirm you have completed these steps
|
||||
options:
|
||||
- label: >-
|
||||
I have read the [Code of
|
||||
Conduct](https://github.com/wshobson/agents/blob/main/.github/CODE_OF_CONDUCT.md)
|
||||
required: true
|
||||
- label: >-
|
||||
I have searched existing issues to ensure this is not a duplicate
|
||||
required: true
|
||||
- label: This is a constructive feature request for legitimate use cases
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: scope
|
||||
attributes:
|
||||
label: Scope
|
||||
description: What part of the marketplace would this feature affect?
|
||||
options:
|
||||
- Existing plugin / agent / skill / command (improvement)
|
||||
- Adapter framework (Codex / Cursor / OpenCode / Gemini)
|
||||
- New harness support
|
||||
- Quality tooling (validators, gardener, plugin-eval)
|
||||
- Documentation
|
||||
- CI / build / release
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: affected-component
|
||||
attributes:
|
||||
label: Target component (if scope is plugin/agent/skill/command)
|
||||
description: >-
|
||||
Which marketplace component would this enhance? Use the source path
|
||||
or namespaced name. Leave blank for framework-wide features.
|
||||
placeholder: >-
|
||||
e.g., python-development,
|
||||
plugins/backend-development/agents/backend-architect.md
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: problem-description
|
||||
attributes:
|
||||
label: Problem description
|
||||
description: What problem does this feature solve?
|
||||
placeholder: Currently, users need to... This is frustrating because...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: proposed-solution
|
||||
attributes:
|
||||
label: Proposed solution
|
||||
description: Describe your preferred solution.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: use-cases
|
||||
attributes:
|
||||
label: Use cases
|
||||
description: Concrete scenarios where this feature would be valuable.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: alternatives
|
||||
attributes:
|
||||
label: Alternatives considered
|
||||
description: >-
|
||||
Other approaches you considered. Cross-harness implications:
|
||||
which harnesses benefit / are unaffected / would need changes?
|
||||
placeholder: |
|
||||
- Approach A: ...
|
||||
- Approach B: ...
|
||||
- Harness impact: works for Claude Code natively, would need ...
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: Links to docs, examples, or related issues.
|
||||
@@ -0,0 +1,118 @@
|
||||
---
|
||||
name: Moderation Report
|
||||
description: >-
|
||||
Report inappropriate content, behavior, or Code of Conduct violations
|
||||
title: "[MODERATION] "
|
||||
labels: ["moderation"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
**⚠️ Use this template to report violations of our Code of Conduct,
|
||||
inappropriate content, or concerning behavior.**
|
||||
|
||||
All reports are taken seriously and will be reviewed by maintainers.
|
||||
False reports may result in moderation action.
|
||||
|
||||
**For urgent safety concerns or severe violations, you may also use
|
||||
GitHub's built-in reporting tools.**
|
||||
|
||||
- type: checkboxes
|
||||
id: preliminary-checks
|
||||
attributes:
|
||||
label: Reporting Guidelines
|
||||
description: Please confirm you understand these guidelines
|
||||
options:
|
||||
- label: I am reporting a genuine violation of the Code of Conduct
|
||||
required: true
|
||||
- label: I understand that false reports may result in moderation action
|
||||
required: true
|
||||
- label: >-
|
||||
I have attempted to resolve minor issues through direct
|
||||
communication (if applicable)
|
||||
required: false
|
||||
|
||||
- type: dropdown
|
||||
id: violation-type
|
||||
attributes:
|
||||
label: Type of Violation
|
||||
description: What type of inappropriate behavior are you reporting?
|
||||
options:
|
||||
- Hate speech or discriminatory language
|
||||
- Personal attacks or harassment
|
||||
- Spam or off-topic content
|
||||
- Inappropriate or offensive content
|
||||
- Threats or intimidation
|
||||
- Doxxing or privacy violations
|
||||
- Impersonation
|
||||
- Other Code of Conduct violation
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: location
|
||||
attributes:
|
||||
label: Location of Violation
|
||||
description: >-
|
||||
Where did this occur? (issue number, PR, comment link, etc.)
|
||||
placeholder: e.g., Issue #123, PR #456, comment in issue #789 # Location
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: user-involved
|
||||
attributes:
|
||||
label: User(s) Involved
|
||||
description: >-
|
||||
GitHub username(s) of the person(s) involved (if known)
|
||||
placeholder: e.g., @username1, @username2
|
||||
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: Description of Violation
|
||||
description: >-
|
||||
Detailed description of what happened and why it violates
|
||||
our Code of Conduct
|
||||
placeholder: Please provide specific details about the violation...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: evidence
|
||||
attributes:
|
||||
label: Evidence
|
||||
description: Any additional evidence (quotes, screenshots, etc.)
|
||||
placeholder: |
|
||||
You can include:
|
||||
- Direct quotes (use > for blockquotes)
|
||||
- Links to specific comments
|
||||
- Screenshots (drag and drop images here)
|
||||
- Timestamps of when violations occurred
|
||||
|
||||
- type: textarea
|
||||
id: impact
|
||||
attributes:
|
||||
label: Impact
|
||||
description: How has this affected you or the community?
|
||||
placeholder: >-
|
||||
This behavior has made me feel... It affects the community by...
|
||||
|
||||
- type: checkboxes
|
||||
id: previous-reports
|
||||
attributes:
|
||||
label: Previous Reports
|
||||
options:
|
||||
- label: This is the first time I'm reporting this user/behavior
|
||||
- label: I have reported this user/behavior before
|
||||
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
**What happens next:**
|
||||
- Maintainers will review your report within 24-48 hours
|
||||
- We may follow up with questions if needed
|
||||
- We will take appropriate action based on our Code of Conduct
|
||||
- We will update you on the resolution when possible
|
||||
|
||||
Thank you for helping maintain a respectful community.
|
||||
@@ -0,0 +1,156 @@
|
||||
---
|
||||
name: New Component Proposal
|
||||
description: >-
|
||||
Propose a new plugin, agent, skill, or command for the marketplace
|
||||
title: "[NEW] "
|
||||
labels: ["new-component"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thank you for proposing a new component! Quality components require
|
||||
careful design, clear specialization, and cross-harness portability.
|
||||
|
||||
Read [docs/authoring.md](https://github.com/wshobson/agents/blob/main/docs/authoring.md)
|
||||
before submitting — it covers the portable-content style guide that
|
||||
keeps your component working across Claude Code, Codex CLI, Cursor,
|
||||
OpenCode, and Gemini CLI.
|
||||
|
||||
**Important**: Only propose components for legitimate, constructive
|
||||
use cases. Proposals for malicious or harmful capabilities will be
|
||||
rejected and may result in moderation action.
|
||||
|
||||
- type: checkboxes
|
||||
id: preliminary-checks
|
||||
attributes:
|
||||
label: Preliminary Checks
|
||||
description: Please confirm you have completed these steps
|
||||
options:
|
||||
- label: >-
|
||||
I have read the [Code of
|
||||
Conduct](https://github.com/wshobson/agents/blob/main/.github/CODE_OF_CONDUCT.md)
|
||||
required: true
|
||||
- label: >-
|
||||
I have read
|
||||
[docs/authoring.md](https://github.com/wshobson/agents/blob/main/docs/authoring.md)
|
||||
required: true
|
||||
- label: >-
|
||||
I have reviewed
|
||||
[docs/plugins.md](https://github.com/wshobson/agents/blob/main/docs/plugins.md)
|
||||
to ensure this isn't a duplicate
|
||||
required: true
|
||||
- label: This proposal is for legitimate, constructive use cases only
|
||||
required: true
|
||||
- label: I have domain expertise in the proposed area
|
||||
required: true
|
||||
|
||||
- type: dropdown
|
||||
id: component-type
|
||||
attributes:
|
||||
label: Component type
|
||||
description: What kind of component are you proposing?
|
||||
options:
|
||||
- New plugin (bundles multiple agents / skills / commands)
|
||||
- New agent (under an existing plugin)
|
||||
- New skill (under an existing plugin)
|
||||
- New command (under an existing plugin)
|
||||
- New harness adapter (for an additional CLI / IDE)
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: component-name
|
||||
attributes:
|
||||
label: Proposed name
|
||||
description: lowercase, hyphen-separated; no `__` (adapter namespace separator)
|
||||
placeholder: e.g., devops-security, llm-quantization, k8s-architect
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: input
|
||||
id: parent-plugin
|
||||
attributes:
|
||||
label: Parent plugin (if adding an agent / skill / command)
|
||||
description: >-
|
||||
Existing plugin directory name. Leave blank if proposing a new plugin
|
||||
or harness adapter.
|
||||
placeholder: e.g., python-development, security-scanning
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: domain-expertise
|
||||
attributes:
|
||||
label: Domain & expertise
|
||||
description: What domain does this component specialize in?
|
||||
placeholder: This component specializes in...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: unique-value
|
||||
attributes:
|
||||
label: Unique value proposition
|
||||
description: >-
|
||||
How is this different from existing components? What gap does it fill?
|
||||
placeholder: Unlike existing components, this one would...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: use-cases
|
||||
attributes:
|
||||
label: Primary use cases
|
||||
description: Concrete tasks this component handles.
|
||||
placeholder: |
|
||||
1. Design and implement...
|
||||
2. Optimize performance for...
|
||||
3. Debug issues related to...
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: portability-notes
|
||||
attributes:
|
||||
label: Cross-harness portability notes
|
||||
description: >-
|
||||
Are there features unique to one harness this component relies on
|
||||
(e.g., TodoWrite, MCP, parallel Task tool)? How does it degrade
|
||||
gracefully on harnesses without those features?
|
||||
placeholder: |
|
||||
- Works on all 5 harnesses; uses only file ops and shell commands.
|
||||
- OR: uses `Task` (Claude/Cursor/OpenCode/Gemini), degrades to
|
||||
sequential execution on Codex.
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: examples
|
||||
attributes:
|
||||
label: Example interactions
|
||||
description: 2-3 example user prompts → agent / skill behavior
|
||||
placeholder: |
|
||||
**Example 1:**
|
||||
User: "Implement a smart contract for..."
|
||||
Response: "I'll create a secure smart contract with..."
|
||||
|
||||
**Example 2:**
|
||||
User: "Optimize this blockchain query..."
|
||||
Response: "Let me analyze the query..."
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: expertise-evidence
|
||||
attributes:
|
||||
label: Your expertise
|
||||
description: Experience or credentials in this domain.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: additional-context
|
||||
attributes:
|
||||
label: Additional context
|
||||
description: >-
|
||||
Links to docs, similar tools, potential portability challenges.
|
||||
@@ -0,0 +1,66 @@
|
||||
<!--
|
||||
Thanks for the PR. A few quick orientation pointers:
|
||||
- AGENTS.md is the canonical context file (read by Codex/Cursor/OpenCode; CLAUDE.md is a
|
||||
symlink to AGENTS.md; Gemini reads it via `.gemini/settings.json`).
|
||||
- docs/authoring.md is the portable-content style guide.
|
||||
- If you touched plugin source, run `make generate-all` to refresh the committed
|
||||
native-install registries (CI gates registry drift).
|
||||
- CI runs make validate STRICT=1, make garden, make test, make smoke-test, and the
|
||||
code-quality workflow (ruff/ty/markdownlint).
|
||||
-->
|
||||
|
||||
## Summary
|
||||
|
||||
<!-- 1-3 bullet points: what changed and why. -->
|
||||
|
||||
## Scope
|
||||
|
||||
<!-- Tick everything that applies. -->
|
||||
|
||||
- [ ] Plugin authoring (new or modified `plugins/<name>/...`)
|
||||
- [ ] Adapter framework (`tools/adapters/`)
|
||||
- [ ] Quality tooling (validators, gardener, plugin-eval)
|
||||
- [ ] Per-harness setup or docs (GEMINI.md / docs/harnesses.md)
|
||||
- [ ] AGENTS.md / ARCHITECTURE.md / docs/
|
||||
- [ ] CI / build / release
|
||||
- [ ] Other
|
||||
|
||||
## Affected harnesses
|
||||
|
||||
<!-- Which harnesses are affected? Multi-harness changes need cross-harness review. -->
|
||||
|
||||
- [ ] Claude Code
|
||||
- [ ] OpenAI Codex CLI
|
||||
- [ ] Cursor
|
||||
- [ ] OpenCode
|
||||
- [ ] Gemini CLI
|
||||
- [ ] Pure tooling / framework (no harness behavior change)
|
||||
|
||||
## Test plan
|
||||
|
||||
<!-- Check what you ran locally. CI runs all of these. -->
|
||||
|
||||
- [ ] `make validate STRICT=1`
|
||||
- [ ] `make garden`
|
||||
- [ ] `make test` (full pytest suite)
|
||||
- [ ] `make smoke-test` (real-CLI subprocess tests)
|
||||
- [ ] Ran ruff + ty in `plugins/plugin-eval/`
|
||||
- [ ] Spot-checked behavior in a real harness (which?): ____________
|
||||
|
||||
## Cross-harness portability notes
|
||||
|
||||
<!--
|
||||
Only if you touched plugin content. Per docs/authoring.md:
|
||||
- Codex caps skill bodies at 8 KB
|
||||
- OpenCode requires lowercase tool names
|
||||
- Cursor doesn't honor per-agent tools: allowlists
|
||||
- Context files ≤150 lines
|
||||
-->
|
||||
|
||||
N/A — or:
|
||||
|
||||
-
|
||||
|
||||
## Related issue(s)
|
||||
|
||||
Closes #
|
||||
@@ -0,0 +1,42 @@
|
||||
# Dependabot config for the uv-managed Python projects in this repo.
|
||||
# See AGENTS.md for the canonical toolchain (uv + ruff + ty).
|
||||
#
|
||||
# Native uv support landed in GitHub Dependabot in 2024; the ecosystem
|
||||
# key is "uv" and Dependabot will update both pyproject.toml and uv.lock
|
||||
# in lock-step.
|
||||
|
||||
version: 2
|
||||
updates:
|
||||
# tools/yt-design-extractor — internal tooling
|
||||
- package-ecosystem: "uv"
|
||||
directory: "/tools/yt-design-extractor"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
day: "monday"
|
||||
open-pull-requests-limit: 5
|
||||
labels:
|
||||
- "dependencies"
|
||||
commit-message:
|
||||
prefix: "deps(yt-design-extractor)"
|
||||
groups:
|
||||
python-minor-and-patch:
|
||||
update-types:
|
||||
- "minor"
|
||||
- "patch"
|
||||
|
||||
# plugins/plugin-eval — plugin quality evaluation framework
|
||||
- package-ecosystem: "uv"
|
||||
directory: "/plugins/plugin-eval"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
day: "monday"
|
||||
open-pull-requests-limit: 5
|
||||
labels:
|
||||
- "dependencies"
|
||||
commit-message:
|
||||
prefix: "deps(plugin-eval)"
|
||||
groups:
|
||||
python-minor-and-patch:
|
||||
update-types:
|
||||
- "minor"
|
||||
- "patch"
|
||||
@@ -0,0 +1,147 @@
|
||||
name: Claude Code Review
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, synchronize, ready_for_review, reopened]
|
||||
|
||||
jobs:
|
||||
claude-review:
|
||||
if: |
|
||||
github.event.pull_request.draft == false &&
|
||||
github.actor != 'dependabot[bot]' &&
|
||||
github.event.pull_request.user.login != 'dependabot[bot]' &&
|
||||
github.event.pull_request.head.repo.full_name == github.repository
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
issues: read
|
||||
id-token: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
persist-credentials: false
|
||||
|
||||
- name: Run Claude Code Review
|
||||
id: claude-review
|
||||
uses: anthropics/claude-code-action@4481e6d3c7bbb88db2a928ca3444c536f589c7c1 # v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
track_progress: true
|
||||
|
||||
prompt: |
|
||||
REPO: ${{ github.repository }}
|
||||
PR NUMBER: ${{ github.event.pull_request.number }}
|
||||
|
||||
You are reviewing a PR for claude-agents — a multi-harness
|
||||
agentic plugin marketplace (Claude Code, Codex CLI, Cursor,
|
||||
OpenCode, Gemini CLI) with 82+ plugins, 191+ agents, 155+
|
||||
skills, and 102+ commands. The source of truth is Markdown
|
||||
under `plugins/`; per-harness artifacts under `.codex/`,
|
||||
`.cursor-plugin/`, `.cursor/rules/`, `.opencode/`, and the
|
||||
top-level `skills/`/`agents/`/`commands/` directories are
|
||||
generated by `make generate HARNESS=<name>` and are
|
||||
gitignored.
|
||||
|
||||
Read `AGENTS.md` at the repo root (canonical context) before
|
||||
starting. Consult `docs/authoring.md` for plugin / agent /
|
||||
skill frontmatter shapes, `docs/harnesses.md` for per-harness
|
||||
capability deltas, and `docs/plugins.md` for the catalog.
|
||||
|
||||
## Review checklist
|
||||
|
||||
1. **Source-of-truth invariant** — Only `plugins/`,
|
||||
`.claude-plugin/marketplace.json`, `docs/`, `tools/`, and
|
||||
top-level Markdown (`AGENTS.md`, `CLAUDE.md`, `GEMINI.md`,
|
||||
`README.md`, `ARCHITECTURE.md`, `CONTRIBUTING.md`) should
|
||||
be hand-edited. Flag any change under `.codex/`,
|
||||
`.cursor-plugin/`, `.cursor/rules/`, `.opencode/`, or
|
||||
top-level `skills/`/`agents/`/`commands/` at the extension
|
||||
root — those are generated artifacts and must not be
|
||||
committed by hand.
|
||||
|
||||
2. **Plugin / agent / skill frontmatter** — Every agent under
|
||||
`plugins/*/agents/*.md` needs `name:`, `description:`, and
|
||||
a `model:` tier. Every skill under
|
||||
`plugins/*/skills/*/SKILL.md` needs `name:` and
|
||||
`description:`. Plugin directory names must be lowercase,
|
||||
hyphen-separated, and must NOT contain `__` (that is the
|
||||
adapter namespace separator — see `docs/authoring.md`).
|
||||
|
||||
3. **Cross-harness portability** — Content should work across
|
||||
all five harnesses unless explicitly marked Claude-Code-
|
||||
only in `CLAUDE.md`. Watch for hard dependencies on
|
||||
Claude-Code-only primitives (`TodoWrite`, the `Task` /
|
||||
`Agent` spawn tool, per-agent `tools:` frontmatter) without
|
||||
a documented fallback. Locked agents (`tools: []`) get
|
||||
special-cased by the OpenCode adapter — preserve that
|
||||
contract.
|
||||
|
||||
4. **Codex 8 KB skill body cap** — Skill bodies in
|
||||
`plugins/*/skills/*/SKILL.md` should fit under ~8 KB after
|
||||
adapter transpilation; overflow belongs in
|
||||
`references/details.md`. `make garden` flags oversize
|
||||
skills; if a new or edited skill is borderline, suggest
|
||||
splitting before merge.
|
||||
|
||||
5. **Canonical context sync** — `AGENTS.md` is the single
|
||||
source of truth; `CLAUDE.md` imports it via `@AGENTS.md`
|
||||
and `GEMINI.md` mirrors it. If any of the three diverge in
|
||||
this PR, call it out. Per OpenAI's harness-engineering
|
||||
practice, `AGENTS.md` must stay under ~150 lines — detail
|
||||
belongs in `docs/`.
|
||||
|
||||
6. **Quality gates** — Will this break
|
||||
`make validate STRICT=1`, `make garden`, `make test`, or
|
||||
`make smoke-test`? Reference the relevant gate by name in
|
||||
your finding so the author knows what to run.
|
||||
|
||||
7. **Catalog drift** — New, removed, or renamed plugins,
|
||||
agents, skills, or commands should be reflected in
|
||||
`docs/plugins.md`, `docs/agents.md`, and
|
||||
`docs/agent-skills.md` counts and lists. The plugin total
|
||||
in `AGENTS.md` and `README.md` should also stay in sync.
|
||||
|
||||
8. **Python tooling correctness** — Code under `tools/` uses
|
||||
uv + ruff + ty (NOT pip / mypy / black). Flag any
|
||||
reintroduction of `pip`, `requirements.txt`, `mypy`, or
|
||||
`black`. Watch for unhandled errors in adapter code,
|
||||
broken JSON in `plugin.json` or `marketplace.json`, and
|
||||
missing tests under `tools/tests/`.
|
||||
|
||||
9. **Security** — No secrets in code or workflow files; no
|
||||
destructive git in scripts (`push --force`,
|
||||
`reset --hard`, `branch -D`); no shell injection in
|
||||
`Bash(...)` allowlists or hook scripts; pinned action SHAs
|
||||
and `persist-credentials: false` on any new GitHub
|
||||
workflow (match the style in
|
||||
`.github/workflows/validate.yml`).
|
||||
|
||||
## Output rules
|
||||
|
||||
- Use inline comments
|
||||
(`mcp__github_inline_comment__create_inline_comment`) for
|
||||
specific issues tied to a line of code.
|
||||
- Use `gh pr comment` for a short summary (≤ 10 lines) at
|
||||
the end.
|
||||
- If no critical issues are found, post a single one-line
|
||||
✅ summary via `gh pr comment` and stop — do NOT attempt to
|
||||
submit a formal review approval (no review-submission tool
|
||||
is exposed).
|
||||
- Do NOT comment on formatting, import order, or naming
|
||||
style — `ruff` and `ty` handle that.
|
||||
- Do NOT suggest documentation or comment additions unless a
|
||||
public-facing doc count or catalog entry is wrong.
|
||||
- Do NOT re-raise issues already flagged by `coderabbitai`
|
||||
or `chatgpt-codex-connector` in this PR's existing
|
||||
comments — read them first via
|
||||
`gh api repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/comments`.
|
||||
- Be concise: one sentence per finding is enough. Prefer
|
||||
actionable suggestions (diff blocks) over prose.
|
||||
|
||||
claude_args: |
|
||||
--model claude-opus-4-7
|
||||
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh api repos/*/pulls/*/comments:*)"
|
||||
@@ -0,0 +1,79 @@
|
||||
name: Claude Code
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
pull_request_review_comment:
|
||||
types: [created]
|
||||
issues:
|
||||
# `edited` (not `assigned`) is the actionable event here: the `if:`
|
||||
# block below checks `github.event.issue.body` / `.title` for
|
||||
# `@claude`, which only changes on edit. Reassignment would fire
|
||||
# the workflow without any matching body/title change.
|
||||
types: [opened, edited]
|
||||
pull_request_review:
|
||||
types: [submitted]
|
||||
|
||||
jobs:
|
||||
claude:
|
||||
# Gate on author_association so only repo owners / members /
|
||||
# collaborators (who already have write access) can trigger this
|
||||
# workflow — required because the job is granted write scopes below.
|
||||
if: |
|
||||
(
|
||||
github.event_name == 'issue_comment' &&
|
||||
contains(github.event.comment.body, '@claude') &&
|
||||
contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association)
|
||||
) ||
|
||||
(
|
||||
github.event_name == 'pull_request_review_comment' &&
|
||||
contains(github.event.comment.body, '@claude') &&
|
||||
contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association)
|
||||
) ||
|
||||
(
|
||||
github.event_name == 'pull_request_review' &&
|
||||
contains(github.event.review.body, '@claude') &&
|
||||
contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.review.author_association)
|
||||
) ||
|
||||
(
|
||||
github.event_name == 'issues' &&
|
||||
(contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')) &&
|
||||
contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.issue.author_association)
|
||||
)
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
issues: write
|
||||
id-token: write
|
||||
actions: read # Required for Claude to read CI results on PRs
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
persist-credentials: false
|
||||
|
||||
- name: Run Claude Code
|
||||
id: claude
|
||||
uses: anthropics/claude-code-action@4481e6d3c7bbb88db2a928ca3444c536f589c7c1 # v1
|
||||
with:
|
||||
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
|
||||
|
||||
# This is an optional setting that allows Claude to read CI results on PRs
|
||||
additional_permissions: |
|
||||
actions: read
|
||||
|
||||
# Tools tailored to this repo's uv-native Python toolchain.
|
||||
# See AGENTS.md for the canonical command reference.
|
||||
#
|
||||
# Both `gh` and `git` are enumerated to safe subcommands only.
|
||||
# Writes that need a commit/branch are handled by the action
|
||||
# itself via the GitHub API (see action.yml `branch_prefix` /
|
||||
# `branch_name_template` inputs) — local `git add/commit/push`
|
||||
# is not in the allowlist, so destructive variants
|
||||
# (`push --force`, `reset --hard`, `branch -D`, `tag -d`,
|
||||
# `checkout -- .`, `clean -fd`) are unreachable.
|
||||
claude_args: |
|
||||
--model claude-opus-4-7
|
||||
--allowedTools "Edit,Write,Read,Bash(uv:*),Bash(make:*),Bash(python:*),Bash(python3:*),Bash(ruff:*),Bash(ty:*),Bash(pytest:*),Bash(gh pr view:*),Bash(gh pr diff:*),Bash(gh pr list:*),Bash(gh pr comment:*),Bash(gh pr edit:*),Bash(gh pr checks:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh issue comment:*),Bash(gh issue edit:*),Bash(gh api repos/*/pulls/*/comments:*),Bash(gh api repos/*/issues/*/comments:*),Bash(gh api repos/*/issues/*/timeline:*),Bash(gh run view:*),Bash(gh workflow view:*),Bash(gh search:*),Bash(git status:*),Bash(git diff:*),Bash(git log:*),Bash(git show:*),Bash(git rev-parse:*),Bash(git ls-files:*),mcp__github_inline_comment__create_inline_comment"
|
||||
@@ -0,0 +1,162 @@
|
||||
name: Code Quality
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
# Read-only lint workflow — no writes back to the repo, no deployments.
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
python-lint:
|
||||
name: Python (ruff + ty)
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: plugins/plugin-eval
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@e58605a9b6da7c637471fab8847a5e5a6b8df081 # v5
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Set up Python
|
||||
run: uv python install
|
||||
|
||||
- name: Sync plugin-eval dev dependencies
|
||||
run: uv sync --all-extras
|
||||
|
||||
- name: ruff check (lint)
|
||||
# Scope: adapter framework + plugin-eval. yt-design-extractor.py is legacy
|
||||
# and out of scope for the multi-harness work; not gated.
|
||||
run: |
|
||||
uv run ruff check \
|
||||
../../tools/adapters/ \
|
||||
../../tools/generate.py \
|
||||
../../tools/validate_generated.py \
|
||||
../../tools/doc_gardener.py \
|
||||
../../tools/tests/ \
|
||||
src/plugin_eval/
|
||||
|
||||
- name: ruff format --check
|
||||
run: |
|
||||
uv run ruff format --check \
|
||||
../../tools/adapters/ \
|
||||
../../tools/generate.py \
|
||||
../../tools/validate_generated.py \
|
||||
../../tools/doc_gardener.py \
|
||||
../../tools/tests/ \
|
||||
src/plugin_eval/
|
||||
|
||||
- name: ty type-check
|
||||
run: |
|
||||
uv run ty check \
|
||||
../../tools/adapters/ \
|
||||
../../tools/generate.py \
|
||||
../../tools/validate_generated.py \
|
||||
../../tools/doc_gardener.py \
|
||||
../../tools/tests/ \
|
||||
src/plugin_eval/
|
||||
|
||||
markdown-lint:
|
||||
name: Markdown (markdownlint-cli2)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: markdownlint
|
||||
# Lints top-level guides (README, AGENTS, ARCHITECTURE, CLAUDE, per-harness
|
||||
# setup, CONTRIBUTING) and our authored docs/. Per-plugin READMEs are owned
|
||||
# by their plugin authors and not lint-gated here — they should be authored
|
||||
# to spec, but lint enforcement happens at the plugin-author layer, not at
|
||||
# the framework PR layer.
|
||||
# Config in .markdownlint.json keeps the ruleset narrow and pragmatic.
|
||||
uses: DavidAnson/markdownlint-cli2-action@eb5ca3ab411449c66620fe7f1b3c9e10547144b0 # v18
|
||||
with:
|
||||
globs: |
|
||||
*.md
|
||||
docs/*.md
|
||||
config: .markdownlint.json
|
||||
|
||||
json-lint:
|
||||
name: JSON / TOML / YAML syntax
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@e58605a9b6da7c637471fab8847a5e5a6b8df081 # v5
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Set up Python
|
||||
run: uv python install 3.12
|
||||
|
||||
- name: Validate every JSON file
|
||||
run: |
|
||||
set -e
|
||||
shopt -s globstar nullglob
|
||||
failed=0
|
||||
for f in **/*.json; do
|
||||
# Skip generated trees and node_modules
|
||||
case "$f" in
|
||||
.codex/*|.cursor/*|.cursor-plugin/*|.opencode/*|node_modules/*|skills/*|agents/*|commands/*) continue ;;
|
||||
esac
|
||||
if ! uv run python -m json.tool "$f" > /dev/null 2>&1; then
|
||||
echo "::error file=$f::invalid JSON"
|
||||
failed=1
|
||||
fi
|
||||
done
|
||||
exit $failed
|
||||
|
||||
- name: Validate every TOML file
|
||||
run: |
|
||||
set -e
|
||||
shopt -s globstar nullglob
|
||||
failed=0
|
||||
for f in **/*.toml; do
|
||||
case "$f" in
|
||||
.codex/*|.cursor/*|.cursor-plugin/*|.opencode/*|node_modules/*|commands/*) continue ;;
|
||||
esac
|
||||
if ! uv run python -c "import tomllib, sys; tomllib.loads(open(sys.argv[1]).read())" "$f" 2>/dev/null; then
|
||||
echo "::error file=$f::invalid TOML"
|
||||
failed=1
|
||||
fi
|
||||
done
|
||||
exit $failed
|
||||
|
||||
- name: Validate every YAML file
|
||||
run: |
|
||||
set -e
|
||||
shopt -s globstar nullglob
|
||||
failed=0
|
||||
for f in **/*.yml **/*.yaml; do
|
||||
case "$f" in
|
||||
.codex/*|.cursor/*|.cursor-plugin/*|.opencode/*|node_modules/*) continue ;;
|
||||
esac
|
||||
# Use safe_load_all to handle multi-document YAML (e.g. Kubernetes manifests
|
||||
# with `---` document separators — valid YAML, but safe_load only reads the
|
||||
# first doc and would mis-flag the file as invalid).
|
||||
if ! uv run --with pyyaml python -c "import yaml, sys; list(yaml.safe_load_all(open(sys.argv[1]).read()))" "$f" 2>/dev/null; then
|
||||
echo "::error file=$f::invalid YAML"
|
||||
failed=1
|
||||
fi
|
||||
done
|
||||
exit $failed
|
||||
@@ -0,0 +1,73 @@
|
||||
name: Plugin Eval Report
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
depth:
|
||||
description: 'Evaluation depth (quick = static only, standard = +LLM judge, deep = +Monte Carlo)'
|
||||
required: true
|
||||
default: 'quick'
|
||||
type: choice
|
||||
options:
|
||||
- quick
|
||||
- standard
|
||||
- deep
|
||||
only_changed:
|
||||
description: 'Comma-separated plugin names to evaluate (blank = all)'
|
||||
required: false
|
||||
default: ''
|
||||
type: string
|
||||
schedule:
|
||||
# Weekly full static sweep, Mondays at 06:00 UTC
|
||||
- cron: '0 6 * * 1'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
eval:
|
||||
name: Evaluate plugins (${{ inputs.depth || 'quick' }})
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 180
|
||||
env:
|
||||
DEPTH: ${{ inputs.depth || 'quick' }}
|
||||
ONLY_CHANGED: ${{ inputs.only_changed || '' }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@v5
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Sync plugin-eval dependencies
|
||||
working-directory: plugins/plugin-eval
|
||||
run: uv sync --all-extras
|
||||
|
||||
- name: Run eval sweep
|
||||
working-directory: plugins/plugin-eval
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
run: |
|
||||
args=( --depth "$DEPTH" --output-dir "$GITHUB_WORKSPACE/eval-reports" --concurrency 4 )
|
||||
if [ -n "$ONLY_CHANGED" ]; then
|
||||
args+=( --only-changed "$ONLY_CHANGED" )
|
||||
fi
|
||||
uv run python scripts/eval_all.py "${args[@]}"
|
||||
|
||||
- name: Post report to job summary
|
||||
if: always()
|
||||
run: |
|
||||
if [ -f eval-reports/summary.md ]; then
|
||||
cat eval-reports/summary.md >> "$GITHUB_STEP_SUMMARY"
|
||||
else
|
||||
echo "No summary produced." >> "$GITHUB_STEP_SUMMARY"
|
||||
fi
|
||||
|
||||
- name: Upload reports artifact
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: eval-reports-${{ env.DEPTH }}-${{ github.run_id }}
|
||||
path: eval-reports/
|
||||
retention-days: 30
|
||||
@@ -0,0 +1,272 @@
|
||||
name: Validate
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [main]
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
validate-json:
|
||||
name: Validate JSON files
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Validate marketplace.json
|
||||
run: python3 -m json.tool .claude-plugin/marketplace.json > /dev/null
|
||||
|
||||
- name: Validate every plugin.json
|
||||
run: |
|
||||
set -e
|
||||
shopt -s globstar nullglob
|
||||
failed=0
|
||||
for f in plugins/*/.claude-plugin/plugin.json; do
|
||||
if ! python3 -m json.tool "$f" > /dev/null 2>&1; then
|
||||
echo "::error file=$f::invalid JSON"
|
||||
failed=1
|
||||
fi
|
||||
done
|
||||
exit $failed
|
||||
|
||||
- name: Validate hooks.json files
|
||||
run: |
|
||||
set -e
|
||||
shopt -s globstar nullglob
|
||||
failed=0
|
||||
for f in plugins/*/hooks/*.json; do
|
||||
if ! python3 -m json.tool "$f" > /dev/null 2>&1; then
|
||||
echo "::error file=$f::invalid JSON"
|
||||
failed=1
|
||||
fi
|
||||
done
|
||||
exit $failed
|
||||
|
||||
- name: Validate marketplace entries resolve to plugin dirs
|
||||
run: |
|
||||
python3 - <<'PY'
|
||||
import json, os, posixpath, sys
|
||||
from urllib.parse import urlparse
|
||||
with open('.claude-plugin/marketplace.json') as f:
|
||||
mp = json.load(f)
|
||||
errors = []
|
||||
for p in mp.get('plugins', []):
|
||||
src = p.get('source')
|
||||
if isinstance(src, str) and src.startswith('./plugins/'):
|
||||
path = src.lstrip('./')
|
||||
if not os.path.isdir(path):
|
||||
errors.append(f"{p['name']}: source {src} does not exist")
|
||||
elif not os.path.isfile(os.path.join(path, '.claude-plugin', 'plugin.json')):
|
||||
errors.append(f"{p['name']}: missing .claude-plugin/plugin.json in {src}")
|
||||
elif isinstance(src, dict):
|
||||
if src.get('source') != 'git-subdir':
|
||||
errors.append(f"{p['name']}: unsupported source object {src.get('source')!r}")
|
||||
continue
|
||||
|
||||
extra_keys = set(src) - {'source', 'url', 'path'}
|
||||
if extra_keys:
|
||||
errors.append(f"{p['name']}: git-subdir entry has unsupported keys: {sorted(extra_keys)}")
|
||||
|
||||
url = src.get('url')
|
||||
path = src.get('path')
|
||||
|
||||
if not url:
|
||||
errors.append(f"{p['name']}: git-subdir entry missing url")
|
||||
else:
|
||||
parsed = urlparse(url)
|
||||
if parsed.scheme != 'https' or parsed.netloc != 'github.com' or not parsed.path.endswith('.git'):
|
||||
errors.append(f"{p['name']}: git-subdir url must be an https://github.com/*.git URL")
|
||||
|
||||
if not path:
|
||||
errors.append(f"{p['name']}: git-subdir entry missing path")
|
||||
elif path != '.':
|
||||
normalized = posixpath.normpath(path)
|
||||
if (
|
||||
path.startswith('/')
|
||||
or '\\' in path
|
||||
or normalized != path
|
||||
or normalized == '..'
|
||||
or normalized.startswith('../')
|
||||
):
|
||||
errors.append(f"{p['name']}: git-subdir path must be a normalized relative path")
|
||||
if errors:
|
||||
for e in errors:
|
||||
print(f"::error::{e}")
|
||||
sys.exit(1)
|
||||
print(f"OK: {len(mp.get('plugins', []))} marketplace entries validated")
|
||||
PY
|
||||
|
||||
- name: Check agent name uniqueness
|
||||
run: python3 tools/check_agent_name_collisions.py --fail-on-duplicates
|
||||
|
||||
plugin-eval-tests:
|
||||
name: plugin-eval pytest
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: plugins/plugin-eval
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@e58605a9b6da7c637471fab8847a5e5a6b8df081 # v5
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Set up Python
|
||||
run: uv python install
|
||||
|
||||
- name: Sync dependencies
|
||||
run: uv sync --all-extras
|
||||
|
||||
- name: Run tests
|
||||
run: uv run pytest
|
||||
|
||||
tools-tests:
|
||||
name: tools pytest (adapters + validators + gardener)
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: plugins/plugin-eval
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@e58605a9b6da7c637471fab8847a5e5a6b8df081 # v5
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Set up Python
|
||||
run: uv python install
|
||||
|
||||
- name: Sync dependencies
|
||||
run: uv sync --all-extras
|
||||
|
||||
- name: Run tools test suite
|
||||
run: uv run pytest -q ../../tools/tests/
|
||||
|
||||
multi-harness-generate:
|
||||
name: Cross-harness generation + validation
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@e58605a9b6da7c637471fab8847a5e5a6b8df081 # v5
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Set up Python
|
||||
run: uv python install 3.12
|
||||
|
||||
- name: Sync plugin-eval venv (provides pyyaml + adapter imports)
|
||||
working-directory: plugins/plugin-eval
|
||||
run: uv sync --all-extras
|
||||
|
||||
- name: Generate all harness artifacts
|
||||
run: make generate-all
|
||||
|
||||
- name: Verify committed artifacts are in sync with sources
|
||||
# Native-install artifacts are committed so the repo installs from a clone.
|
||||
# Regeneration must produce no diff — if it does, a source change was committed
|
||||
# without running `make generate-all`. Run it locally and commit the result.
|
||||
run: |
|
||||
if [ -n "$(git status --porcelain)" ]; then
|
||||
echo "::error::Generated artifacts drifted from the committed tree. Run 'make generate-all' and commit the result."
|
||||
git status --short
|
||||
git --no-pager diff --stat
|
||||
exit 1
|
||||
fi
|
||||
echo "OK: committed harness artifacts match a fresh 'make generate-all'."
|
||||
|
||||
- name: Structural validation (strict)
|
||||
run: make validate STRICT=1
|
||||
|
||||
- name: Doc-gardener (no errors allowed)
|
||||
# Errors fail this step; warnings (e.g. oversized source skills with no
|
||||
# references/) are surfaced but don't block. Use STRICT=1 to gate on warnings too.
|
||||
run: make garden
|
||||
|
||||
- name: Upload generated artifacts for inspection
|
||||
if: always()
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
|
||||
with:
|
||||
name: multi-harness-output
|
||||
path: |
|
||||
.codex/
|
||||
.cursor/
|
||||
.cursor-plugin/
|
||||
.opencode/
|
||||
opencode.json
|
||||
commands/
|
||||
agents/
|
||||
skills/
|
||||
AGENTS.md
|
||||
retention-days: 7
|
||||
|
||||
cli-smoke-test:
|
||||
name: Real-CLI smoke test (OpenCode + Gemini)
|
||||
runs-on: ubuntu-latest
|
||||
# Real-CLI subprocess tests: invokes the actual harness binaries against our
|
||||
# generated artifacts to catch issues that pure-Python parsing misses (CLI
|
||||
# version drift, schema-loader surprises, plugin discovery bugs).
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
|
||||
with:
|
||||
persist-credentials: false
|
||||
|
||||
- name: Set up Node.js (for Gemini CLI)
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Set up Bun (for OpenCode CLI)
|
||||
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
|
||||
with:
|
||||
bun-version: latest
|
||||
|
||||
- name: Install OpenCode CLI
|
||||
run: |
|
||||
curl -fsSL https://opencode.ai/install | bash
|
||||
echo "$HOME/.opencode/bin" >> "$GITHUB_PATH"
|
||||
|
||||
- name: Install Gemini CLI
|
||||
# Pinned to the released line we developed against; bump alongside any
|
||||
# gemini-extension.json schema changes.
|
||||
run: npm install -g @google/gemini-cli@latest
|
||||
|
||||
- name: Verify CLI versions
|
||||
run: |
|
||||
opencode --version
|
||||
gemini --version
|
||||
|
||||
- name: Install uv
|
||||
uses: astral-sh/setup-uv@e58605a9b6da7c637471fab8847a5e5a6b8df081 # v5
|
||||
with:
|
||||
enable-cache: true
|
||||
|
||||
- name: Set up Python
|
||||
run: uv python install 3.12
|
||||
|
||||
- name: Sync plugin-eval dependencies
|
||||
working-directory: plugins/plugin-eval
|
||||
run: uv sync --all-extras
|
||||
|
||||
- name: Generate all harness artifacts
|
||||
run: make generate-all
|
||||
|
||||
- name: Run real-CLI smoke tests
|
||||
working-directory: plugins/plugin-eval
|
||||
run: uv run pytest -v ../../tools/tests/test_cli_smoke.py
|
||||
Reference in New Issue
Block a user