chore: import upstream snapshot with attribution
CI / lint (push) Waiting to run
CI / mypy (push) Waiting to run
CI / docs (push) Waiting to run
CI / test on 3.10 (standard) (push) Waiting to run
CI / test on 3.11 (standard) (push) Waiting to run
CI / test on 3.12 (standard) (push) Waiting to run
CI / test on 3.10 (all-extras) (push) Waiting to run
CI / test on 3.11 (all-extras) (push) Waiting to run
CI / test on 3.12 (all-extras) (push) Waiting to run
CI / test on 3.13 (all-extras) (push) Waiting to run
CI / test on 3.14 (pydantic-evals) (push) Waiting to run
Harness Compat / harness compat (push) Waiting to run
CI / test on 3.13 (standard) (push) Waiting to run
CI / test on 3.14 (standard) (push) Waiting to run
CI / test on 3.14 (all-extras) (push) Waiting to run
CI / test on 3.10 (pydantic-ai-slim) (push) Waiting to run
CI / test on 3.11 (pydantic-ai-slim) (push) Waiting to run
CI / test on 3.12 (pydantic-ai-slim) (push) Waiting to run
CI / test on 3.13 (pydantic-ai-slim) (push) Waiting to run
CI / test on 3.14 (pydantic-ai-slim) (push) Waiting to run
CI / test on 3.10 (pydantic-evals) (push) Waiting to run
CI / test on 3.11 (pydantic-evals) (push) Waiting to run
CI / test on 3.12 (pydantic-evals) (push) Waiting to run
CI / test on 3.13 (pydantic-evals) (push) Waiting to run
CI / test on 3.10 (lowest-versions) (push) Waiting to run
CI / test on 3.11 (lowest-versions) (push) Waiting to run
CI / test on 3.12 (lowest-versions) (push) Waiting to run
CI / test on 3.13 (lowest-versions) (push) Waiting to run
CI / test on 3.14 (lowest-versions) (push) Waiting to run
CI / test examples on 3.11 (push) Waiting to run
CI / test examples on 3.12 (push) Waiting to run
CI / test examples on 3.13 (push) Waiting to run
CI / test examples on 3.14 (push) Waiting to run
CI / coverage (push) Blocked by required conditions
CI / check (push) Blocked by required conditions
CI / deploy-docs (push) Blocked by required conditions
CI / deploy-docs-preview (push) Blocked by required conditions
CI / build release artifacts (push) Blocked by required conditions
CI / publish to PyPI (push) Blocked by required conditions
CI / Send tweet (push) Blocked by required conditions

This commit is contained in:
wehub-resource-sync
2026-07-13 13:27:52 +08:00
commit 9201ef759e
2096 changed files with 1232387 additions and 0 deletions
+201
View File
@@ -0,0 +1,201 @@
---
name: complete-partial-pr
description: Evaluate and complete an issue or PR where the submitted patch fixes only a narrow symptom of the reported pain point. Use when a contribution may miss adjacent integration surfaces, provider/spec semantics, roundtrip behavior, tests, docs, or historical maintainer decisions.
user-invocable: true
allowed-tools: Bash(git:*), Bash(gh:*), Bash(jq:*), Bash(rg:*), Bash(sed:*), Bash(ls:*), Bash(cat:*), Bash(mkdir:*), Bash(date:*), Bash(uv:*), Read, Write, Edit, Glob, Grep, WebFetch, AskUserQuestion, Agent
---
# Complete Partial PR
Use this when a PR or issue patch fixes a small visible failure but may not address the full integration contract. The goal is to turn a narrow contribution into a maintainable Pydantic AI change, or to explain precisely why it should stay narrow.
This is not limited to UI or provider integrations. Apply it to any patch that touches one variant of a broader surface: parse/dump, request/response, streaming/non-streaming, static/dynamic, sync/async, native/provider/local tools, metadata, state machines, durable execution, docs, or tests.
## When To Use
- A contributor's PR addresses the immediate error but not the rest of the user's workflow.
- A fix accepts one shape of data but may not preserve roundtrip semantics.
- A provider or external protocol has more states, fields, or variants than the PR covers.
- The user asks "is this enough?", "what did the contributor miss?", "verify this against the spec", or "improve this branch".
- A reviewer suspects the patch contradicts historical decisions or creates future integration debt.
Do not use this as a replacement for `/review-branch` when the task is only a general code review. If the PR has no local context yet, run `/adopt-pr` first.
## Operating Principle
Separate three things before implementing:
1. The contributor's exact patch.
2. The underlying user pain point.
3. The integration contract Pydantic AI should support.
Only (3) determines the final shape. The submitted patch is evidence, not the boundary.
## Delegation
Use subagents for independent research and review lanes. Keep the critical path local: branch selection, final synthesis, implementation, and push decisions.
Good subagent lanes:
- **Spec researcher**: read the linked issue/PR, provider docs, SDK types, protocol docs, and relevant project docs. Return source-backed facts only, with URLs or file paths.
- **Codepath mapper**: map affected code and adjacent surfaces: loaders, dumpers, stream handlers, request builders, response parsers, tool/native-tool paths, model profiles, docs, and tests.
- **History researcher**: inspect `git blame`, `git log`, previous PRs, review comments, and decision logs around the touched code. Return historical decisions with source links.
- **Test-shape reviewer**: decide what coverage proves the full contract without bloating tests. Identify where parametrization, snapshots, VCR, or direct adapter tests are appropriate.
Give each subagent a narrow question, known PR/issue number, relevant file paths, and the exact output you need. Do not ask multiple subagents to answer the same question. Claims that answer the task need a source.
## Workflow
### 1. Startup
Read the local context before scoping:
- `CLAUDE.md` and `CLAUDE.local.md`
- `agent_docs/index.md`
- `.claude/skills/branch-context/issue-brief.md` and `.claude/skills/branch-context/pr-decisions.md`, if present
- `.claude/skills/pyai-knowledge/feature-map.md` for the affected feature group
- `.claude/skills/pyai-knowledge/internals-model.md` for layer ownership
- `.claude/skills/pyai-knowledge/maintainer-mindset.md` for review tells
If the investigation spans multiple surfaces or subagents, create a short working note under `local-notes/`, for example `local-notes/complete-partial-pr.md`. Keep facts, sources, and open questions there. Do not put research prose in `issue-brief.md`.
### 2. Resolve The Work Item
Use `gh` for GitHub context.
```bash
gh pr view <PR> --json number,title,url,state,body,author,headRepository,headRepositoryOwner,headRefName,maintainerCanModify,baseRefName,comments,reviews,closingIssuesReferences
```
Read linked issues, PR comments, review threads, and CI context. If the work item is an issue with no PR, inspect the linked branch or proposed patch if one exists.
For fork PRs, record:
- `headRepository.nameWithOwner`
- `headRefName`
- `maintainerCanModify`
- whether a plain `git push` targets the actual PR branch
Never force push. If updating a contributor PR and `maintainerCanModify` allows it, push to the contributor's PR branch. If not, create a separate branch and report the limitation.
### 3. Reconstruct The Pain Point
Write a short local working note, even if it only lives in the final report:
- What user workflow failed?
- Which exact symptom does the PR fix?
- Which broader contract might users reasonably expect?
- Which variants or states are implied by the same contract?
- What would be a deliberate non-goal?
This note keeps the implementation from being scoped by the first test the contributor happened to write.
### 4. Verify The Spec
Find authoritative sources before changing code:
- Provider docs, SDK types, protocol references, or API schemas.
- Pydantic AI docs and public API contracts.
- Existing tests and snapshots that encode current behavior.
For external technical specs, use primary sources. If the spec is versioned, note the version used by Pydantic AI and whether the PR targets the same version. Distinguish confirmed spec behavior from inference.
### 5. Map The Integration Surface
Search for symmetric and adjacent code paths. Typical pairs and variants:
- load / dump
- parse / serialize
- request / response
- streaming / non-streaming
- initial / intermediate / final states
- success / error / denied / unavailable
- static / dynamic
- function tools / native tools / provider-executed tools
- IDs, provider names, metadata, and roundtrip preservation
- sync / async
- public API / internal adapter / docs / examples / skills
Do not assume a one-line parser fix is sufficient until this map is complete.
### 6. Check History
Use local git history and GitHub history around the touched code:
```bash
git blame -- <path>
git log --oneline -- <path>
gh pr list --search '<symbol-or-file> repo:pydantic/pydantic-ai' --state all --json number,title,url,state
```
Read previous PRs or comments that introduced the relevant behavior. Capture any decision that constrains the new change. If the new approach contradicts history, call that out explicitly before implementing.
### 7. Decide The Correct Scope
Classify missing work:
- **Required**: needed for spec compliance, roundtrip correctness, backwards compatibility, or the reported user workflow.
- **Regression test**: behavior that should remain deliberately unsupported or should not silently regress.
- **Nice to have**: adjacent polish that is not required for this PR.
- **Out of scope**: broader feature work that deserves a separate issue or design discussion.
Push back on expanding the PR only when the broader behavior is genuinely a separate product decision. Otherwise, complete the contract.
If choosing broader scope or explicitly deferring an adjacent surface, append a concise entry to `.claude/skills/branch-context/pr-decisions.md` with the source that justified the decision.
If the broader fix changes public API, provider semantics, durable behavior, or safety posture in a non-obvious way, ask David before coding or draft a PR comment/proposal instead of silently expanding the branch.
### 8. Implement Conservatively
Prefer small changes that fit existing abstractions. Preserve backwards compatibility and avoid new public API unless the full contract requires it.
For tests:
- Cover the behavior at the same level users rely on it.
- Prefer integration/adapter-level tests over clusters of helper tests.
- Parametrize related variants and states to reduce bloat.
- Use VCR only when the real provider interaction is the behavior under test. Local protocol conversion usually belongs in direct tests.
- Include regression tests for deliberate non-support decisions.
Run targeted checks first, for example:
```bash
uv run pytest tests/<target>.py::<test_name>
PYRIGHT_PYTHON_IGNORE_WARNINGS=1 uv run pyright <changed-file.py>
```
Escalate to broader checks only when the blast radius warrants it.
After code changes, run `/review-branch` unless the change is documentation-only or the user explicitly asks for a narrower pass.
### 9. Communicate And Push
Before pushing, verify the remote target:
```bash
git remote -v
git branch -vv
git status --short --branch
```
If posting a GitHub comment, start with `David's AICA here: `. Summarize:
- What the original PR covered.
- What additional integration surfaces were checked.
- What improvements were added.
- Which spec and historical sources were used.
- Which tests were run.
If the work changes durable project knowledge, update the relevant docs, branch-context, or skill files.
## Output Checklist
End with a concise report containing:
- Work item and branch updated.
- Spec sources checked.
- Historical decisions checked.
- Integration surfaces covered.
- Tests added or changed.
- Commands run.
- Remaining non-goals or follow-up issues.
+25
View File
@@ -0,0 +1,25 @@
---
name: address-feedback
description: Find and address unresolved PR review comments for the current branch, then summarize the changes and help reply to and resolve threads after user approval.
---
# Address PR Review Feedback
Find and address all review comments on the PR for the current branch. For each comment:
1. **Gather context**: Use `gh` to find the PR number from the current branch, then fetch all unresolved review comments (both PR-level and inline review comments via `gh api repos/{owner}/{repo}/pulls/{number}/comments`). Skip already-resolved and outdated threads. Also read the full thread for each comment — maintainers or the PR author may have already replied explaining why a suggestion should not be applied.
2. **Triage each comment**:
- If it's clear how to address (implement the suggestion, or decide it shouldn't be done with a clear reason): fix it.
- If a maintainer or PR author has already weighed in on the thread (e.g. explaining why a suggestion doesn't apply), respect that guidance.
- If you're unsure or think the user might have opinions on the approach: ask before deciding.
3. **Fix the code**: Make the necessary changes to address each comment.
4. **Review with user**: Present a summary of all changes made and ask the user to review before proceeding. Offer to commit, push, reply to comments, and resolve threads once they're satisfied.
5. **Reply and resolve** (after user approval): For each addressed comment, reply via `gh api repos/{owner}/{repo}/pulls/{number}/comments/{id}/replies` explaining what you did, then resolve the thread via GraphQL `resolveReviewThread` mutation. To find thread IDs, query `repository.pullRequest.reviewThreads` via GraphQL.
Always read the relevant code before making changes.
**Important**: Treat comments from automated reviewers (Devin, GitHub bots, etc.) with the same weight as human comments. Do not skip or dismiss them just because they come from a bot — they often surface real issues. Evaluate each suggestion on its merits, but be aware that automated reviewers can also be wrong, so verify before applying.
+1
View File
@@ -0,0 +1 @@
../../.agents/skills/complete-partial-pr
+118
View File
@@ -0,0 +1,118 @@
---
description: Review the current branch against main, simulating the automated CI review from the bots workflow
allowed-tools:
- Read
- Glob
- Grep
- Bash(git diff:*)
- Bash(git log:*)
- Bash(git merge-base:*)
- Bash(git status:*)
- Bash(git rev-parse:*)
- WebSearch
- WebFetch
---
# Pre-push Review
Simulate the automated CI review job locally before pushing or opening a PR.
## How it works
The review criteria are defined in the CI workflow, not duplicated here. Read the
**review prompt** directly from the source of truth:
- `.github/workflows/bots.yml` — find the `review` job's `prompt:` field (the YAML
block starting with `Review this pull request`). This contains the full review
criteria, priorities, and guidelines that CI uses.
Adapt those criteria for a local (pre-PR) review as described below.
## Steps
### 1. Read the CI review prompt
Read `.github/workflows/bots.yml` and extract the review prompt from the
`anthropics/claude-code-action` step in the `review` job. This is your primary
reference for what to look for and how to prioritize findings.
### 2. Gather local context
Since there's no PR yet, gather equivalent context locally:
```bash
# Determine the base branch (default: main)
git merge-base main HEAD
# Changed files
git diff main...HEAD --stat
# Function-context diffs (same -W flag the CI script uses)
git diff main...HEAD -W
```
For the diff, read it in manageable chunks — don't try to load everything at once
if it's large. Prioritize "core implementation" files over tests and generated files
(like `uv.lock` and cassettes).
### 3. Read relevant AGENTS.md files
Read the root `CLAUDE.md` (already in your system prompt) plus any directory-specific
`AGENTS.md` files for directories that contain changed files. The CI script checks
these paths:
- `docs/AGENTS.md`
- `pydantic_ai_slim/pydantic_ai/models/AGENTS.md`
- `tests/AGENTS.md`
Only read ones relevant to the changed directories.
### 4. Review
Apply the review criteria from the CI prompt, with these local adaptations:
- **Skip PR-specific checks**: no PR description, linked issues, duplicate PR checks,
or prior review comments to consider.
- **Skip "should this PR exist" check**: assume the user intends to make these changes.
- **Output findings as text** instead of posting inline comments.
Focus areas (in priority order, per the CI prompt):
1. **Public API** — abstractions, class hierarchy, method signatures, type safety, backward compat
2. **Concepts & behavior** — design decisions, tradeoffs needing discussion
3. **Documentation** — voice, patterns, completeness
4. **Tests** — coverage, style, integration vs unit
5. **Code style** — AGENTS.md rule compliance, idiomatic Python
If there are high-level problems that would require significant rework, focus on those
and hold off on lower-level nits.
### 5. Present findings
Organize findings by priority tier. For each finding:
- Reference the specific file and line (`file:line`)
- Explain the issue concisely
- Suggest a fix if appropriate
If there are no findings, say so.
## Comment quality
Every finding must earn its place. Your review should never add noise:
- **Actionable only**: each finding must request a specific change, flag a concern that
needs discussion, or suggest a concrete improvement. Do not comment on positive aspects
("excellent design", "good choice") — those are noise.
- **Concise**: 1-3 sentences per finding is almost always enough. No unnecessary lists,
subheadings, or emojis.
- **Non-repetitive**: don't flag the same issue multiple times unless it appears in
meaningfully different contexts.
- **No filler**: do not pad the review with observations that don't require action.
If a choice is correct, don't mention it. If code follows the project's patterns,
don't praise it. Focus exclusively on what needs to change or be discussed.
- **Friendly but not sycophantic**: use the tone of a helpful project maintainer.
No compliments, no "great work", just clear, direct feedback.
If there are high-level problems that would require significant rework, focus on those
and skip lower-level nits entirely — they'll need to be revisited anyway.
+97
View File
@@ -0,0 +1,97 @@
---
name: testing-skill
description: Record, rewrite, and debug VCR cassettes for HTTP recordings. Use when running tests with --record-mode, verifying cassette playback, or inspecting request/response bodies in YAML cassettes.
allowed-tools: Bash(uv run pytest *), Bash(uv run python .claude/skills/testing-skill/parse_cassette.py *), Bash(source .env && uv run pytest *), Bash(git diff *)
---
# Pytest VCR Workflow
Use this skill when recording or re-recording VCR cassettes for tests, or when debugging cassette contents.
## Prerequisites
- Verify `.env` exists: `test -f .env && echo 'ok' || echo 'missing'`
- Missing API keys will cause clear test errors at runtime
## Important flags
- `--record-mode=rewrite` : Record cassettes (works for both new and existing)
- `--lf` : Run only the last failed tests
- `-vv` : Verbose output
- `--tb=line` : Short traceback output
- `-k=""` : Run tests matching the given substring expression
## Recording Cassettes
### Step 1: Record cassettes
```bash
source .env && uv run pytest path/to/test.py::test_function_name -v --tb=line --record-mode=rewrite
```
Multiple tests can be specified:
```bash
source .env && uv run pytest path/to/test.py::test_one path/to/test.py::test_two -v --tb=line --record-mode=rewrite
```
### Step 2: Verify recordings
Run the same tests WITHOUT `--record-mode` to verify cassettes play back correctly:
```bash
source .env && uv run pytest path/to/test.py::test_function_name -vv --tb=line
```
### Step 3: Review snapshots
If tests use [`snapshot()`](https://github.com/15r10nk/inline-snapshot) assertions:
- The test run in Step 2 auto-fills snapshot content
- Review the generated snapshot files to ensure they match expected output
- You only review - don't manually write snapshot contents
- Snapshots capture what the test actually produced, additional to explicit assertions
## Parsing Cassettes
Parse VCR cassette YAML files to inspect request/response bodies without dealing with raw YAML.
### Usage
```bash
uv run python .claude/skills/testing-skill/parse_cassette.py <cassette_path> [--interaction N]
```
### Examples
```bash
# Parse all interactions in a cassette
uv run python .claude/skills/testing-skill/parse_cassette.py tests/models/cassettes/test_foo/test_bar.yaml
# Parse only interaction 1 (0-indexed)
uv run python .claude/skills/testing-skill/parse_cassette.py tests/models/cassettes/test_foo/test_bar.yaml --interaction 1
```
### Output
For each interaction, shows:
- Request: method, URI, parsed body (truncated base64)
- Response: status code, parsed body (truncated base64)
Base64 strings longer than 100 chars are truncated for readability.
## Full Workflow Example
```bash
# 1. Record cassette
source .env && uv run pytest tests/models/test_openai.py::test_chat_completion -v --tb=line --record-mode=rewrite
# 2. Verify playback and fill snapshots
source .env && uv run pytest tests/models/test_openai.py::test_chat_completion -vv --tb=line
# 3. Review test code diffs (excludes cassettes)
git diff tests/ -- ':!**/cassettes/**'
# 4. List new/changed cassettes (name only - use parse_cassette.py to inspect)
git diff --name-only tests/ -- '**/cassettes/**'
# 5. Inspect cassette contents if needed
uv run python .claude/skills/testing-skill/parse_cassette.py tests/models/cassettes/test_openai/test_chat_completion.yaml
```
+104
View File
@@ -0,0 +1,104 @@
#!/usr/bin/env python3
"""Parse VCR cassette files and pretty-print request/response bodies."""
import argparse
import json
import re
import sys
from pathlib import Path
import yaml
def truncate_base64(obj: object, max_len: int = 100) -> object:
"""Recursively truncate base64-like strings in nested structures."""
if isinstance(obj, str):
if len(obj) > max_len and re.match(r'^[A-Za-z0-9+/=]+$', obj[:100]):
return f'{obj[:50]}...[truncated {len(obj)} chars]...{obj[-20:]}'
if obj.startswith('data:') and len(obj) > max_len:
return f'{obj[:80]}...[truncated {len(obj)} chars]'
return obj
elif isinstance(obj, dict):
return {k: truncate_base64(v, max_len) for k, v in obj.items()}
elif isinstance(obj, list):
return [truncate_base64(item, max_len) for item in obj]
return obj
def _extract_body(part: dict[str, object]) -> object | None:
"""Extract body from a request/response, trying parsed_body first, then standard VCR body.string."""
if 'parsed_body' in part:
return part['parsed_body']
body = part.get('body')
if isinstance(body, dict):
body_str = body.get('string')
if isinstance(body_str, str) and body_str:
try:
return json.loads(body_str)
except json.JSONDecodeError:
return body_str
elif isinstance(body, str) and body:
try:
return json.loads(body)
except json.JSONDecodeError:
return body
return None
def parse_cassette(path: Path, interaction_idx: int | None = None) -> None:
"""Parse and print cassette contents."""
with open(path) as f:
data = yaml.safe_load(f)
interactions = data.get('interactions', [])
if not interactions:
print('No interactions found in cassette')
return
indices = [interaction_idx] if interaction_idx is not None else range(len(interactions))
for i in indices:
if i < 0 or i >= len(interactions):
print(f'Interaction {i} not found (only {len(interactions)} interactions)')
continue
interaction = interactions[i]
req = interaction.get('request', {})
resp = interaction.get('response', {})
print(f'\n{"="*60}')
print(f'INTERACTION {i}')
print('='*60)
print(f'\n--- REQUEST ---')
print(f'Method: {req.get("method", "N/A")}')
print(f'URI: {req.get("uri", "N/A")}')
req_body = _extract_body(req)
if req_body is not None:
truncated = truncate_base64(req_body)
print(f'Body:\n{json.dumps(truncated, indent=2)}')
print(f'\n--- RESPONSE ---')
status = resp.get('status', {})
print(f'Status: {status.get("code", "N/A")} {status.get("message", "")}')
resp_body = _extract_body(resp)
if resp_body is not None:
truncated = truncate_base64(resp_body)
print(f'Body:\n{json.dumps(truncated, indent=2)}')
def main() -> None:
parser = argparse.ArgumentParser(description='Parse VCR cassette files')
parser.add_argument('cassette', type=Path, help='Path to cassette YAML file')
parser.add_argument('--interaction', '-i', type=int, help='Specific interaction index (0-based)')
args = parser.parse_args()
if not args.cassette.exists():
print(f'File not found: {args.cassette}', file=sys.stderr)
sys.exit(1)
parse_cassette(args.cassette, args.interaction)
if __name__ == '__main__':
main()
+2
View File
@@ -0,0 +1,2 @@
code_review:
disable: true
+1
View File
@@ -0,0 +1 @@
.github/workflows/*.lock.yml linguist-generated=true merge=ours
+72
View File
@@ -0,0 +1,72 @@
name: 🐛 Pydantic AI Bug
description: Report a bug or unexpected behavior in Pydantic AI
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thank you for contributing to Pydantic AI! ✊
Please provide as much information as possible to help us reproduce and fix your issue quickly.
- type: checkboxes
id: checks
attributes:
label: Initial Checks
description: Just making sure we're on the same page.
options:
- label: I'm using the [latest version](https://github.com/pydantic/pydantic-ai/releases/latest) of Pydantic AI
required: true
- label: I've searched for my issue in [the issue tracker](https://github.com/pydantic/pydantic-ai/issues) before opening this issue
required: true
- type: textarea
id: description
attributes:
label: Description
description: |
Please explain what you're seeing and what you would expect to see.
- **A line of code is worth a thousand words**: you can include minimal, reproducible example code below.
- **A trace is worth a thousand log lines**: you can link to a [Pydantic Logfire](https://ai.pydantic.dev/logfire) trace below.
validations:
required: true
- type: textarea
id: example
attributes:
label: Minimal, Reproducible Example
description: >
If at all possible, please add a self-contained,
[minimal, reproducible example](https://stackoverflow.com/help/minimal-reproducible-example)
that demonstrates the bug.
(This will be automatically formatted as code, so no need for backticks.)
placeholder: |
from pydantic_ai import Agent
...
render: Python
- type: input
id: logfire_trace
attributes:
label: Logfire Trace
description: >
If at all possible, please capture the agent run or model request leading up to the bug or error using [Pydantic Logfire](https://ai.pydantic.dev/logfire)
and share a link to the [public trace](https://logfire.pydantic.dev/docs/guides/web-ui/public-traces/) here.
- type: textarea
id: version
attributes:
label: Python, Pydantic AI & LLM client version
description: |
Which version of Python, Pydantic AI, and LLM provider SDK are you using?
value: |
- Python:
- Pydantic AI:
- LLM provider SDK:
validations:
required: true
+5
View File
@@ -0,0 +1,5 @@
blank_issues_enabled: false
contact_links:
- name: 💬 Join Slack
url: 'https://logfire.pydantic.dev/docs/join-slack/'
about: Join our Slack community to ask questions, get help, and chat about Pydantic AI
@@ -0,0 +1,29 @@
name: 🚀 Pydantic AI Feature Request
description: "Suggest a new feature for Pydantic AI"
labels: ["feature"]
body:
- type: markdown
attributes:
value: Thank you for contributing to Pydantic AI! ✊
- type: textarea
id: description
attributes:
label: Description
description: |
Please give as much detail as possible about the feature you would like to suggest. 🙏
You might like to add:
* A demo of how code might look when using the feature
* Your use case(s) for the feature
* Reference to other projects that have a similar feature
validations:
required: true
- type: textarea
id: references
attributes:
label: References
description: |
Please add any links or references that might help us understand your feature request better. 📚
+37
View File
@@ -0,0 +1,37 @@
name: ❓ Pydantic AI Question
description: "Ask a question about Pydantic AI"
labels: ["question"]
body:
- type: markdown
attributes:
value: |
Thank you for reaching out to the Pydantic AI community! We're here to help! 🤝
The quickest way to get help or suggestions from our community or team is via our [Slack community](https://logfire.pydantic.dev/docs/join-slack/),
but if you have a little more patience or a lot of context or code to share, GitHub is a great place too.
- type: textarea
id: question
attributes:
label: Question
description: |
Please provide as much detail as possible about your question. 🙏
You might like to include:
* Code snippets showing what you've tried
* Error messages you're encountering (if any)
* Expected vs actual behavior
* Your use case and what you're trying to achieve
validations:
required: true
- type: textarea
id: context
attributes:
label: Additional Context
description: |
Please provide any additional context that might help us better understand your question, such as:
* Your Pydantic AI version
* Your Python version
* Relevant configuration or environment details 📝
@@ -0,0 +1,85 @@
name: Fetch dynamic prompt
description: >-
Resolve an agentic-workflow prompt from a Logfire managed variable, falling
back to a committed default prompt file when the variable is unset or
unreachable. The resolved value is the COMPLETE prompt (never a fragment).
Shared by the pydantic-ai-* agentic workflows so the Logfire/OFREP/fallback
logic lives in exactly one place.
inputs:
logfire-variable-key:
description: Logfire managed variable key holding the full prompt.
required: true
default-prompt-file:
description: Path to the committed default prompt file (the complete fallback prompt).
required: true
logfire-read-key:
description: >-
Logfire read key — pass the LOGFIRE_READ_EXTERNAL_VARIABLES secret from
the calling workflow. Empty/unset disables the Logfire lookup and uses
the committed default.
required: false
default: ""
logfire-base-url:
description: Logfire OFREP base URL.
required: false
default: https://logfire-api.pydantic.dev
outputs:
dynamic_prompt:
description: The resolved prompt — the Logfire managed-variable value, or the committed default.
value: ${{ steps.resolve.outputs.dynamic_prompt }}
runs:
using: composite
steps:
- id: resolve
shell: bash
# Inputs are passed via env (never interpolated into the script body) so
# the resolver is injection-safe.
env:
LOGFIRE_READ_KEY: ${{ inputs.logfire-read-key }}
LOGFIRE_BASE_URL: ${{ inputs.logfire-base-url }}
LOGFIRE_VARIABLE_KEY: ${{ inputs.logfire-variable-key }}
DEFAULT_PROMPT_FILE: ${{ inputs.default-prompt-file }}
TARGETING_KEY: gh-aw-${{ github.repository }}
run: |
set -euo pipefail
emit_prompt() {
{
echo "dynamic_prompt<<__GH_AW_DYNAMIC_PROMPT_EOF__"
printf '%s\n' "$1"
echo "__GH_AW_DYNAMIC_PROMPT_EOF__"
} >> "$GITHUB_OUTPUT"
}
# The Logfire managed variable holds the COMPLETE prompt. When it is
# unset or unreachable, fall back to the full committed default so the
# agent always receives a complete prompt — never a partial one.
use_default() {
echo "::notice::$1 — using the committed default prompt (${DEFAULT_PROMPT_FILE})."
emit_prompt "$(cat "${DEFAULT_PROMPT_FILE}")"
exit 0
}
[ -n "${LOGFIRE_READ_KEY:-}" ] || use_default "LOGFIRE_READ_EXTERNAL_VARIABLES not set"
RESPONSE="$(curl --fail --silent --show-error \
--max-time 20 \
-X POST \
-H "Authorization: Bearer ${LOGFIRE_READ_KEY}" \
-H "Content-Type: application/json" \
-d "{\"context\":{\"targetingKey\":\"${TARGETING_KEY}\"}}" \
"${LOGFIRE_BASE_URL%/}/v1/ofrep/v1/evaluate/flags/${LOGFIRE_VARIABLE_KEY}")" \
|| use_default "Logfire OFREP request failed"
PROMPT="$(printf '%s' "$RESPONSE" | jq -r '.value // empty')"
if [ -z "$PROMPT" ]; then
REASON="$(printf '%s' "$RESPONSE" | jq -r '.reason // "UNKNOWN"')"
use_default "No Logfire value for ${LOGFIRE_VARIABLE_KEY} (reason: ${REASON})"
fi
emit_prompt "$PROMPT"
echo "Loaded full prompt (${#PROMPT} chars) from Logfire variable '${LOGFIRE_VARIABLE_KEY}'."
+19
View File
@@ -0,0 +1,19 @@
{
"entries": {
"actions/github-script@v9.0.0": {
"repo": "actions/github-script",
"version": "v9.0.0",
"sha": "3a2844b7e9c422d3c10d287c895573f7108da1b3"
},
"github/gh-aw-actions/setup-cli@v0.74.8": {
"repo": "github/gh-aw-actions/setup-cli",
"version": "v0.74.8",
"sha": "efa55847f72aadb03490d955263ff911bf758700"
},
"github/gh-aw-actions/setup@v0.74.8": {
"repo": "github/gh-aw-actions/setup",
"version": "v0.74.8",
"sha": "efa55847f72aadb03490d955263ff911bf758700"
}
}
}
+24
View File
@@ -0,0 +1,24 @@
updates:
- cooldown:
default-days: 7
directory: /
groups:
python-packages:
patterns:
- "*"
package-ecosystem: uv
schedule:
interval: monthly
- cooldown:
default-days: 7
directory: /
groups:
github-actions:
patterns:
- "*"
ignore:
- dependency-name: "github/gh-aw-actions/**" # Managed by gh aw compile. Version-locked to the gh-aw compiler; do not bump.
package-ecosystem: github-actions
schedule:
interval: monthly
version: 2
+18
View File
@@ -0,0 +1,18 @@
<!-- Thank you for contributing to Pydantic AI! -->
<!-- Please read the contributing guide: https://ai.pydantic.dev/contributing/ -->
<!-- For non-trivial changes, link an issue that a maintainer has agreed on -->
<!-- and assigned to you. Unassigned PRs may be auto-closed. -->
<!-- Trivial fixes (typos, broken links, small doc improvements) don't need an issue. -->
<!-- Adding a capability? Most capabilities belong in Pydantic AI Harness, not here. -->
<!-- See: https://ai.pydantic.dev/harness/overview/#what-goes-where -->
- Closes #<issue>
### Checklist
- [ ] Any **AI generated code** has been reviewed line-by-line by the human PR author, who stands by it.
- [ ] No **breaking changes** in accordance with the [version policy](https://github.com/pydantic/pydantic-ai/blob/main/docs/version-policy.md).
- [ ] **PR title** is fit for the [release changelog](https://github.com/pydantic/pydantic-ai/releases).
+15
View File
@@ -0,0 +1,15 @@
changelog:
exclude:
labels:
- docs
- chore
categories:
- title: 🚀 Features
labels:
- feature
- title: 🐛 Bug Fixes
labels:
- bug
- title: 📦 Dependencies
labels:
- dependency
+890
View File
@@ -0,0 +1,890 @@
from __future__ import annotations
import argparse
import json
import os
import re
import ssl
import statistics
import sys
import time
import urllib.error
import urllib.parse
import urllib.request
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Literal
SCHEMA_VERSION = 1
CI_WORKFLOW_FILE = 'ci.yml'
REPORT_MARKER = '<!-- ci-duration-report -->'
REPORT_LABEL = 'trigger:ci-duration-report'
BASELINE_MAIN_RUN_LIMIT = 30
BASELINE_PR_RUN_LIMIT = 60
BASELINE_COLLECTION_MAX_SECONDS = 90
MIN_BASELINE_SAMPLES = 10
WARNING_MIN_SECONDS = 60
SLOW_THRESHOLD_MULTIPLIER = 1.25
FAST_THRESHOLD_MULTIPLIER = 0.75
VERY_SLOW_MIN_SECONDS = 300
VERY_SLOW_THRESHOLD_MULTIPLIER = 1.5
JsonValue = None | bool | int | float | str | list['JsonValue'] | dict[str, 'JsonValue']
JsonObject = dict[str, JsonValue]
MetricAttributes = dict[str, bool | int | float | str]
@dataclass(frozen=True)
class WorkflowRunRecord:
repo: str
workflow_id: int
workflow_name: str
workflow_path: str
run_id: int
run_attempt: int
run_number: int
event: str
status: str
conclusion: str | None
head_branch: str | None
base_branch: str | None
head_sha: str
pr_numbers: list[int]
run_started_at: str | None
updated_at: str | None
duration_seconds: float | None
html_url: str
actor: str | None
@dataclass(frozen=True)
class StepRecord:
number: int
name: str
status: str
conclusion: str | None
started_at: str | None
completed_at: str | None
duration_seconds: float | None
@dataclass(frozen=True)
class JobRecord:
job_id: int
raw_name: str
job_family: str
job_signature: str
matrix_python: str | None
matrix_extra: str | None
conclusion: str | None
status: str
started_at: str | None
completed_at: str | None
duration_seconds: float | None
runner_name: str | None
runner_group_name: str | None
runner_class: str
html_url: str
steps: list[StepRecord]
@dataclass(frozen=True)
class Baseline:
sample_size: int
median_seconds: float
p75_seconds: float
p90_seconds: float
mad_seconds: float
@dataclass(frozen=True)
class ReportRow:
job_name: str
job_signature: str
duration_seconds: float | None
baseline: Baseline | None
delta_seconds: float | None
delta_percent: float | None
status: Literal['normal', 'fast', 'slow', 'very_slow', 'no_baseline', 'not_completed']
class GitHubClient:
def __init__(self, repo: str, token: str):
self.repo = repo
self.token = token
self.ssl_context = _ssl_context()
def request_json(self, path: str, *, method: str = 'GET', body: JsonObject | None = None) -> JsonValue:
data = json.dumps(body).encode() if body is not None else None
request = urllib.request.Request(
self._url(path),
data=data,
method=method,
headers={
'Accept': 'application/vnd.github+json',
'Authorization': f'Bearer {self.token}',
'Content-Type': 'application/json',
'X-GitHub-Api-Version': '2022-11-28',
},
)
with urllib.request.urlopen(request, timeout=30, context=self.ssl_context) as response:
response_body = response.read()
if not response_body:
return None
return json.loads(response_body)
def request_paginated(self, path: str, *, max_items: int | None = None) -> list[JsonObject]:
parsed = urllib.parse.urlsplit(path)
query = urllib.parse.parse_qsl(parsed.query, keep_blank_values=True)
query = [item for item in query if item[0] not in {'page', 'per_page'}]
query.append(('per_page', '100'))
page = 1
results: list[JsonObject] = []
while True:
page_query = [*query, ('page', str(page))]
page_path = urllib.parse.urlunsplit(
(parsed.scheme, parsed.netloc, parsed.path, urllib.parse.urlencode(page_query), '')
)
value = self.request_json(page_path)
page_items = _extract_page_items(value)
if not page_items:
return results
results.extend(page_items)
if max_items is not None and len(results) >= max_items:
return results[:max_items]
if len(page_items) < 100:
return results
page += 1
def _url(self, path: str) -> str:
if path.startswith('http://') or path.startswith('https://'):
return path
if not path.startswith('/'):
path = f'/repos/{self.repo}/{path}'
return f'https://api.github.com{path}'
def main() -> None:
parser = argparse.ArgumentParser(description='Collect and report GitHub Actions CI duration telemetry.')
subparsers = parser.add_subparsers(dest='command', required=True)
collect_parser = subparsers.add_parser('collect', help='Collect one CI workflow run and emit it to Logfire.')
collect_parser.add_argument('--run-id', default=os.getenv('CI_RUN_ID') or os.getenv('GITHUB_EVENT_WORKFLOW_RUN_ID'))
collect_parser.add_argument(
'--run-attempt', default=os.getenv('CI_RUN_ATTEMPT') or os.getenv('GITHUB_EVENT_WORKFLOW_RUN_ATTEMPT')
)
collect_parser.add_argument('--output', default='ci-duration-record.json')
collect_parser.add_argument('--skip-logfire', action='store_true')
report_parser = subparsers.add_parser('report', help='Post or update a PR CI duration report.')
report_parser.add_argument('--pr-number', default=os.getenv('PR_NUMBER'))
report_parser.add_argument('--head-sha', default=os.getenv('HEAD_SHA'))
report_parser.add_argument('--poll-seconds', type=int, default=0)
report_parser.add_argument('--dry-run', action='store_true')
args = parser.parse_args()
client = _github_client_from_env()
if args.command == 'collect':
if args.run_id is None:
raise SystemExit('CI_RUN_ID is required')
run_attempt = int(args.run_attempt) if args.run_attempt else None
record = collect_run(client, int(args.run_id), run_attempt)
Path(args.output).write_text(json.dumps(record, indent=2, sort_keys=True) + '\n', encoding='utf-8')
print(f'Wrote CI duration record to {args.output}')
if not args.skip_logfire:
emit_logfire(record)
else:
if args.pr_number is None:
raise SystemExit('PR_NUMBER is required')
if args.head_sha is None:
raise SystemExit('HEAD_SHA is required')
body = build_pr_report(client, int(args.pr_number), args.head_sha, args.poll_seconds)
if args.dry_run:
print(body)
else:
upsert_pr_comment(client, int(args.pr_number), body)
def collect_run(client: GitHubClient, run_id: int, run_attempt: int | None) -> JsonObject:
run = _expect_object(client.request_json(f'actions/runs/{run_id}'), 'workflow run')
attempt = run_attempt or _expect_int(run.get('run_attempt'), 'run_attempt')
jobs = client.request_paginated(f'actions/runs/{run_id}/attempts/{attempt}/jobs')
workflow = normalize_workflow_run(client.repo, run, attempt)
job_records: list[JobRecord] = []
for job_object in jobs:
job = normalize_job(job_object)
if is_tracked_test_job(job):
job_records.append(job)
return {
'schema_version': SCHEMA_VERSION,
'fetched_at': _now(),
'workflow_run': workflow_to_json(workflow),
'jobs': [job_to_json(job) for job in job_records],
}
def build_pr_report(client: GitHubClient, pr_number: int, head_sha: str, poll_seconds: int) -> str:
run = wait_for_completed_ci_run(client, head_sha, poll_seconds)
if run is None:
return render_waiting_report(head_sha)
run_id = _expect_int(run.get('id'), 'run id')
run_attempt = _expect_int(run.get('run_attempt'), 'run_attempt')
current_record = collect_run(client, run_id, run_attempt)
current_jobs = [_job_from_json(job) for job in _expect_list(current_record['jobs'], 'jobs')]
baselines = collect_baselines(client, head_sha)
rows = [classify_job(job, baselines.get(job.job_signature)) for job in current_jobs]
workflow = _expect_object(current_record['workflow_run'], 'workflow_run')
return render_report(pr_number, head_sha, workflow, rows)
def wait_for_completed_ci_run(client: GitHubClient, head_sha: str, poll_seconds: int) -> JsonObject | None:
deadline = time.monotonic() + poll_seconds
while True:
run = find_latest_ci_run(client, head_sha)
if run is not None and run.get('status') == 'completed':
return run
if poll_seconds <= 0 or time.monotonic() >= deadline:
return None
time.sleep(20)
def find_latest_ci_run(client: GitHubClient, head_sha: str) -> JsonObject | None:
runs = client.request_paginated(f'actions/workflows/{CI_WORKFLOW_FILE}/runs?head_sha={head_sha}', max_items=10)
matching = [run for run in runs if run.get('head_sha') == head_sha]
if not matching:
return None
matching.sort(key=lambda run: str(run.get('created_at') or ''), reverse=True)
return matching[0]
def collect_baselines(client: GitHubClient, current_head_sha: str) -> dict[str, Baseline]:
try:
main_runs = client.request_paginated(
f'actions/workflows/{CI_WORKFLOW_FILE}/runs?branch=main&event=push&status=success',
max_items=BASELINE_MAIN_RUN_LIMIT,
)
pr_runs = client.request_paginated(
f'actions/workflows/{CI_WORKFLOW_FILE}/runs?event=pull_request&status=success',
max_items=BASELINE_PR_RUN_LIMIT,
)
except (TimeoutError, urllib.error.URLError) as exc:
print(f'Unable to collect baseline CI runs: {exc}', file=sys.stderr)
return {}
durations: dict[str, list[float]] = {}
seen_run_ids: set[int] = set()
baseline_deadline = time.monotonic() + BASELINE_COLLECTION_MAX_SECONDS
for run in [*main_runs, *pr_runs]:
if time.monotonic() >= baseline_deadline:
print('Baseline collection time budget exhausted; using partial baseline samples', file=sys.stderr)
break
run_id = _expect_int(run.get('id'), 'baseline run id')
if run_id in seen_run_ids or run.get('head_sha') == current_head_sha:
continue
seen_run_ids.add(run_id)
run_attempt = _expect_int(run.get('run_attempt'), 'baseline run_attempt')
try:
jobs = client.request_paginated(f'actions/runs/{run_id}/attempts/{run_attempt}/jobs')
except (TimeoutError, urllib.error.URLError) as exc:
print(f'Unable to collect baseline jobs for run {run_id}: {exc}', file=sys.stderr)
continue
for job_object in jobs:
job = normalize_job(job_object)
if job.conclusion == 'success' and job.duration_seconds is not None and is_tracked_test_job(job):
durations.setdefault(job.job_signature, []).append(job.duration_seconds)
return {
signature: compute_baseline(values)
for signature, values in durations.items()
if len(values) >= MIN_BASELINE_SAMPLES
}
def normalize_workflow_run(repo: str, run: JsonObject, run_attempt: int) -> WorkflowRunRecord:
run_started_at = _expect_optional_str(run.get('run_started_at'), 'run_started_at')
updated_at = _expect_optional_str(run.get('updated_at'), 'updated_at')
return WorkflowRunRecord(
repo=repo,
workflow_id=_expect_int(run.get('workflow_id'), 'workflow_id'),
workflow_name=_expect_str(run.get('name'), 'workflow name'),
workflow_path=_expect_str(run.get('path'), 'workflow path'),
run_id=_expect_int(run.get('id'), 'run id'),
run_attempt=run_attempt,
run_number=_expect_int(run.get('run_number'), 'run_number'),
event=_expect_str(run.get('event'), 'event'),
status=_expect_str(run.get('status'), 'status'),
conclusion=_expect_optional_str(run.get('conclusion'), 'conclusion'),
head_branch=_expect_optional_str(run.get('head_branch'), 'head_branch'),
base_branch=_expect_optional_str(run.get('base_ref'), 'base_ref'),
head_sha=_expect_str(run.get('head_sha'), 'head_sha'),
pr_numbers=_pull_request_numbers(run.get('pull_requests')),
run_started_at=run_started_at,
updated_at=updated_at,
duration_seconds=_duration_seconds(run_started_at, updated_at),
html_url=_expect_str(run.get('html_url'), 'html_url'),
actor=_actor_login(run.get('actor')),
)
def normalize_job(job: JsonObject) -> JobRecord:
raw_name = _expect_str(job.get('name'), 'job name')
started_at = _expect_optional_str(job.get('started_at'), 'job started_at')
completed_at = _expect_optional_str(job.get('completed_at'), 'job completed_at')
matrix_python, matrix_extra = parse_job_matrix(raw_name)
job_family = parse_job_family(raw_name)
runner_class = parse_runner_class(
_expect_optional_str(job.get('runner_group_name'), 'runner_group_name'),
_expect_optional_str(job.get('runner_name'), 'runner_name'),
_expect_list_or_none(job.get('labels'), 'labels'),
)
return JobRecord(
job_id=_expect_int(job.get('id'), 'job id'),
raw_name=raw_name,
job_family=job_family,
job_signature=job_signature(job_family, matrix_python, matrix_extra, runner_class),
matrix_python=matrix_python,
matrix_extra=matrix_extra,
conclusion=_expect_optional_str(job.get('conclusion'), 'job conclusion'),
status=_expect_str(job.get('status'), 'job status'),
started_at=started_at,
completed_at=completed_at,
duration_seconds=_duration_seconds(started_at, completed_at),
runner_name=_expect_optional_str(job.get('runner_name'), 'runner_name'),
runner_group_name=_expect_optional_str(job.get('runner_group_name'), 'runner_group_name'),
runner_class=runner_class,
html_url=_expect_str(job.get('html_url'), 'job html_url'),
steps=[normalize_step(step) for step in _expect_list_or_none(job.get('steps'), 'steps') or []],
)
def normalize_step(step: JsonValue) -> StepRecord:
step_object = _expect_object(step, 'step')
started_at = _expect_optional_str(step_object.get('started_at'), 'step started_at')
completed_at = _expect_optional_str(step_object.get('completed_at'), 'step completed_at')
return StepRecord(
number=_expect_int(step_object.get('number'), 'step number'),
name=_expect_str(step_object.get('name'), 'step name'),
status=_expect_str(step_object.get('status'), 'step status'),
conclusion=_expect_optional_str(step_object.get('conclusion'), 'step conclusion'),
started_at=started_at,
completed_at=completed_at,
duration_seconds=_duration_seconds(started_at, completed_at),
)
def workflow_to_json(workflow: WorkflowRunRecord) -> JsonObject:
pr_numbers: list[JsonValue] = [number for number in workflow.pr_numbers]
return {
'repo': workflow.repo,
'workflow_id': workflow.workflow_id,
'workflow_name': workflow.workflow_name,
'workflow_path': workflow.workflow_path,
'run_id': workflow.run_id,
'run_attempt': workflow.run_attempt,
'run_number': workflow.run_number,
'event': workflow.event,
'status': workflow.status,
'conclusion': workflow.conclusion,
'head_branch': workflow.head_branch,
'base_branch': workflow.base_branch,
'head_sha': workflow.head_sha,
'pr_numbers': pr_numbers,
'run_started_at': workflow.run_started_at,
'updated_at': workflow.updated_at,
'duration_seconds': workflow.duration_seconds,
'html_url': workflow.html_url,
'actor': workflow.actor,
}
def job_to_json(job: JobRecord) -> JsonObject:
return {
'job_id': job.job_id,
'raw_name': job.raw_name,
'job_family': job.job_family,
'job_signature': job.job_signature,
'matrix_python': job.matrix_python,
'matrix_extra': job.matrix_extra,
'conclusion': job.conclusion,
'status': job.status,
'started_at': job.started_at,
'completed_at': job.completed_at,
'duration_seconds': job.duration_seconds,
'runner_name': job.runner_name,
'runner_group_name': job.runner_group_name,
'runner_class': job.runner_class,
'html_url': job.html_url,
'steps': [step_to_json(step) for step in job.steps],
}
def step_to_json(step: StepRecord) -> JsonObject:
return {
'number': step.number,
'name': step.name,
'status': step.status,
'conclusion': step.conclusion,
'started_at': step.started_at,
'completed_at': step.completed_at,
'duration_seconds': step.duration_seconds,
}
def parse_job_matrix(job_name: str) -> tuple[str | None, str | None]:
match = re.fullmatch(r'test on (?P<python>[^ ]+) \((?P<extra>[^)]+)\)', job_name)
if match:
return match.group('python'), match.group('extra')
match = re.fullmatch(r'test examples on (?P<python>[^ ]+)', job_name)
if match:
return match.group('python'), 'examples'
return None, None
def parse_job_family(job_name: str) -> str:
if job_name.startswith('test on '):
return 'test'
if job_name.startswith('test examples on '):
return 'test-examples'
if job_name in {'lint', 'mypy', 'docs', 'coverage', 'check'}:
return job_name
return job_name
def is_tracked_test_job(job: JobRecord) -> bool:
return job.job_family == 'test'
def parse_runner_class(runner_group_name: str | None, runner_name: str | None, labels: list[JsonValue] | None) -> str:
label_values = [value for value in labels or [] if isinstance(value, str)]
lower_values = ' '.join([runner_group_name or '', runner_name or '', *label_values]).lower()
if 'depot' in lower_values:
return 'depot'
if 'ubicloud' in lower_values:
return 'ubicloud'
if 'github actions' in lower_values or 'ubuntu' in lower_values:
return 'github-hosted'
if 'self-hosted' in lower_values:
return 'self-hosted'
return 'unknown'
def job_signature(job_family: str, matrix_python: str | None, matrix_extra: str | None, runner_class: str) -> str:
parts = [f'job={job_family}', f'runner={runner_class}']
if matrix_python is not None:
parts.append(f'py={matrix_python}')
if matrix_extra is not None:
parts.append(f'extra={matrix_extra}')
return ' / '.join(parts)
def compute_baseline(values: list[float]) -> Baseline:
sorted_values = sorted(values)
median = statistics.median(sorted_values)
deviations = [abs(value - median) for value in sorted_values]
return Baseline(
sample_size=len(sorted_values),
median_seconds=median,
p75_seconds=percentile(sorted_values, 75),
p90_seconds=percentile(sorted_values, 90),
mad_seconds=statistics.median(deviations),
)
def percentile(sorted_values: list[float], percentile_value: int) -> float:
if len(sorted_values) == 1:
return sorted_values[0]
index = (len(sorted_values) - 1) * percentile_value / 100
lower = int(index)
upper = min(lower + 1, len(sorted_values) - 1)
fraction = index - lower
return sorted_values[lower] * (1 - fraction) + sorted_values[upper] * fraction
def classify_job(job: JobRecord, baseline: Baseline | None) -> ReportRow:
if job.conclusion != 'success' or job.duration_seconds is None:
return ReportRow(job.raw_name, job.job_signature, job.duration_seconds, baseline, None, None, 'not_completed')
if baseline is None:
return ReportRow(job.raw_name, job.job_signature, job.duration_seconds, None, None, None, 'no_baseline')
delta = job.duration_seconds - baseline.median_seconds
delta_percent = delta / baseline.median_seconds * 100 if baseline.median_seconds else None
slow_threshold = max(
baseline.p75_seconds * SLOW_THRESHOLD_MULTIPLIER, baseline.median_seconds + 2 * baseline.mad_seconds
)
very_slow_threshold = max(
baseline.p90_seconds * VERY_SLOW_THRESHOLD_MULTIPLIER,
baseline.median_seconds + 4 * baseline.mad_seconds,
)
if job.duration_seconds > very_slow_threshold or delta >= VERY_SLOW_MIN_SECONDS:
status: Literal['normal', 'fast', 'slow', 'very_slow', 'no_baseline', 'not_completed'] = 'very_slow'
elif job.duration_seconds > slow_threshold and delta >= WARNING_MIN_SECONDS:
status = 'slow'
elif job.duration_seconds < baseline.median_seconds * FAST_THRESHOLD_MULTIPLIER and delta <= -WARNING_MIN_SECONDS:
status = 'fast'
else:
status = 'normal'
return ReportRow(job.raw_name, job.job_signature, job.duration_seconds, baseline, delta, delta_percent, status)
def render_report(pr_number: int, head_sha: str, workflow: JsonObject, rows: list[ReportRow]) -> str:
slow_rows = [row for row in rows if row.status in {'slow', 'very_slow'}]
fast_rows = [row for row in rows if row.status == 'fast']
failed_rows = [row for row in rows if row.status == 'not_completed']
sorted_rows = sorted(
rows,
key=lambda row: (
row.status not in {'very_slow', 'slow', 'fast', 'not_completed'},
-(row.delta_seconds or 0),
row.job_name,
),
)
tracked_duration = sum(row.duration_seconds or 0 for row in rows)
run_url = _expect_str(workflow.get('html_url'), 'workflow html_url')
sha7 = head_sha[:7]
lines = [
REPORT_MARKER,
'## CI Duration Report',
'',
f'PR #{pr_number}, commit `{sha7}`: [CI run]({run_url})',
'',
'**Summary**',
f'- Tracked test jobs: {len(rows)}',
f'- Total tracked test job duration: {_format_seconds(tracked_duration)}',
f'- Slow jobs: {len(slow_rows)}',
f'- Fast jobs: {len(fast_rows)}',
f'- Failed/cancelled jobs: {len(failed_rows)}',
f'- Baseline: up to {BASELINE_MAIN_RUN_LIMIT} successful `main` CI runs and {BASELINE_PR_RUN_LIMIT} successful PR CI runs, matched by job signature and runner class',
f'- Minimum baseline sample: {MIN_BASELINE_SAMPLES} successful matching jobs',
f'- Slow threshold: duration > max(p75 * {SLOW_THRESHOLD_MULTIPLIER}, median + 2 * MAD), with at least {WARNING_MIN_SECONDS}s increase',
'',
'| Job | Duration | Baseline median | p75 | Delta | Status |',
'|---|---:|---:|---:|---:|---|',
]
for row in sorted_rows[:20]:
lines.append(
'| '
+ ' | '.join(
[
row.job_name,
_format_seconds(row.duration_seconds),
_format_seconds(row.baseline.median_seconds if row.baseline else None),
_format_seconds(row.baseline.p75_seconds if row.baseline else None),
_format_delta(row.delta_seconds, row.delta_percent),
row.status.replace('_', ' '),
]
)
+ ' |'
)
if len(sorted_rows) > 20:
lines.append(f'| ... | {len(sorted_rows) - 20} more jobs omitted | | | | |')
lines.extend(
[
'',
'<sub>Re-add the `trigger:ci-duration-report` label to refresh this report.</sub>',
]
)
return '\n'.join(lines)
def render_waiting_report(head_sha: str) -> str:
return '\n'.join(
[
REPORT_MARKER,
'## CI Duration Report — waiting for CI',
'',
f'No completed `CI` run was found for commit `{head_sha[:7]}` yet.',
'',
'<sub>Re-add the `trigger:ci-duration-report` label after CI completes, or rerun with a longer poll window.</sub>',
]
)
def upsert_pr_comment(client: GitHubClient, pr_number: int, body: str) -> None:
comments = client.request_paginated(f'issues/{pr_number}/comments')
existing_url: str | None = None
for comment in comments:
user = _expect_object(comment.get('user'), 'comment user')
if user.get('login') == 'github-actions[bot]' and str(comment.get('body') or '').startswith(REPORT_MARKER):
existing_url = _expect_str(comment.get('url'), 'comment url')
break
if existing_url:
client.request_json(existing_url, method='PATCH', body={'body': body})
print('Updated existing CI duration report comment')
else:
client.request_json(f'issues/{pr_number}/comments', method='POST', body={'body': body})
print('Created CI duration report comment')
def emit_logfire(record: JsonObject) -> None:
token = os.getenv('LOGFIRE_WRITE_TOKEN') or os.getenv('LOGFIRE_TOKEN')
if not token:
print('LOGFIRE_WRITE_TOKEN is not set; skipping Logfire emission')
return
try:
import logfire
except ImportError:
print('logfire is not installed; skipping Logfire emission')
return
logfire_base_url = os.getenv('LOGFIRE_URL')
advanced_options = logfire.AdvancedOptions(base_url=logfire_base_url) if logfire_base_url else None
logfire.configure(
token=token,
service_name='pydantic-ai-ci',
environment='github-actions',
console=False,
advanced=advanced_options,
)
workflow = _expect_object(record['workflow_run'], 'workflow_run')
jobs = _expect_list(record['jobs'], 'jobs')
tracked_test_duration_seconds = sum(
_expect_optional_float(_expect_object(job, 'job').get('duration_seconds'), 'duration_seconds') or 0
for job in jobs
)
test_run_duration_metric = logfire.metric_histogram(
'ci.test_run.tracked_duration',
unit='s',
description='Total duration of tracked CI test jobs in one workflow run.',
)
test_job_duration_metric = logfire.metric_histogram(
'ci.test_job.duration',
unit='s',
description='Duration of one tracked CI test matrix job.',
)
test_run_duration_metric.record(
tracked_test_duration_seconds,
metric_attributes(
{
'repo': workflow.get('repo'),
'workflow_name': workflow.get('workflow_name'),
'event': workflow.get('event'),
'base_branch': workflow.get('base_branch'),
'conclusion': workflow.get('conclusion'),
}
),
)
with logfire.span(
'ci.duration.test_run',
_tags=['ci-duration'],
schema_version=SCHEMA_VERSION,
repo=workflow.get('repo'),
workflow_name=workflow.get('workflow_name'),
run_id=workflow.get('run_id'),
run_attempt=workflow.get('run_attempt'),
event=workflow.get('event'),
conclusion=workflow.get('conclusion'),
head_branch=workflow.get('head_branch'),
base_branch=workflow.get('base_branch'),
head_sha=workflow.get('head_sha'),
pr_numbers=workflow.get('pr_numbers'),
duration_seconds=workflow.get('duration_seconds'),
tracked_test_jobs=len(jobs),
tracked_test_duration_seconds=tracked_test_duration_seconds,
html_url=workflow.get('html_url'),
):
for job in jobs:
job_object = _expect_object(job, 'job')
duration_seconds = _expect_optional_float(job_object.get('duration_seconds'), 'duration_seconds')
if duration_seconds is not None:
test_job_duration_metric.record(
duration_seconds,
metric_attributes(
{
'repo': workflow.get('repo'),
'workflow_name': workflow.get('workflow_name'),
'event': workflow.get('event'),
'base_branch': workflow.get('base_branch'),
'job_name': job_object.get('raw_name'),
'job_signature': job_object.get('job_signature'),
'matrix_python': job_object.get('matrix_python'),
'matrix_extra': job_object.get('matrix_extra'),
'runner_class': job_object.get('runner_class'),
'conclusion': job_object.get('conclusion'),
}
),
)
logfire.info(
'ci.duration.test_job',
_tags=['ci-duration'],
schema_version=SCHEMA_VERSION,
repo=workflow.get('repo'),
run_id=workflow.get('run_id'),
run_attempt=workflow.get('run_attempt'),
event=workflow.get('event'),
head_branch=workflow.get('head_branch'),
base_branch=workflow.get('base_branch'),
head_sha=workflow.get('head_sha'),
pr_numbers=workflow.get('pr_numbers'),
job_id=job_object.get('job_id'),
job_name=job_object.get('raw_name'),
job_family=job_object.get('job_family'),
job_signature=job_object.get('job_signature'),
matrix_python=job_object.get('matrix_python'),
matrix_extra=job_object.get('matrix_extra'),
runner_class=job_object.get('runner_class'),
conclusion=job_object.get('conclusion'),
duration_seconds=job_object.get('duration_seconds'),
html_url=job_object.get('html_url'),
)
logfire.force_flush()
def _github_client_from_env() -> GitHubClient:
repo = os.getenv('GITHUB_REPOSITORY')
token = os.getenv('GITHUB_TOKEN')
if not repo:
raise SystemExit('GITHUB_REPOSITORY is required')
if not token:
raise SystemExit('GITHUB_TOKEN is required')
return GitHubClient(repo, token)
def metric_attributes(attributes: JsonObject) -> MetricAttributes:
return {key: value for key, value in attributes.items() if isinstance(value, bool | int | float | str)}
def _ssl_context() -> ssl.SSLContext | None:
try:
import certifi
except ImportError:
return None
return ssl.create_default_context(cafile=certifi.where())
def _extract_page_items(value: JsonValue) -> list[JsonObject]:
if isinstance(value, list):
return [_expect_object(item, 'paginated item') for item in value]
if isinstance(value, dict):
for key in ('jobs', 'workflow_runs', 'comments'):
items = value.get(key)
if isinstance(items, list):
return [_expect_object(item, key) for item in items]
raise RuntimeError(f'Unexpected paginated response shape: {value!r}')
def _job_from_json(value: JsonValue) -> JobRecord:
job = _expect_object(value, 'job')
return JobRecord(
job_id=_expect_int(job.get('job_id'), 'job_id'),
raw_name=_expect_str(job.get('raw_name'), 'raw_name'),
job_family=_expect_str(job.get('job_family'), 'job_family'),
job_signature=_expect_str(job.get('job_signature'), 'job_signature'),
matrix_python=_expect_optional_str(job.get('matrix_python'), 'matrix_python'),
matrix_extra=_expect_optional_str(job.get('matrix_extra'), 'matrix_extra'),
conclusion=_expect_optional_str(job.get('conclusion'), 'conclusion'),
status=_expect_str(job.get('status'), 'status'),
started_at=_expect_optional_str(job.get('started_at'), 'started_at'),
completed_at=_expect_optional_str(job.get('completed_at'), 'completed_at'),
duration_seconds=_expect_optional_float(job.get('duration_seconds'), 'duration_seconds'),
runner_name=_expect_optional_str(job.get('runner_name'), 'runner_name'),
runner_group_name=_expect_optional_str(job.get('runner_group_name'), 'runner_group_name'),
runner_class=_expect_str(job.get('runner_class'), 'runner_class'),
html_url=_expect_str(job.get('html_url'), 'html_url'),
steps=[],
)
def _pull_request_numbers(value: JsonValue) -> list[int]:
if not isinstance(value, list):
return []
numbers: list[int] = []
for item in value:
if isinstance(item, dict):
number = item.get('number')
if isinstance(number, int):
numbers.append(number)
return numbers
def _actor_login(value: JsonValue) -> str | None:
if isinstance(value, dict):
login = value.get('login')
if isinstance(login, str):
return login
return None
def _duration_seconds(start: str | None, end: str | None) -> float | None:
if start is None or end is None:
return None
return (_parse_timestamp(end) - _parse_timestamp(start)).total_seconds()
def _parse_timestamp(value: str) -> datetime:
return datetime.fromisoformat(value.replace('Z', '+00:00'))
def _now() -> str:
return datetime.now(timezone.utc).isoformat().replace('+00:00', 'Z')
def _format_seconds(value: float | None) -> str:
if value is None:
return 'n/a'
seconds = int(round(value))
minutes, remainder = divmod(seconds, 60)
if minutes:
return f'{minutes}m {remainder:02d}s'
return f'{remainder}s'
def _format_delta(delta_seconds: float | None, delta_percent: float | None) -> str:
if delta_seconds is None or delta_percent is None:
return 'n/a'
sign = '+' if delta_seconds >= 0 else '-'
return f'{sign}{_format_seconds(abs(delta_seconds))} ({delta_percent:+.0f}%)'
def _expect_object(value: JsonValue, label: str) -> JsonObject:
if isinstance(value, dict):
return value
raise RuntimeError(f'Expected {label} to be an object, got {type(value).__name__}')
def _expect_list(value: JsonValue, label: str) -> list[JsonValue]:
if isinstance(value, list):
return value
raise RuntimeError(f'Expected {label} to be a list, got {type(value).__name__}')
def _expect_list_or_none(value: JsonValue, label: str) -> list[JsonValue] | None:
if value is None:
return None
return _expect_list(value, label)
def _expect_str(value: JsonValue, label: str) -> str:
if isinstance(value, str):
return value
raise RuntimeError(f'Expected {label} to be a string, got {type(value).__name__}')
def _expect_optional_str(value: JsonValue, label: str) -> str | None:
if value is None:
return None
return _expect_str(value, label)
def _expect_int(value: JsonValue, label: str) -> int:
if isinstance(value, int) and not isinstance(value, bool):
return value
raise RuntimeError(f'Expected {label} to be an integer, got {type(value).__name__}')
def _expect_optional_float(value: JsonValue, label: str) -> float | None:
if value is None:
return None
if isinstance(value, int | float) and not isinstance(value, bool):
return float(value)
raise RuntimeError(f'Expected {label} to be a number, got {type(value).__name__}')
if __name__ == '__main__':
try:
main()
except urllib.error.HTTPError as exc:
print(f'GitHub API request failed: HTTP {exc.code}\n{exc.read().decode()}', file=sys.stderr)
raise
+41
View File
@@ -0,0 +1,41 @@
#!/usr/bin/env bash
# Install tools that the agent needs inside the AWF sandbox.
#
# Runs in `pre-agent-steps` (on the runner, open network, after checkout)
# before AWF starts the firewalled container. Tools are exposed via:
#
# 1. /opt/hostedtoolcache/gh-aw-tools/current/x64/bin — AWF auto-scans
# hostedtoolcache bin dirs and merges them into the container PATH.
# 2. $GITHUB_PATH — AWF reads this file at container startup and merges
# entries into AWF_HOST_PATH (the container's PATH).
#
# This follows the pattern used by elastic/ai-github-actions.
set -euo pipefail
toolcache_bin="/opt/hostedtoolcache/gh-aw-tools/current/x64/bin"
sudo mkdir -p "$toolcache_bin"
# --- ripgrep ---
# The agent's native Grep tool wraps `rg` for fast code search.
echo "[install-sandbox-tools] Installing ripgrep..."
uv tool install ripgrep --force --quiet
rg_path="$(uv tool dir --bin)/rg"
if [ -x "$rg_path" ]; then
sudo ln -sf "$rg_path" "$toolcache_bin/rg"
echo "[install-sandbox-tools] rg -> $toolcache_bin/rg"
else
echo "::warning::ripgrep install succeeded but rg binary not found at $rg_path"
fi
# --- uv ---
# Symlink uv into the toolcache so the launcher and Bash tool can find it.
uv_path="$(command -v uv)"
if [ -n "$uv_path" ]; then
sudo ln -sf "$uv_path" "$toolcache_bin/uv"
echo "[install-sandbox-tools] uv -> $toolcache_bin/uv"
fi
# Belt-and-suspenders: also write to $GITHUB_PATH so AWF's GITHUB_PATH
# merge picks it up (works on AWF versions that support this).
echo "$toolcache_bin" >> "${GITHUB_PATH:-/dev/null}"
echo "[install-sandbox-tools] Added $toolcache_bin to GITHUB_PATH"
+64
View File
@@ -0,0 +1,64 @@
#!/usr/bin/env bash
set -euo pipefail
repo="${GITHUB_REPOSITORY:?GITHUB_REPOSITORY is required}"
out_root="/tmp/gh-aw/agent"
issues_root="${out_root}/issues"
all_dir="${issues_root}/all"
batches_dir="${issues_root}/batches"
index_file="${out_root}/open-issues.tsv"
manifest_file="${issues_root}/batch-manifest.tsv"
raw_file="${issues_root}/open-issues.raw.json"
batch_size="${BATCH_SIZE:-25}"
issue_limit="${ISSUE_LIMIT:-1000}"
rm -rf "${issues_root}"
mkdir -p "${all_dir}" "${batches_dir}"
printf "number\ttitle\tupdated_at\tcreated_at\tlabel_names\n" > "${index_file}"
printf "batch\tissue_number\tupdated_at\tlabel_names\n" > "${manifest_file}"
# Fetch all open issues sorted by oldest update time first.
gh issue list \
--repo "${repo}" \
--state open \
--limit "${issue_limit}" \
--search "sort:updated-asc" \
--json number,title,body,updatedAt,createdAt,url,labels,author,assignees \
> "${raw_file}"
issue_count="$(jq 'length' "${raw_file}")"
if [[ "${issue_count}" == "0" ]]; then
echo "No open issues found."
exit 0
fi
count=0
while IFS= read -r issue_json; do
count=$((count + 1))
number="$(jq -r '.number' <<< "${issue_json}")"
updated_at="$(jq -r '.updatedAt' <<< "${issue_json}")"
created_at="$(jq -r '.createdAt' <<< "${issue_json}")"
title="$(jq -r '.title' <<< "${issue_json}" | tr '\t\n' ' ' | sed 's/ */ /g')"
labels="$(jq -r '[.labels[].name] | join(",")' <<< "${issue_json}")"
printf '%s\n' "${issue_json}" > "${all_dir}/${number}.json"
printf "%s\t%s\t%s\t%s\t%s\n" "${number}" "${title}" "${updated_at}" "${created_at}" "${labels}" >> "${index_file}"
batch_index=$(((count - 1) / batch_size + 1))
batch_name="$(printf 'batch-%03d' "${batch_index}")"
batch_path="${batches_dir}/${batch_name}"
mkdir -p "${batch_path}"
cp "${all_dir}/${number}.json" "${batch_path}/${number}.json"
printf "%s\t%s\t%s\t%s\n" "${batch_name}" "${number}" "${updated_at}" "${labels}" >> "${manifest_file}"
done < <(jq -c '.[]' "${raw_file}")
batch_count="$(find "${batches_dir}" -mindepth 1 -maxdepth 1 -type d | wc -l | tr -d ' ')"
echo "Prescanned ${count} open issues into ${all_dir}"
echo "Created ${batch_count} batch folder(s) in ${batches_dir} (batch size: ${batch_size})"
echo "Index file: ${index_file}"
echo "Batch manifest: ${manifest_file}"
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
# Pre-warm the harness's uv script environment on the OPEN network.
#
# Runs in each workflow's `pre-agent-steps` — after checkout + Setup uv but
# before the firewalled agent step — into the same uv dirs the in-sandbox
# launcher uses, so the agent run reuses the warm cache instead of depending
# on PyPI access through the AWF firewall.
#
# Strictly non-fatal: on any failure the sandboxed `uv run --script` (with
# the `python` allowlist in each workflow) still installs from scratch.
set -uo pipefail
export UV_CACHE_DIR=/tmp/gh-aw/uv/cache
export UV_PYTHON_INSTALL_DIR=/tmp/gh-aw/uv/python
export UV_TOOL_DIR=/tmp/gh-aw/uv/tool
export XDG_DATA_HOME=/tmp/gh-aw/uv/data
export XDG_CACHE_HOME=/tmp/gh-aw/uv/xdg-cache
mkdir -p "$UV_CACHE_DIR" "$UV_PYTHON_INSTALL_DIR" "$UV_TOOL_DIR" "$XDG_DATA_HOME" "$XDG_CACHE_HOME"
runner="${GITHUB_WORKSPACE}/.github/scripts/pydantic-ai-runner"
uv_bin="$(command -v uv 2>/dev/null || true)"
if [ -z "${uv_bin}" ]; then
echo "::warning::uv not found for pre-warm; agent will install under the firewall"
exit 0
fi
echo "[harness-prewarm] using uv=${uv_bin} cache=${UV_CACHE_DIR}"
"${uv_bin}" sync --script "${runner}" \
|| echo "::warning::harness uv pre-warm failed; agent will install under the firewall"
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "pydantic-ai-slim[anthropic,mcp]>=1.105.0",
# # The file/shell tools (Bash/Read/Write/Edit/Grep/Glob/LS) are backed by
# # harness FileSystem/Shell, which ship as stable top-level modules from 0.4.0
# # onwards. Keep this in sync with the lint-group pin in the root pyproject.toml.
# "pydantic-ai-harness>=0.4.0",
# "logfire",
# "opentelemetry-instrumentation-httpx",
# ]
# ///
"""Pydantic AI gh-aw shim launcher.
Thin entry point that defers to the `pydantic_ai_gh_aw_shim` package
beside this script. The real shim lives in `pydantic_ai_gh_aw_shim.cli`;
the package's `__main__.py` calls `cli.main()`. This file exists only to
satisfy gh-aw's expectation of a single executable command (and to
carry the PEP 723 inline-metadata dependency block for `uv run --script`).
"""
import pathlib
import runpy
import sys
# `pydantic_ai_gh_aw_shim/` lives next to this script — put its parent on
# `sys.path` so `runpy.run_module` can find it (and the shim's `from .`
# relative imports resolve).
sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent))
runpy.run_module("pydantic_ai_gh_aw_shim", run_name="__main__")
+42
View File
@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# In-sandbox launcher for the Pydantic AI gh-aw shim.
#
# The checked-out workspace is mounted no-exec in the AWF sandbox, so this
# script is installed into gh-aw's exec-able /tmp/gh-aw/bin/ by the workflow's
# pre-step (`install -m 755 ... /tmp/gh-aw/bin/pydantic-ai-runner-launch`).
# It is gh-aw's `engine.command` for every Pydantic AI agentic workflow and
# `uv run --script`s the in-tree runner stub with the agent's argv. The stub
# (`pydantic-ai-runner`) is a tiny `runpy` shim that hands off to the
# `pydantic_ai_gh_aw_shim` package in the same directory.
#
# AWF propagates setup-* tool paths into the container via $GITHUB_PATH and
# /opt/hostedtoolcache — so `uv` and `rg` should be on PATH already. The
# launcher just sets up the uv cache dirs (the default cache dir from
# setup-uv isn't writable by the sandbox user UID 1001).
set -euo pipefail
export UV_CACHE_DIR=/tmp/gh-aw/uv/cache
export UV_PYTHON_INSTALL_DIR=/tmp/gh-aw/uv/python
export UV_TOOL_DIR=/tmp/gh-aw/uv/tool
export XDG_DATA_HOME=/tmp/gh-aw/uv/data
export XDG_CACHE_HOME=/tmp/gh-aw/uv/xdg-cache
mkdir -p "$UV_CACHE_DIR" "$UV_PYTHON_INSTALL_DIR" "$UV_TOOL_DIR" "$XDG_DATA_HOME" "$XDG_CACHE_HOME"
runner="${GITHUB_WORKSPACE}/.github/scripts/pydantic-ai-runner"
echo "[harness-launch] cwd=$(pwd) GITHUB_WORKSPACE=${GITHUB_WORKSPACE:-unset} UV_CACHE_DIR=${UV_CACHE_DIR}" >&2
echo "[harness-launch] runner=${runner} exists=$([ -f "${runner}" ] && echo yes || echo no)" >&2
# Find uv — should be on PATH via AWF's hostedtoolcache propagation.
# Fall back to known paths if not (older AWF versions or non-GHA runners).
uv_bin=""
if command -v uv >/dev/null 2>&1; then
uv_bin="$(command -v uv)"
else
for c in /opt/hostedtoolcache/gh-aw-tools/current/x64/bin/uv /opt/hostedtoolcache/uv/*/*/uv /tmp/gh-aw/bin/uv /usr/local/bin/uv; do
[ -x "$c" ] && uv_bin="$c" && break
done
fi
if [ -z "${uv_bin}" ]; then
echo "[harness-launch] FATAL: uv not found; PATH=${PATH}" >&2
exit 127
fi
echo "[harness-launch] using uv=${uv_bin}" >&2
exec "${uv_bin}" run --script "${runner}" "$@"
@@ -0,0 +1,131 @@
"""Claude Code tools the Pydantic AI gh-aw shim exposes to the agent.
Each tool lives in its own module so individual implementations can be
swapped without touching the registry or the other tools. The toolset
builder below is the public surface the main shim consumes.
To add a new tool: drop `mytool.py` next to this file exporting one
callable, then add the `(name, callable, description)` row in
`_BASE_TOOLS`.
To replace a tool: edit the matching `<tool>.py` file. The signature is
what gh-aw / Claude pass; the docstring becomes the tool's description.
Two tools are *not* in this package:
- **WebFetch** — registered as a `pydantic_ai.capabilities.NativeTool`
wrapping `pydantic_ai.native_tools.WebFetchTool`. The model fetches
server-side through Anthropic's native web-fetch capability.
- **Task** — sub-agent dispatcher; lives in the main shim because it
needs the shim-level `Agent` factory, the `INSTRUCTIONS` /
`SUBAGENT_INSTRUCTIONS` / `RUN_TRIGGER` constants, and the
event-stream handler. Pass it to `build_claude_code_toolset(task=...)` to
add it as a regular tool.
"""
from collections.abc import Awaitable, Callable
from typing import TypeAlias
from pydantic_ai import RunContext
from pydantic_ai.tools import Tool
from pydantic_ai.toolsets import FunctionToolset
from .bash import bash
from .edit import edit_file
from .exit_plan_mode import exit_plan_mode
from .glob import glob_search
from .grep import grep
from .list_dir import list_dir
from .multi_edit import multi_edit
from .read import read_file
from .todo_write import todo_write
from .write import write_file
# Claude Code tools are callables returning `str`, sync or async: the
# harness-backed tools (`Bash`, `Read`, `Write`, `Edit`, `Grep`, `Glob`, `LS`
# awaiting `ShellToolset` / `FileSystemToolset`, and `TodoWrite` awaiting the
# `planning` capability) are async, while the remaining ones (`MultiEdit`,
# `ExitPlanMode`) stay sync.
# Their argument signatures vary by tool (Claude's `Bash` takes
# `(command, timeout?)`, `MultiEdit` takes `(file_path, edits)`, etc.), so the
# precise per-tool shape is enforced at the tool's own definition site — at the
# registry layer the meaningful contract is "callable that returns (or awaits)
# a string the model can read".
ClaudeCodeToolFn: TypeAlias = Callable[..., str | Awaitable[str]]
# `Task` is async like the harness-backed file/shell tools, but unlike them its
# signature is fully pinned here (it takes a `RunContext`) so that consumers of
# `build_claude_code_toolset(task=...)` pass a compatible callable.
TaskCallable: TypeAlias = Callable[[RunContext[object], str, str], Awaitable[str]]
__all__ = [
'MUTATING_TOOLS',
'CLAUDE_CODE_TOOL_NAMES',
'READ_ONLY_SUBAGENT_TOOLS',
'bash',
'build_claude_code_toolset',
'edit_file',
'exit_plan_mode',
'glob_search',
'grep',
'list_dir',
'multi_edit',
'read_file',
'todo_write',
'write_file',
]
# Claude tool name → (callable, one-line description). The function names
# stay idiomatic snake_case; pydantic-ai's `Tool` exposes them under the
# Claude names so the model sees the Claude Code surface it was trained on.
_BASE_TOOLS: tuple[tuple[str, ClaudeCodeToolFn, str], ...] = (
('Bash', bash, 'Run a shell command in the repository workspace.'),
('Read', read_file, 'Read a UTF-8 text file (optional line offset/limit).'),
('Write', write_file, 'Create or overwrite a workspace text file.'),
('Edit', edit_file, 'Replace a string in a workspace file.'),
('MultiEdit', multi_edit, 'Apply multiple string replacements to one file atomically.'),
('Grep', grep, 'Recursively regex-search workspace files.'),
('Glob', glob_search, 'List workspace paths matching a glob pattern.'),
('LS', list_dir, "List a workspace directory's entries."),
('TodoWrite', todo_write, "Record the agent's task checklist."),
('ExitPlanMode', exit_plan_mode, 'Signal the end of planning and proceed.'),
)
# Claude Code tool names the shim implements as Python callables.
# Excludes `WebFetch` (a `NativeTool` capability) and `Task` (registered by
# the main shim via `build_claude_code_toolset(task=...)`). Kept as a separate
# tuple for tests / introspection that just need the name list.
CLAUDE_CODE_TOOL_NAMES: tuple[str, ...] = tuple(name for name, _, _ in _BASE_TOOLS)
# Tools that mutate the workspace — withheld in `plan` permission mode.
MUTATING_TOOLS = frozenset({'Bash', 'Write', 'Edit', 'MultiEdit'})
# Tools handed to read-only `Task` sub-agents — strictly non-mutating and
# excluding `Task` itself to prevent recursive sub-agent spawning.
# `WebFetch` is wired separately as a `NativeTool` capability.
READ_ONLY_SUBAGENT_TOOLS = frozenset({'Read', 'Grep', 'Glob', 'LS', 'TodoWrite', 'ExitPlanMode'})
def build_claude_code_toolset(*, task: TaskCallable | None = None) -> FunctionToolset[object]:
"""Build the shim's Claude Code tool `FunctionToolset`.
Pass `task=` to register the sub-agent dispatcher as an additional
tool named `Task`. Sub-agent toolsets call this with `task=None` so
sub-agents can't spawn their own sub-agents.
Filter for permission mode / allow-list with `.filtered(predicate)`
on the returned toolset (see `select_claude_code_toolset` in the main
shim).
"""
tools: list[Tool[object]] = [Tool(fn, name=name, description=desc) for name, fn, desc in _BASE_TOOLS]
if task is not None:
tools.append(
Tool(
task,
name='Task',
description='Dispatch a read-only sub-agent to investigate a focused task and return its findings.',
)
)
return FunctionToolset(tools=tools)
@@ -0,0 +1,14 @@
"""Entry point for `python -m pydantic_ai_gh_aw_shim` / `runpy.run_module`.
All shim logic lives in `cli.py` (which tests import directly). Keeping
this file a one-call stub avoids the `runpy.run_module(..., run_name="__main__")`
+ PEP-563 corner case where pydantic-ai's runtime annotation inspection
can't find `RunContext` in a module loaded under `__name__ == "__main__"`.
"""
import sys
from .cli import main
if __name__ == '__main__':
sys.exit(main())
@@ -0,0 +1,91 @@
"""Harness-backed toolsets that execute the Claude file/shell tools.
The Claude-named tool callables in this package (`Bash`, `Read`, `Write`,
`Edit`) keep their Claude Code signatures but delegate the actual work to
pydantic-ai-harness's `ShellToolset` and `FileSystemToolset`. The harness owns
the parts that were previously hand-rolled here: subprocess execution and
output truncation, path containment, symlink resolution before access, and
binary-file detection. The shim keeps only the thin signature adapters.
The toolsets are built per call so `workspace()` (and a test's
`GITHUB_WORKSPACE`) is read live; construction is cheap. They are used by
calling their methods directly, not by registering them on an agent, so the
agent still sees exactly the Claude tool surface gh-aw expects.
"""
import os
from pathlib import Path
from pydantic_ai_harness.filesystem import FileSystemToolset
from pydantic_ai_harness.shell import ShellToolset
from .shared import MAX_TOOL_OUTPUT, workspace
# Standard Unix binary locations prepended to PATH so `rg`, `make`, `git`, and
# `uv` are reachable even when the AWF sandbox starts with a minimal inherited
# PATH. Moved here from the old hand-rolled `bash` tool.
_STANDARD_PATHS = [
'/opt/hostedtoolcache/gh-aw-tools/current/x64/bin', # rg + uv (install-sandbox-tools.sh)
'/tmp/gh-aw/bin', # fallback; launcher lives here too
'/usr/local/bin',
'/usr/bin',
'/bin',
'/usr/local/sbin',
'/usr/sbin',
'/sbin',
]
# Claude's `Bash` timeout contract: default 120s, hard-capped at 600s.
BASH_DEFAULT_TIMEOUT = 120
BASH_MAX_TIMEOUT = 600
def augmented_env() -> dict[str, str]:
"""The process environment with the standard tool paths prepended to PATH."""
env = dict(os.environ)
current = env.get('PATH', '')
existing = set(current.split(':'))
extra = ':'.join(p for p in _STANDARD_PATHS if p not in existing)
env['PATH'] = f'{extra}:{current}' if extra else current
return env
def filesystem() -> FileSystemToolset[None]:
"""`FileSystemToolset` rooted at the live workspace.
`protected_patterns=[]` keeps the prior behavior of allowing writes anywhere
under the workspace. The harness still enforces containment (no path escapes
the workspace root) and resolves symlinks before access -- a change from the
old tools, which resolved any absolute path. For gh-aw the agent operates
within `$GITHUB_WORKSPACE`, so containment to that root is the intended scope.
"""
return FileSystemToolset[None](
root_dir=Path(workspace()),
allowed_patterns=[],
denied_patterns=[],
protected_patterns=[],
max_read_lines=2000,
max_search_results=1000,
max_find_results=1000,
)
def shell() -> ShellToolset[None]:
"""`ShellToolset` rooted at the live workspace, PATH augmented for the sandbox.
Command/operator denylists are left empty to preserve the old `Bash` tool's
"run anything" contract; the AWF sandbox is the security boundary. Output is
capped at `MAX_TOOL_OUTPUT`, keeping the tail (where errors and exit info land).
"""
return ShellToolset[None](
cwd=Path(workspace()),
allowed_commands=[],
denied_commands=[],
denied_operators=[],
default_timeout=float(BASH_DEFAULT_TIMEOUT),
max_output_chars=MAX_TOOL_OUTPUT,
persist_cwd=False,
allow_interactive=True,
env=augmented_env(),
denied_env_patterns=[],
)
@@ -0,0 +1,36 @@
"""Claude's `Bash` tool -- run a shell command in the workspace.
Backed by pydantic-ai-harness's `ShellToolset`, which handles subprocess
execution, the per-command timeout, and tail-keeping output truncation. The
Claude `Bash` signature (`command`, optional `timeout` in seconds) and the
sandbox PATH augmentation are preserved by the adapter.
"""
from pydantic_ai.exceptions import ModelRetry
from ._backends import BASH_DEFAULT_TIMEOUT, BASH_MAX_TIMEOUT, shell
async def bash(command: str, timeout: int | None = None) -> str:
"""Run a shell command in the repository workspace.
Returns the command's labeled stdout/stderr (truncated). `timeout` is in
seconds (default 120, capped at 600).
"""
secs = BASH_DEFAULT_TIMEOUT if not timeout or timeout <= 0 else min(int(timeout), BASH_MAX_TIMEOUT)
try:
out = await shell().run_command(command, timeout_seconds=float(secs))
except (ModelRetry, OSError) as exc:
# ModelRetry: the harness blocked the command; the shim's tools have always
# surfaced such conditions as a returned error string rather than a
# model-facing retry. OSError: subprocess startup failed (e.g. the
# workspace cwd does not exist) -- the harness doesn't convert it, so catch
# it here rather than let it abort the whole run.
return f'error: {exc}'
# On timeout the harness *returns* a `[Command timed out after Ns]` sentinel
# rather than raising. The old tool surfaced timeouts as an `error:` string
# (and `Grep` already wraps the same sentinel), so do the same here instead
# of handing the model an unprefixed result it might read as success.
if out.startswith('[Command timed out'):
return f'error: {out}'
return out
@@ -0,0 +1,944 @@
r"""Pydantic AI gh-aw shim — Claude Code CLI compatibility for gh-aw.
gh-aw runs the agent engine like the Claude Code CLI:
<command> --print --no-chrome --allowed-tools '<csv>' --debug-file <path> \\
--verbose --permission-mode <mode> --output-format stream-json \\
--mcp-config <mcp-servers.json> --prompt-file <prompt.txt> \\
[--model <model>] "<rendered prompt>"
With `engine.command` set, `<command>` is this shim. It speaks Claude
Code's argv, recovers the prompt, builds a `pydantic-ai` agent backed by
the gh-aw-injected Anthropic-compatible proxy, exposes Claude-named
tools plus gh-aw's MCP servers (GitHub + the `safeoutputs` write-sink),
enforces gh-aw's `--allowed-tools` allow-list, and emits Claude-compatible
`stream-json` so gh-aw's log parser and token accounting keep working.
Like Claude Code itself, the shim only talks to Anthropic-shape APIs
(`ANTHROPIC_BASE_URL` → real Anthropic, MiniMax's Anthropic-compatible
endpoint, etc.). No OpenAI path — the workflow's `engine.id: claude`
contract is Anthropic-shape end to end.
Credentials note: under gh-aw the real API key is *excluded* from the
agent container (`awf --exclude-env ANTHROPIC_API_KEY`). The AWF
api-proxy injects it transparently; the shim only ever sends a
placeholder bearer to the proxy base URL — never a real upstream key.
This module is loaded as the `pydantic_ai_gh_aw_shim.cli` submodule;
`__main__.py` is a 3-line entry stub that calls `cli.main()`. Tests
import this module directly (`from pydantic_ai_gh_aw_shim import cli`),
which is why the runner stub doesn't live in `__main__.py` — running it
under `runpy.run_module(..., run_name="__main__")` plus PEP-563
annotations breaks pydantic-ai's `takes_run_context` detection.
"""
import argparse
import asyncio
import dataclasses
import json
import logging
import os
import pathlib
import sys
import time
import uuid
from collections.abc import AsyncIterable, Awaitable, Callable, Mapping, Sequence
from dataclasses import dataclass
from typing import TypeAlias, cast
import httpx
import logfire
from anthropic import AsyncAnthropic
from pydantic import ValidationError
from pydantic_ai import Agent, RunContext
from pydantic_ai.capabilities import NativeTool, ProcessEventStream, ProcessHistory
from pydantic_ai.mcp import load_mcp_toolsets
from pydantic_ai.messages import (
AgentStreamEvent,
ModelMessage,
ModelRequest,
ModelRequestPart,
ModelResponse,
ModelResponsePart,
NativeToolCallPart,
NativeToolSearchCallPart,
RetryPromptPart,
ToolCallEvent,
ToolCallPart,
ToolResultEvent,
ToolReturnPart,
ToolSearchCallPart,
UserPromptPart,
)
from pydantic_ai.models import Model
from pydantic_ai.models.anthropic import AnthropicModel
from pydantic_ai.native_tools import WebFetchTool
from pydantic_ai.providers.anthropic import AnthropicProvider
from pydantic_ai.tools import ToolDefinition
from pydantic_ai.toolsets import AbstractToolset, PrefixedToolset
from pydantic_ai.usage import RunUsage, UsageLimits
from . import (
CLAUDE_CODE_TOOL_NAMES,
MUTATING_TOOLS,
READ_ONLY_SUBAGENT_TOOLS,
build_claude_code_toolset,
)
from .shared import logger, reset_context_state
# Type aliases for the public surface — the shim runs `None`-deps agents
# throughout, so every `RunContext` is concretely `RunContext[object]`.
MessagePart: TypeAlias = ModelRequestPart | ModelResponsePart
ToolPredicate: TypeAlias = Callable[[RunContext[object], ToolDefinition], bool | Awaitable[bool]]
TaskCallable: TypeAlias = Callable[[RunContext[object], str, str], Awaitable[str]]
# Placeholder bearer token sent to the AWF api-proxy. The proxy strips this
# header and injects the real `ANTHROPIC_API_KEY` on the outbound wire — so
# the agent container never sees the real key. Sent verbatim only when no
# `ANTHROPIC_AUTH_TOKEN` / `ANTHROPIC_API_KEY` env is provided locally.
PROXY_BEARER_PLACEHOLDER = 'gh-aw-proxy-injected'
def _anthropic_native_capabilities() -> list[NativeTool]:
"""`NativeTool(WebFetchTool())` for real Anthropic only.
Anthropic-compatible endpoints (MiniMax, etc.) reject the
`web_fetch_20250910` server-side tool with `invalid_request_error
(2013)` because they don't implement Anthropic's server-side tool
types. Detect via `ANTHROPIC_BASE_URL` — empty/unset means the
Anthropic SDK default (real Anthropic).
"""
base_url = os.environ.get('ANTHROPIC_BASE_URL', '')
if not base_url or 'api.anthropic.com' in base_url:
return [NativeTool(WebFetchTool())]
return []
# pydantic-ai's built-in request_limit default of 50 is too low for the
# deep multi-step workflows here; gh-aw's api-proxy still caps the run.
REQUEST_LIMIT = 200
SUBAGENT_REQUEST_LIMIT = 75
# Per-request HTTP timeout for every LLM call. The read timeout is the
# critical one: MiniMax's proxy can hold a streaming connection open without
# sending data. 5 min is generous enough for large generations but prevents
# indefinite hangs. SDK-level retries cover transient 429/5xx before raising.
_LLM_TIMEOUT = httpx.Timeout(timeout=120.0, connect=10.0)
_LLM_MAX_RETRIES = 4
# Wall-clock caps (seconds). These are last-resort guards on top of the
# per-request timeout so a burst of slow requests can't accumulate forever.
RUN_TIMEOUT_SECS = 28 * 60 # 28 min — just under the 30 min gh-aw job cap
SUBAGENT_TIMEOUT_SECS = 15 * 60 # 15 min per Task sub-agent
COMPACTION_TIMEOUT_SECS = 120 # 2 min for the compaction summariser call
# Static prefix for `Agent(instructions=[INSTRUCTIONS, prompt])`. Sequence
# form lets Anthropic's prompt-prefix cache hit `INSTRUCTIONS` across runs.
INSTRUCTIONS = (
'## Parallel tool calls\n\n'
'The model supports parallel tool calls. When multiple reads, searches, or '
"lookups are independent — meaning one doesn't need another's result — "
'issue them all in the same response. They execute concurrently. Only '
'chain sequentially when one call genuinely needs a previous result.\n\n'
'## File reading\n\n'
'Read files in large ranges (500+ lines per call). MAX_TOOL_OUTPUT is '
'50 000 chars so most Python source files fit in one or two reads. '
'Avoid reading 3080 lines at a time.\n\n'
'## Search tools\n\n'
'Use the native Grep and Glob tools for codebase search. '
'`rg` and `uv` are also available as plain commands via Bash.\n\n'
'## Dev environment\n\n'
'The repo is checked out at $GITHUB_WORKSPACE. Dev dependencies are NOT '
'pre-installed — run `make install` once before using pytest, ruff, or '
'pyright. Prefer `uv run pytest <test_file>` over a bare `pytest` call; '
'uv handles the virtual env automatically.\n\n'
'## GitHub issue search\n\n'
'The GitHub toolset runs in gh-proxy mode: there are NO `mcp__github__*` '
'tools, and the /search/issues endpoint (`gh issue list --search`, '
'`gh search issues`) returns HTTP 403 via the AWF firewall proxy. The '
'issue-list endpoint IS allowed, including its server-side `?labels=` '
'filter. When the sweep files under a dedicated label, prefer a narrow label '
"query (`gh api 'repos/pydantic/pydantic-ai/issues?state=open&labels=<label>&per_page=100' "
"--jq '.[] | select(.pull_request == null) | {number, title}'`); if it has no "
'dedicated label or the filter is inconclusive, widen to a full open-issue scan '
"(`gh api --paginate 'repos/pydantic/pydantic-ai/issues?state=open&per_page=100' "
"--jq '.[] | select(.pull_request == null) | {number, title, labels: [.labels[].name]}'`). "
'`select(.pull_request == null)` drops PRs, which the issues endpoint also returns.'
)
# The real task spec rides in `instructions=`; the user message is a trigger.
RUN_TRIGGER = 'Begin the task per the instructions above.'
SUBAGENT_INSTRUCTIONS = (
'You are a focused, read-only sub-agent. You can read files, search the '
'codebase, and fetch web content, but you cannot modify the workspace or '
'shell out. Investigate the task you were given and return a concise, '
'evidence-grounded answer to your caller — do not try to act on it.'
)
# History compaction (pydantic-ai `ProcessHistory` capability). Two stages
# inside one callback: a cheap dedup+truncate trim, then an LLM summary as
# fallback. `Agent(instructions=...)` is re-applied on every request, so
# the workflow prompt is never in the message list and never compacted.
# ~100k tokens at 4 chars/token = half of a 200k window.
COMPACTION_TRIGGER_CHARS = 400_000
COMPACTION_KEEP_RECENT = 10
TOOL_RESULT_HEAD_TAIL_CHARS = 4_000
TOOL_RESULT_TRIM_THRESHOLD = 10_000
COMPACTION_TRANSCRIPT_MAX_CHARS = 80_000
COMPACTION_SUMMARY_INSTRUCTIONS = (
'Summarise the agent transcript below for resumption in a fresh '
'context window. Produce a structured brief, not free prose. Use this '
'exact section layout, omitting any section that is empty:\n\n'
'## Goal\n'
'One short paragraph: what the agent was asked to do.\n\n'
'## Files inspected\n'
'- `<full/path>`: one-line note on what was found there.\n\n'
'## Commands run\n'
'- `<command>`: outcome in one line.\n\n'
'## Errors encountered\n'
'Verbatim error messages or unexpected behaviour, with the file or '
'command that triggered each.\n\n'
'## Decisions and approaches\n'
'- Concrete decisions with reasoning. Include approaches already tried '
'that did **not** work, so they are not re-attempted.\n\n'
'## Open questions\n'
'- Anything still unresolved.\n\n'
'## Next step\n'
'The single most likely next action.\n\n'
'Preserve specifics (paths, identifiers, exact strings) over prose. '
'Respond with text only — do not call any tools.'
)
def _part_text(part: MessagePart) -> str:
"""Best-effort text rendering of any pydantic-ai message part."""
if isinstance(part, (ToolCallPart, NativeToolCallPart, ToolSearchCallPart, NativeToolSearchCallPart)):
return f'{part.tool_name}({part.args_as_dict()!r})'
return str(part.content)
def _render_messages_for_summary(messages: list[ModelMessage]) -> str:
"""Render a slice of pydantic-ai messages into a compact transcript."""
out: list[str] = []
for m in messages:
kind = 'user' if isinstance(m, ModelRequest) else 'assistant'
for part in m.parts:
out.append(f'[{kind}/{type(part).__name__}] {_part_text(part)[:1500]}')
return '\n'.join(out)
def _history_size_chars(messages: list[ModelMessage]) -> int:
"""Char-count proxy for token cost — used as the compaction trigger."""
return sum(len(_part_text(part)) for m in messages for part in m.parts)
def _head_tail(text: str, side: int) -> str:
"""Keep the first and last `side` chars, mark the elided middle."""
skipped = len(text) - side * 2
return f'{text[:side]}\n…[trimmed {skipped} chars]…\n{text[-side:]}'
def _superseded_read_calls(messages: list[ModelMessage]) -> tuple[set[str], dict[str, str]]:
"""For each `Read` call, key on (path, offset, limit); older calls with the same key are superseded."""
label_by_call_id: dict[str, str] = {}
latest_for_args: dict[tuple[str, object, object], str] = {}
superseded: set[str] = set()
for m in messages:
for p in m.parts:
if not (isinstance(p, ToolCallPart) and p.tool_name == 'Read'):
continue
args = p.args_as_dict()
if not isinstance(args, dict):
continue
path = args.get('file_path')
if not isinstance(path, str):
continue
offset, limit = args.get('offset'), args.get('limit')
label = path if offset is None and limit is None else f'{path}[offset={offset!r}, limit={limit!r}]'
label_by_call_id[p.tool_call_id] = label
key = (path, offset, limit)
prior = latest_for_args.get(key)
if prior is not None:
superseded.add(prior)
latest_for_args[key] = p.tool_call_id
return superseded, label_by_call_id
def _trim_tool_results(messages: list[ModelMessage]) -> list[ModelMessage]:
"""Dedupe re-reads of the same file slice and head/tail-truncate oversized older tool returns."""
if len(messages) <= COMPACTION_KEEP_RECENT:
return messages
superseded, label_by_call_id = _superseded_read_calls(messages)
tail_start = len(messages) - COMPACTION_KEEP_RECENT
out: list[ModelMessage] = []
dedup_count = truncate_count = bytes_saved = 0
def _rewrite(part: ModelRequestPart | ModelResponsePart) -> ModelRequestPart | ModelResponsePart:
nonlocal dedup_count, truncate_count, bytes_saved
if not isinstance(part, ToolReturnPart):
return part
if part.tool_call_id in superseded:
new_content = f'[superseded read: {label_by_call_id[part.tool_call_id]} — see later read with same args]'
bytes_saved += len(str(part.content)) - len(new_content)
dedup_count += 1
return dataclasses.replace(part, content=new_content)
content = str(part.content)
if len(content) > TOOL_RESULT_TRIM_THRESHOLD:
new_content = _head_tail(content, TOOL_RESULT_HEAD_TAIL_CHARS)
bytes_saved += len(content) - len(new_content)
truncate_count += 1
return dataclasses.replace(part, content=new_content)
return part
for idx, m in enumerate(messages):
if idx >= tail_start:
out.append(m)
continue
new_parts = [_rewrite(p) for p in m.parts]
out.append(dataclasses.replace(m, parts=new_parts) if new_parts != list(m.parts) else m)
if dedup_count or truncate_count:
logger.info(
'trim: deduped %d superseded read(s), truncated %d oversized result(s), saved %d chars',
dedup_count,
truncate_count,
bytes_saved,
)
emit(
{
'type': 'system',
'subtype': 'compaction_trim',
'deduped_reads': dedup_count,
'truncated_results': truncate_count,
'chars_saved': bytes_saved,
}
)
return out
return messages
_SYNTHETIC_SUMMARY_TAG = '[compacted history]'
def _is_synthetic_summary(message: ModelMessage) -> bool:
"""A `ModelRequest` we synthesised in a prior `_compact_history` round."""
if not isinstance(message, ModelRequest):
return False
parts = message.parts
return (
len(parts) == 1
and isinstance(parts[0], UserPromptPart)
and str(parts[0].content).startswith(_SYNTHETIC_SUMMARY_TAG)
)
async def _compact_history(ctx: RunContext[object], messages: list[ModelMessage]) -> list[ModelMessage]:
"""Cheap trim first; LLM-summarise the middle as fallback if still over budget."""
if len(messages) <= COMPACTION_KEEP_RECENT:
return messages
trimmed = _trim_tool_results(messages)
size = _history_size_chars(trimmed)
if size < COMPACTION_TRIGGER_CHARS:
return trimmed
middle = trimmed[:-COMPACTION_KEEP_RECENT]
tail = trimmed[-COMPACTION_KEEP_RECENT:]
transcript = _render_messages_for_summary(middle)
logger.info(
'compaction summary firing: %d chars / %d messages -> summarising %d middle, keeping last %d',
size,
len(trimmed),
len(middle),
COMPACTION_KEEP_RECENT,
)
emit(
{
'type': 'system',
'subtype': 'compaction_summary_start',
'history_chars': size,
'history_messages': len(trimmed),
'middle_messages': len(middle),
'keep_recent': COMPACTION_KEEP_RECENT,
}
)
# Preserve any earlier-round synthetic at the head of the middle so a
# fallback (`return [prior_synthetic, *tail]`) doesn't silently forget
# the entire run's compacted history.
prior_synthetic = middle[0] if middle and _is_synthetic_summary(middle[0]) else None
# Fresh `RunUsage` so `request_limit=2` bounds the summariser, not
# (parent + summariser). Merge the totals back regardless of outcome.
sub_usage = RunUsage()
try:
r = await asyncio.wait_for(
Agent(ctx.model, instructions=COMPACTION_SUMMARY_INSTRUCTIONS).run(
f'Transcript to summarise:\n\n{transcript[:COMPACTION_TRANSCRIPT_MAX_CHARS]}',
usage_limits=UsageLimits(request_limit=2),
usage=sub_usage,
),
timeout=COMPACTION_TIMEOUT_SECS,
)
summary = str(r.output or '').strip() or '(empty summary)'
except Exception as exc:
# Well-handled fallback: the run continues on the trimmed history, so the
# stack is kept available (`exc_info`) without escalating the log level.
# A bare `TimeoutError` stringifies to '' — fall back to the type name so
# the emitted `error` is never empty.
ctx.usage.incr(sub_usage)
detail = f'{type(exc).__name__}: {exc}' if str(exc) else type(exc).__name__
logger.warning('compaction summarisation failed (%s); falling back', detail, exc_info=True)
emit({'type': 'system', 'subtype': 'compaction_summary_failed', 'error': detail})
return [prior_synthetic, *tail] if prior_synthetic else tail
ctx.usage.incr(sub_usage)
# If the summariser produces output larger than the middle it's replacing,
# the next compaction round would trip on the same too-large synthetic
# and never converge — fall back to the prior synthetic + tail.
middle_size = _history_size_chars(middle)
if len(summary) >= middle_size:
logger.info('compaction summary discarded (%d >= %d chars); falling back', len(summary), middle_size)
emit(
{
'type': 'system',
'subtype': 'compaction_summary_discarded',
'summary_chars': len(summary),
'middle_chars': middle_size,
}
)
return [prior_synthetic, *tail] if prior_synthetic else tail
logger.info(
'compaction summary done: %d middle messages (%d chars) -> %d-char summary',
len(middle),
middle_size,
len(summary),
)
emit(
{
'type': 'system',
'subtype': 'compaction_summary_done',
'middle_messages': len(middle),
'middle_chars': middle_size,
'summary_chars': len(summary),
'input_tokens': sub_usage.input_tokens,
'output_tokens': sub_usage.output_tokens,
}
)
synthetic = ModelRequest(parts=[UserPromptPart(content=f'{_SYNTHETIC_SUMMARY_TAG}\n{summary}')])
return [synthetic, *tail]
@dataclass(slots=True)
class Args:
"""The subset of Claude Code's CLI surface the shim acts on."""
model: str | None = None
mcp_config: str | None = None
prompt_file: str | None = None
prompt_positional: str | None = None
# None = flag absent (local/dev: no restriction). A set = enforce it.
allowed_tools: frozenset[str] | None = None
permission_mode: str | None = None
def _split_allowed_tools(value: str | None) -> frozenset[str] | None:
"""Parse Claude's `--allowed-tools` CSV into base tool names.
Entries may carry a permission scope, e.g. `Edit(/tmp/*)` or
`Bash(git:*)` — only the base name gates availability here, so the
parenthesised scope is stripped. Returns `None` when the flag is absent
so non-gh-aw/local runs keep every tool.
"""
if value is None:
return None
names: set[str] = set()
for raw in value.split(','):
entry = raw.strip()
if not entry:
continue
names.add(entry.split('(', 1)[0].strip())
return frozenset(names)
def parse_args(argv: Sequence[str]) -> Args:
"""Parse Claude Code's CLI surface into `Args`, tolerating unknown flags so a future Claude flag never breaks the engine."""
p = argparse.ArgumentParser(add_help=False)
p.add_argument('--model')
p.add_argument('--mcp-config')
p.add_argument('--prompt-file')
p.add_argument('--output-format', default='stream-json')
p.add_argument('--allowed-tools')
p.add_argument('--permission-mode')
p.add_argument('--debug-file')
for flag in ('--print', '--no-chrome', '--verbose', '--continue'):
p.add_argument(flag, action='store_true')
known, unknown = p.parse_known_args(list(argv))
# gh-aw appends the rendered prompt as the trailing positional argument.
positionals = [a for a in unknown if not a.startswith('-')]
return Args(
model=known.model,
mcp_config=known.mcp_config,
prompt_file=known.prompt_file,
prompt_positional=positionals[-1] if positionals else None,
allowed_tools=_split_allowed_tools(known.allowed_tools),
permission_mode=known.permission_mode,
)
def resolve_prompt(args: Args) -> str:
"""Prompt precedence: trailing positional -> --prompt-file -> $GH_AW_PROMPT."""
if args.prompt_positional:
return args.prompt_positional
path = args.prompt_file or os.environ.get('GH_AW_PROMPT')
if path and os.path.isfile(path):
return pathlib.Path(path).read_text(encoding='utf-8')
return ''
def build_model(args: Args) -> tuple[Model, str]:
"""Build the `pydantic-ai` model and a human-readable label.
Anthropic-only — the shim behaves like the stock Claude Code CLI:
gh-aw sets `ANTHROPIC_BASE_URL` (its in-cluster transparent proxy)
and the AWF api-proxy injects the real key on outgoing requests.
**Why we construct `AsyncAnthropic` ourselves** instead of letting
`pydantic-ai`'s `AnthropicProvider` auto-configure: gh-aw runs the
agent step in a sandbox that excludes `ANTHROPIC_API_KEY` from the
container env (`awf --exclude-env ANTHROPIC_API_KEY` — a security
measure so the real key never reaches the agent). `pydantic-ai`'s
auto-config requires that env var to be present, so it errors out
under gh-aw. The explicit `AsyncAnthropic(auth_token=...)` path
sends a placeholder bearer that the AWF api-proxy swaps for the
real key on the wire — the same dance the Claude Code CLI does.
This is a gh-aw constraint, not a pydantic-ai one; upstream gh-aw
could lift it by allowing the agent to read the key directly, but
that would break the credential-isolation guarantee.
Model name resolution (in priority order):
1. `--model X` argv flag (from Claude Code's CLI surface).
2. `ANTHROPIC_MODEL` env var (standard Anthropic SDK convention;
gh-aw populates this from the workflow's `engine.model:` field).
3. Fallback default `claude-sonnet-4-6`.
"""
model_name = args.model or os.environ.get('ANTHROPIC_MODEL') or 'claude-sonnet-4-6'
anthropic_base = os.environ.get('ANTHROPIC_BASE_URL')
auth_token = (
os.environ.get('ANTHROPIC_AUTH_TOKEN') or os.environ.get('ANTHROPIC_API_KEY') or PROXY_BEARER_PLACEHOLDER
)
logger.info('anthropic model=%s base_url=%s', model_name, anthropic_base or '(default)')
client = AsyncAnthropic(
auth_token=auth_token,
base_url=anthropic_base,
timeout=_LLM_TIMEOUT,
max_retries=_LLM_MAX_RETRIES,
)
return (
AnthropicModel(model_name, provider=AnthropicProvider(anthropic_client=client)),
f'anthropic:{model_name}',
)
def configure_logging() -> None:
"""Configure stderr logging once, at CLI entry."""
logging.basicConfig(
level=logging.INFO,
format='[pydantic-ai-gh-aw-shim] %(message)s',
stream=sys.stderr,
)
def configure_observability() -> None:
"""Wire pydantic-ai + httpx + mcp instrumentation to Logfire/OTLP if configured."""
write_token = os.environ.get('LOGFIRE_WRITE_TOKEN') or os.environ.get('LOGFIRE_TOKEN')
if not (os.environ.get('OTEL_EXPORTER_OTLP_ENDPOINT') or os.environ.get('GH_AW_OTLP_ENDPOINTS') or write_token):
return
try:
logfire.configure(
service_name=os.environ.get('OTEL_SERVICE_NAME', 'gh-aw'),
send_to_logfire='if-token-present',
console=False,
token=write_token or None,
)
logfire.instrument_pydantic_ai(include_content=True, include_binary_content=True)
logfire.instrument_httpx(capture_all=True)
logfire.instrument_mcp()
logger.info('Logfire/OTLP instrumentation enabled (pydantic_ai + httpx + mcp)')
except Exception as exc:
logger.warning('observability disabled: %r', exc)
def _mcp_tool_allowed(server: str, allowed: frozenset[str]) -> ToolPredicate:
"""Allow-list predicate matching gh-aw's `mcp__<server>__<tool>` form (or `mcp__<server>` wildcard)."""
server_wildcard = f'mcp__{server}' in allowed
def predicate(_ctx: RunContext[object], tool_def: ToolDefinition) -> bool:
return server_wildcard or tool_def.name in allowed
return predicate
def _apply_claude_mcp_prefix(entry: AbstractToolset[object]) -> AbstractToolset[object]:
"""Swap the default `<server>_<tool>` prefix for Claude Code's `mcp__<server>__<tool>` wire form.
The trailing `_` combines with `PrefixedToolset`'s `_` separator to
yield the doubled underscores gh-aw and Claude were trained on.
"""
if not isinstance(entry, PrefixedToolset):
return entry
return dataclasses.replace(entry, prefix=f'mcp__{entry.prefix}_')
def build_mcp_servers(args: Args) -> list[AbstractToolset[object]]:
"""Load gh-aw's MCP config, re-prefix to Claude Code wire format, and apply the allow-list filter."""
path = args.mcp_config or os.environ.get('GH_AW_MCP_CONFIG')
if not path or not os.path.isfile(path):
logger.info('no MCP config present — running without external tools')
return []
try:
loaded = load_mcp_toolsets(path)
# `repr` is sufficient diagnostically here (a `ValidationError` already
# enumerates the bad fields, `FileNotFoundError` names the path), so no
# traceback — but returning `[]` drops the *entire* GitHub/safeoutputs tool
# surface, a drastic behaviour change, so log it at `error` to make a run
# that silently lost its tools obvious in the artifact.
except FileNotFoundError as exc:
logger.error('MCP config %r missing (%r) — agent will run with NO external tools', path, exc)
return []
except (ValidationError, ValueError) as exc:
logger.error('MCP config %r is malformed (%r) — agent will run with NO external tools', path, exc)
return []
servers: list[AbstractToolset[object]] = []
for entry in loaded:
name = (entry.wrapped.id if isinstance(entry, PrefixedToolset) else entry.id) or '<unnamed>'
toolset = _apply_claude_mcp_prefix(cast('AbstractToolset[object]', entry))
if args.allowed_tools is not None:
toolset = toolset.filtered(_mcp_tool_allowed(name, args.allowed_tools))
logger.info('registered MCP server %r (allow-list filtered)', name)
else:
logger.info('registered MCP server %r (no allow-list)', name)
servers.append(toolset)
return servers
def _claude_code_tool_predicate(allowed: frozenset[str] | None, permission_mode: str | None) -> ToolPredicate:
"""Allow-list + `plan`-mode filter for the Claude Code toolset."""
plan = permission_mode == 'plan'
def predicate(_ctx: RunContext[object], tool_def: ToolDefinition) -> bool:
name = tool_def.name
if allowed is not None and name not in allowed:
return False
if plan and name in MUTATING_TOOLS:
return False
return True
return predicate
def select_claude_code_toolset(
allowed: frozenset[str] | None,
permission_mode: str | None,
*,
task: TaskCallable | None,
) -> AbstractToolset[object]:
"""Build the Claude Code toolset; `task=None` for sub-agents so they can't recurse."""
return build_claude_code_toolset(task=task).filtered(_claude_code_tool_predicate(allowed, permission_mode))
# --------------------------------------------------------------------------- #
# Claude-compatible stream-json output
# --------------------------------------------------------------------------- #
def emit(obj: Mapping[str, object]) -> None:
"""Write one Claude-style stream-json line to stdout."""
sys.stdout.write(json.dumps(obj) + '\n')
sys.stdout.flush()
def emit_result(
text: str,
usage: RunUsage | None,
session_id: str,
is_error: bool = False,
num_turns: int = 1,
duration_ms: int = 0,
) -> None:
"""Emit the Claude Code stream-json `result` line gh-aw parses for success + token totals."""
if usage is None:
token_usage = {
'input_tokens': 0,
'output_tokens': 0,
'cache_creation_input_tokens': 0,
'cache_read_input_tokens': 0,
}
else:
token_usage = {
'input_tokens': usage.input_tokens,
'output_tokens': usage.output_tokens,
'cache_creation_input_tokens': usage.cache_write_tokens,
'cache_read_input_tokens': usage.cache_read_tokens,
}
emit(
{
'type': 'result',
'subtype': 'error' if is_error else 'success',
'is_error': is_error,
'result': text,
'session_id': session_id,
'num_turns': num_turns,
'duration_ms': duration_ms,
'total_cost_usd': 0,
'usage': token_usage,
}
)
# Live tool-call / tool-result streaming for gh-aw's log parser. Result
# content is truncated for the stream view only — the model sees the full
# result via the message history.
MAX_LIVE_TOOL_RESULT_CHARS = 100
async def _stream_events(_ctx: RunContext[object], events: AsyncIterable[AgentStreamEvent]) -> None:
"""Emit tool_use / tool_result stream-json as events fire."""
async for event in events:
if isinstance(event, ToolCallEvent):
emit(
{
'type': 'assistant',
'message': {
'role': 'assistant',
'content': [
{
'type': 'tool_use',
'id': event.part.tool_call_id,
'name': event.part.tool_name,
'input': event.part.args_as_dict(),
}
],
},
}
)
logger.info('tool_use: %s', event.part.tool_name)
elif isinstance(event, ToolResultEvent):
# `event.part` is `ToolReturnPart | RetryPromptPart`; the latter
# means the tool result failed validation and pydantic-ai is
# asking the model to retry. Tag it so gh-aw doesn't read it as
# success.
is_retry = isinstance(event.part, RetryPromptPart)
content = str(event.part.content)
if len(content) > MAX_LIVE_TOOL_RESULT_CHARS:
content = (
content[:MAX_LIVE_TOOL_RESULT_CHARS] + f'…[+{len(content) - MAX_LIVE_TOOL_RESULT_CHARS} chars]'
)
emit(
{
'type': 'user',
'message': {
'role': 'user',
'content': [
{
'type': 'tool_result',
'tool_use_id': event.part.tool_call_id,
'content': content,
'is_error': is_retry,
}
],
},
}
)
def count_tool_calls(messages: Sequence[ModelMessage]) -> int:
"""Tally tool calls in the final message history (for the end-of-run log)."""
return sum(1 for m in messages for p in m.parts if isinstance(p, ToolCallPart))
def log_safe_outputs_state() -> None:
"""Log whether anything reached the gh-aw safe-outputs sink."""
path = os.environ.get('GH_AW_SAFE_OUTPUTS')
if not path:
logger.info('GH_AW_SAFE_OUTPUTS not set')
return
try:
data = pathlib.Path(path).read_text(encoding='utf-8')
except OSError as exc:
logger.info('GH_AW_SAFE_OUTPUTS unreadable (%s): %r', path, exc)
return
lines = [ln for ln in data.splitlines() if ln.strip()]
logger.info('GH_AW_SAFE_OUTPUTS=%s entries=%d bytes=%d', path, len(lines), len(data))
for ln in lines[:5]:
logger.info(' safe-output: %s', ln[:300])
async def task(ctx: RunContext[object], description: str, prompt: str) -> str:
"""Claude's `Task` tool: spawn a read-only sub-agent on `ctx.model`."""
logger.info('Task spawn: %s', description[:120])
# Fresh dedupe set per sub-agent — otherwise inheriting the parent's
# `seen` AGENTS.md set would silently hide context the sub-agent needs.
reset_context_state()
sub_toolset = select_claude_code_toolset(READ_ONLY_SUBAGENT_TOOLS, permission_mode=None, task=None)
sub = Agent(
ctx.model,
instructions=[INSTRUCTIONS, SUBAGENT_INSTRUCTIONS, prompt],
toolsets=[sub_toolset],
capabilities=[
*_anthropic_native_capabilities(),
ProcessEventStream(_stream_events),
],
)
# Fresh `RunUsage` so `SUBAGENT_REQUEST_LIMIT` bounds the sub-agent, not
# (parent + sub). Merge the deltas back regardless of success/failure.
sub_usage = RunUsage()
try:
result = await asyncio.wait_for(
sub.run(RUN_TRIGGER, usage_limits=UsageLimits(request_limit=SUBAGENT_REQUEST_LIMIT), usage=sub_usage),
timeout=SUBAGENT_TIMEOUT_SECS,
)
except asyncio.TimeoutError:
# A bare `TimeoutError` stringifies to '' — without an explicit message
# the model (and the log) would see `sub-agent failed:` with no payload.
ctx.usage.incr(sub_usage)
logger.error('sub-agent timed out after %.0f min: %s', SUBAGENT_TIMEOUT_SECS / 60, description[:120])
return f'error: sub-agent timed out after {SUBAGENT_TIMEOUT_SECS // 60}min'
except Exception as exc:
# The parent agent reacts to the returned string, but a sub-agent can hit
# the same nested `ExceptionGroup`/`McpError` as the main run — log the
# full stack so the failure isn't reduced to a one-line repr in the logs.
ctx.usage.incr(sub_usage)
logger.exception('sub-agent failed: %s', description[:120])
return f'error: sub-agent failed: {exc}'
ctx.usage.incr(sub_usage)
logger.info('Task done: +%d sub-requests (run total now %d)', sub_usage.requests, ctx.usage.requests)
return str(result.output or '')
async def _run_with_timeout(
prompt: str,
model: Model,
label: str,
claude_code_toolset: AbstractToolset[object],
mcp_servers: list[AbstractToolset[object]],
session_id: str,
) -> int:
"""Wrap `run()` with the global wall-clock cap and emit a clean result on timeout."""
try:
return await asyncio.wait_for(
run(prompt, model, label, claude_code_toolset, mcp_servers, session_id),
timeout=RUN_TIMEOUT_SECS,
)
except asyncio.TimeoutError:
logger.error('run timed out after %.0f min', RUN_TIMEOUT_SECS / 60)
emit_result(
f'run timed out after {RUN_TIMEOUT_SECS // 60}min',
usage=None,
session_id=session_id,
is_error=True,
)
return 1
async def run(
prompt: str,
model: Model,
label: str,
claude_code_toolset: AbstractToolset[object],
mcp_servers: list[AbstractToolset[object]],
session_id: str,
) -> int:
"""Run one agent turn and emit Claude-shape stream-json. Always emits a `result` line."""
reset_context_state()
agent: Agent[object, str] = Agent(
model,
instructions=[INSTRUCTIONS, prompt],
toolsets=[claude_code_toolset, *mcp_servers],
capabilities=[
*_anthropic_native_capabilities(),
ProcessHistory(_compact_history),
ProcessEventStream(_stream_events),
],
)
limits = UsageLimits(request_limit=REQUEST_LIMIT)
emit({'type': 'system', 'subtype': 'init', 'session_id': session_id, 'model': label})
started = time.perf_counter()
try:
async with agent:
result = await agent.run(RUN_TRIGGER, usage_limits=limits)
except Exception as exc:
# `%r` on an `ExceptionGroup` (e.g. the MCP `TaskGroup` failures seen in
# CI) discards every frame and every nested sub-exception's stack, which
# is what made the original incident so hard to root-cause. `exception()`
# renders the full traceback — and, on 3.11+, each group leaf's stack —
# to stderr, which gh-aw captures into the uploaded `agent-stdio.log`,
# so the run is self-explaining without a re-run. The `result` text
# stays a one-liner because gh-aw parses it.
logger.exception('agent run failed')
emit_result(
f'agent run failed: {exc}',
usage=None,
session_id=session_id,
is_error=True,
duration_ms=round((time.perf_counter() - started) * 1000),
)
return 1
duration_ms = round((time.perf_counter() - started) * 1000)
messages = result.all_messages()
tool_calls = count_tool_calls(messages)
num_turns = sum(isinstance(m, ModelResponse) for m in messages)
logger.info('tool calls observed: %d, turns: %d', tool_calls, num_turns)
text = str(result.output or '')
emit({'type': 'assistant', 'message': {'role': 'assistant', 'content': text}})
emit_result(text, result.usage, session_id, num_turns=num_turns, duration_ms=duration_ms)
log_safe_outputs_state()
return 0
def main() -> int:
"""Entry point. Every failure produces a stream-json `result` line so gh-aw never sees an empty log."""
configure_logging()
session_id = (os.environ.get('GITHUB_RUN_ID') or 'local') + '-' + uuid.uuid4().hex[:8]
try:
args = parse_args(sys.argv[1:])
configure_observability()
prompt = resolve_prompt(args)
if not prompt.strip():
logger.info('empty prompt — nothing to do')
emit_result('empty prompt', usage=None, session_id=session_id, is_error=True)
return 1
model, label = build_model(args)
claude_code_toolset = select_claude_code_toolset(args.allowed_tools, args.permission_mode, task=task)
mcp_servers = build_mcp_servers(args)
logger.info(
'model=%s permission_mode=%s request_limit=%d claude_code_tool_names=%s mcp_servers=%d prompt_chars=%d',
label,
args.permission_mode or '(none)',
REQUEST_LIMIT,
list(CLAUDE_CODE_TOOL_NAMES),
len(mcp_servers),
len(prompt),
)
started = time.time()
rc = asyncio.run(_run_with_timeout(prompt, model, label, claude_code_toolset, mcp_servers, session_id))
logger.info('done in %.1fs rc=%d', time.time() - started, rc)
return rc
except SystemExit as exc:
# `argparse` raises `SystemExit` (not `Exception`) on unknown-flag
# rejection — an expected, clean exit, so a traceback would be noise.
# gh-aw still needs a structured result line.
logger.error('FATAL startup error: %r', exc)
emit_result(f'shim startup failed: {exc}', usage=None, session_id=session_id, is_error=True)
return 1
except Exception as exc:
# A real crash before the agent run (model build, MCP load, …) — dump the
# full stack so a blind FATAL doesn't cost another long investigation.
logger.exception('FATAL startup error')
emit_result(f'shim startup failed: {exc}', usage=None, session_id=session_id, is_error=True)
return 1
@@ -0,0 +1,52 @@
"""Claude's `Edit` tool -- replace a string in a workspace file.
The single-occurrence case is backed by pydantic-ai-harness's
`FileSystemToolset.edit_file`, which requires `old_string` to match exactly once
-- the same uniqueness rule Claude Code's own `Edit` enforces. `replace_all=True`
has no harness equivalent, so it keeps the prior in-place read/replace/write.
"""
from pydantic_ai.exceptions import ModelRetry
from ._backends import filesystem
from .shared import attach_context, resolve
async def edit_file(file_path: str, old_string: str, new_string: str, replace_all: bool = False) -> str:
"""Replace `old_string` with `new_string` in a workspace file.
Replaces the single (unique) occurrence, or every occurrence when `replace_all`.
"""
if replace_all:
return await _replace_all(file_path, old_string, new_string)
try:
result = await filesystem().edit_file(file_path, old_string, new_string)
except (ModelRetry, OSError) as exc:
# The harness only converts a fixed set of errors to `ModelRetry`; a bare
# `OSError` (e.g. `ENAMETOOLONG` while resolving the path) would otherwise
# escape and abort the whole run, where the old tool returned an error.
return f'error: {exc}'
return attach_context(file_path) + result
async def _replace_all(file_path: str, old_string: str, new_string: str) -> str:
"""Replace every occurrence of `old_string`.
The harness `edit_file` rejects non-unique matches, so replace-all stays an
in-place rewrite (as the tool did before it was harness-backed). Workspace
containment is still enforced by preflighting the path through the filesystem
capability's `file_info` -- the same check `Grep` uses -- so this branch
can't escape the workspace root while the single-edit branch can't.
"""
try:
await filesystem().file_info(file_path) # rejects a path that escapes the workspace
p = resolve(file_path)
text = p.read_text(encoding='utf-8')
if old_string not in text:
return 'error: `old_string` not found'
p.write_text(text.replace(old_string, new_string, -1), encoding='utf-8')
except ModelRetry as exc:
return f'error: {exc}'
except OSError as exc:
return f'error: {exc}'
return attach_context(file_path) + f'edited {p}'
@@ -0,0 +1,14 @@
"""Claude's `ExitPlanMode` tool — acknowledge the end of planning."""
from .shared import logger
def exit_plan_mode(plan: str) -> str:
"""Acknowledge the end of planning.
The shim has no interactive plan
review, so this is just a structured ack — the agent continues execution
against the same workspace it was already operating on.
"""
logger.info('ExitPlanMode: %s', plan[:200])
return 'Plan acknowledged — proceeding with execution.'
@@ -0,0 +1,51 @@
"""Claude's `Glob` tool -- list workspace paths matching a glob pattern.
Containment comes from a pydantic-ai-harness `FileSystemToolset.file_info`
preflight on the search `path` (the same check `Grep`/`LS` use) plus a
per-match resolved-path check, but the glob itself is hand-rolled with the stdlib
rather than delegated to `FileSystemToolset.find_files`: the harness walker hides
every dot-prefixed path, which would make `.github/**` -- where gh-aw's own
workflows live -- unmatchable. The Claude `Glob` signature is preserved, matches
are returned relative to the workspace root, and the directory-scoped AGENTS.md /
CLAUDE.md context blocks are still prepended.
"""
import glob as globlib
import os
import pathlib
from pydantic_ai.exceptions import ModelRetry
from ._backends import filesystem
from .shared import attach_context, clip, resolve, workspace
async def glob_search(pattern: str, path: str = '.') -> str:
"""Return workspace paths matching a glob `pattern` (supports `**`)."""
# Claude's `Glob` takes a relative pattern plus a separate `path`; an absolute
# pattern would be joined as-is and could escape the search root, so reject it.
if os.path.isabs(pattern):
return 'error: glob pattern must be relative to the search path'
try:
# `file_info` contains the search `path` (a clear error if it escapes); the
# per-match resolve() + is_relative_to() below then contains the matches,
# dropping anything a `..` pattern or an in-workspace symlink points to
# outside the root (a purely lexical `relative_to` would not catch those).
await filesystem().file_info(path)
base = resolve(path)
ws = pathlib.Path(workspace())
root = ws.resolve()
matches: list[str] = []
for match in globlib.glob(str(base / pattern), recursive=True):
# Resolve only to *decide* containment (collapse `..`, follow symlinks);
# the returned path is the matched name relative to the workspace, so a
# symlink that matched (e.g. `CLAUDE.md` -> `AGENTS.md`) is reported as
# itself rather than silently rewritten to its target.
if pathlib.Path(match).resolve().is_relative_to(root):
matches.append(os.path.relpath(match, ws))
except (ModelRetry, OSError, ValueError) as exc:
# ModelRetry: the containment preflight rejected `path`. OSError (e.g.
# `ENAMETOOLONG`) / ValueError: not recoverable in the harness, so catch
# them here rather than let them abort the whole run.
return f'error: {exc}'
return clip(attach_context(path) + ('\n'.join(sorted(set(matches))) or 'No matches found.'))
@@ -0,0 +1,83 @@
r"""Claude's `Grep` tool -- recursively regex-search workspace files.
Runs ripgrep through pydantic-ai-harness's `ShellToolset` -- the same shell
capability that backs `Bash` -- instead of a hand-rolled subprocess: the harness
owns process execution, the sandbox PATH, output truncation, and the timeout,
while ripgrep keeps its speed and `.gitignore` filtering (a poor fit for the
harness's own `FileSystemToolset.search_files`, which walks every non-dotfile
including vendored/ignored trees and matches with Python `re`). The
directory-scoped AGENTS.md / CLAUDE.md context blocks are still prepended.
Two adapters bridge a shell command back into a grep tool:
- Containment. `ShellToolset` roots the cwd at the workspace but, unlike the
harness `FileSystemToolset`, does not validate the `path` argument, so a bare
`rg -- ../..` could escape `$GITHUB_WORKSPACE`. `path` is preflighted through
the filesystem capability -- the same containment `Read`/`Glob`/`LS` enforce.
- Framing. `run_command` frames output as `[stdout]` / `[stderr]` blocks and
appends `\n[exit code: N]` only on a non-zero exit. ripgrep exits 1 on "no
matches" (not an error) and 2+ on a real error, so the exit code -- not the
presence of `[stdout]` -- is what the adapter keys off, unwrapping the generic
framing back into a grep-shaped result.
"""
import re
import shlex
from pydantic_ai.exceptions import ModelRetry
from ._backends import filesystem, shell
from .shared import attach_context, clip
GREP_TIMEOUT = 60.0
# `run_command` appends this only on a non-zero exit; ripgrep's exit code rides
# at the very end of the framed output, after any tail-truncation of the body.
_EXIT_CODE_RE = re.compile(r'\n\[exit code: (\d+)\]\Z')
# The harness tail-truncates an oversized body and prepends this marker, which
# elides the leading `[stdout]` header; tolerate it so a large match set is
# never mistaken for an error.
_TRUNCATION_PREFIX = '[... output truncated'
# `search_files` (the harness's own content search) returns this on no match;
# grep mirrors it even though it's ripgrep-backed, so the two search tools agree.
_NO_MATCHES = 'No matches found.'
def _split_exit_code(out: str) -> tuple[str, int]:
"""Split a `run_command` result into its body and the command's exit code (0 if absent)."""
match = _EXIT_CODE_RE.search(out)
if match is None:
return out, 0
return out[: match.start()], int(match.group(1))
async def grep(pattern: str, path: str = '.') -> str:
"""Recursively regex-search workspace files via ripgrep, returning `file:line:text` matches."""
# `file_info` accepts '' as the workspace root, but `rg -- ''` errors on the
# empty path argument, so normalize to the default search root up front.
path = path or '.'
command = f'rg --line-number --no-heading --color never -e {shlex.quote(pattern)} -- {shlex.quote(path)}'
# Preflight `path` through the filesystem containment check (an escape or a
# missing path comes back as `ModelRetry`) before handing it to the shell.
try:
await filesystem().file_info(path)
out = await shell().run_command(command, timeout_seconds=GREP_TIMEOUT)
except (ModelRetry, OSError) as exc:
# The harness converts only a fixed set of errors to `ModelRetry`; a bare
# `OSError` (e.g. `ENAMETOOLONG` from the `file_info` preflight resolving
# an over-long path) would otherwise escape and abort the whole run.
return f'error: {exc}'
if out.startswith('[Command timed out'):
return f'error: {out}'
body, exit_code = _split_exit_code(out)
if exit_code >= 2: # bad pattern, unreadable path, ripgrep absent (127), ...
return f'error: {out}'
if exit_code == 1: # ripgrep's "nothing matched"
return clip(attach_context(path) + _NO_MATCHES)
# exit 0: matches. Strip the truncation marker first (it precedes and elides
# the `[stdout]` header), then the header itself.
if body.startswith(_TRUNCATION_PREFIX):
body = body.split('\n', 1)[1] if '\n' in body else ''
body = body.removeprefix('[stdout]\n')
return clip(attach_context(path) + (body or _NO_MATCHES))
@@ -0,0 +1,30 @@
"""Claude's `LS` tool -- list a workspace directory's entries.
Containment comes from a pydantic-ai-harness `FileSystemToolset.file_info`
preflight (the same check `Grep` uses), but the listing itself is hand-rolled
rather than delegated to `FileSystemToolset.list_directory`: the harness walker
hides every dot-prefixed entry, which would make `.github/` -- where gh-aw's own
workflows live -- invisible to the agent. The Claude `LS` signature is preserved
and the directory-scoped AGENTS.md / CLAUDE.md context blocks are still prepended.
"""
from pydantic_ai.exceptions import ModelRetry
from ._backends import filesystem
from .shared import attach_context, clip, resolve
async def list_dir(path: str = '.') -> str:
"""List a workspace directory's entries (directories marked with `/`)."""
try:
# `file_info` enforces workspace containment and rejects a missing path;
# the enumeration then keeps the dot-prefixed entries the harness drops.
await filesystem().file_info(path)
entries = resolve(path).iterdir()
listing = '\n'.join(sorted(e.name + ('/' if e.is_dir() else '') for e in entries)) or '(empty)'
except (ModelRetry, OSError) as exc:
# ModelRetry: the containment preflight rejected the path. OSError (e.g.
# `NotADirectoryError` on a file, or `ENAMETOOLONG`): not recoverable in the
# harness, so catch it here rather than let it abort the whole run.
return f'error: {exc}'
return clip(attach_context(path) + listing)
@@ -0,0 +1,41 @@
"""Claude's `MultiEdit` tool — apply multiple string replacements to one file atomically."""
# pydantic requires typing_extensions.TypedDict (not typing.TypedDict) for
# schema generation on Python < 3.12; typing_extensions ships with pydantic.
from typing_extensions import NotRequired, TypedDict
from .shared import attach_context, resolve
class EditOp(TypedDict):
"""One replacement for `MultiEdit` (Claude's edit schema)."""
old_string: str
new_string: str
replace_all: NotRequired[bool]
def multi_edit(file_path: str, edits: list[EditOp]) -> str:
"""Apply a sequence of string replacements to one workspace file atomically.
Each edit replaces the first occurrence of `old_string` (or every
occurrence when `replace_all`). If any `old_string` is missing, nothing is
written — the file is left untouched.
"""
try:
p = resolve(file_path)
text = original = p.read_text(encoding='utf-8')
except OSError as exc:
return f'error: {exc}'
for i, e in enumerate(edits):
old = e.get('old_string', '')
if not old or old not in text:
return f'error: edit #{i + 1} `old_string` not found (no changes written)'
text = text.replace(old, e.get('new_string', ''), -1 if e.get('replace_all') else 1)
if text == original:
return 'no changes'
try:
p.write_text(text, encoding='utf-8')
except OSError as exc:
return f'error: {exc}'
return attach_context(file_path) + f'applied {len(edits)} edit(s) to {p}'
@@ -0,0 +1,88 @@
"""Claude's `Read` tool -- read a UTF-8 text file.
Backed by pydantic-ai-harness's `FileSystemToolset.read_file`: path containment,
symlink resolution, and binary-file detection come from the harness. The Claude
`Read` signature (1-based `offset`, `limit`) is preserved, and the
directory-scoped AGENTS.md / CLAUDE.md context blocks are still prepended.
"""
import re
from pydantic_ai.exceptions import ModelRetry
from ._backends import filesystem
from .shared import MAX_TOOL_OUTPUT, attach_context, clip
# When the harness truncates, it appends a continuation hint carrying *its* 0-based
# offset: `... (N more lines. Use offset=M to continue reading.)`. This tool exposes
# a 1-based offset (mirroring Claude's `Read`), so the advertised value must be
# bumped by one or a model that follows the hint literally re-reads the boundary
# line. The harness writes the hint at the start of its own line, while every
# content line is line-number-prefixed (`{n:>6}\t...`); anchoring to `^` (with
# `MULTILINE`) matches only the real hint, never a file whose own contents happen
# to reproduce the hint text.
_CONTINUE_HINT_RE = re.compile(r'^(\.\.\. \(\d+ more lines\. Use offset=)(\d+)( to continue reading\.\))', re.MULTILINE)
# The harness numbers each content line as `{lineno:>6}\t{text}` (1-based). A
# char-budget truncation reads that leading number back to advertise an exact
# continuation offset for the first line it dropped.
_LINE_NO_RE = re.compile(r' *(\d+)\t')
def _hint_to_one_based(body: str) -> str:
"""Rewrite the harness's 0-based continuation offset into this tool's 1-based one."""
return _CONTINUE_HINT_RE.sub(lambda m: f'{m[1]}{int(m[2]) + 1}{m[3]}', body)
def _fit_to_output_budget(prefix: str, body: str) -> str:
"""Return `prefix + body`, truncated to the output cap on a whole-line boundary.
`clip()` keeps the head and drops the tail, but the harness's continuation hint
rides at the tail; a chunk well over the char cap (e.g. 2000 lines of code) would
lose it, leaving the model with no next offset. Instead keep whole numbered lines
that fit and re-advertise the 1-based offset of the first dropped line, so the
Read round-trip stays exact even when the per-line cap and the per-char cap
disagree.
"""
out = prefix + body
if len(out) <= MAX_TOOL_OUTPUT:
return out
size = len(prefix)
kept: list[str] = []
last_lineno: int | None = None
for line in body.split('\n'):
size += len(line) + 1
if size > MAX_TOOL_OUTPUT and kept:
break
kept.append(line)
numbered = _LINE_NO_RE.match(line)
if numbered:
last_lineno = int(numbered.group(1))
if last_lineno is None:
# Not even one whole numbered line fit (e.g. a single enormous line); fall
# back to a plain head clip so the model still sees partial content.
return clip(out)
tail = f'... (truncated to fit the output limit. Use offset={last_lineno + 1} to continue reading.)'
return prefix + '\n'.join(kept).rstrip('\n') + '\n' + tail
async def read_file(file_path: str, offset: int | None = None, limit: int | None = None) -> str:
"""Read a UTF-8 text file. Relative paths resolve under the workspace.
Optional 1-based line `offset` and line `limit` mirror Claude's Read tool.
"""
# Claude's `offset` is a 1-based line number; the harness uses a 0-based offset.
zero_based = max((offset or 1) - 1, 0)
# A non-positive `limit` is degenerate: passed straight through, `limit=0` makes
# the harness read zero lines and emit a same-offset hint that loops forever.
# Normalize it to `None` so it behaves exactly like an omitted `limit` (the
# harness applies its default line cap) rather than reading nothing.
effective_limit = limit if limit and limit > 0 else None
try:
body = await filesystem().read_file(file_path, offset=zero_based, limit=effective_limit)
except (ModelRetry, OSError) as exc:
# The harness only converts a fixed set of errors to `ModelRetry`; a bare
# `OSError` (e.g. `ENAMETOOLONG` while resolving the path) would otherwise
# escape and abort the whole run, where the old tool returned an error.
return f'error: {exc}'
return _fit_to_output_budget(attach_context(file_path), _hint_to_one_based(body))
@@ -0,0 +1,122 @@
"""Shared helpers used by the native tools in `pydantic_ai_gh_aw_shim/`.
Stdlib-only (apart from the package's own modules). Anything
pydantic-ai-specific belongs in the CLI module, not here — each tool
module should import only `shared` and be otherwise independent so it can
be swapped out cleanly.
"""
import contextvars
import logging
import os
import pathlib
MAX_TOOL_OUTPUT = 50_000
"""Cap on any native tool's stringified result. Larger outputs are clipped with
a `…[truncated N chars]` suffix so the model knows it didn't see everything.
50 000 chars (~500 lines of typical Python) lets most files be read in one shot
without requiring the model to loop over small offset/limit chunks."""
CONTEXT_FILE_NAMES = ('AGENTS.md', 'CLAUDE.md')
MAX_CONTEXT_FILE_CHARS = 8000
# Configured for output by `cli.configure_logging()` (called from `main()`);
# at import time the logger only has whatever handlers the embedding
# application has set up. Library code never calls `logging.basicConfig`.
logger = logging.getLogger('pydantic_ai_gh_aw_shim')
def workspace() -> str:
"""Resolve the workspace root live.
Reading `GITHUB_WORKSPACE` lazily
(rather than capturing at import time) means tests can set it via
`monkeypatch.setenv` instead of patching a module-level constant.
"""
return os.environ.get('GITHUB_WORKSPACE') or os.getcwd()
def resolve(path: str) -> pathlib.Path:
"""Resolve a relative path against the current workspace; absolute paths pass through."""
p = pathlib.Path(path)
return p if p.is_absolute() else pathlib.Path(workspace()) / p
def clip(text: str) -> str:
"""Truncate a tool result string to `MAX_TOOL_OUTPUT` chars."""
if len(text) <= MAX_TOOL_OUTPUT:
return text
return text[:MAX_TOOL_OUTPUT] + f'\n…[truncated {len(text) - MAX_TOOL_OUTPUT} chars]'
# --------------------------------------------------------------------------- #
# Directory-scoped AGENTS.md / CLAUDE.md auto-loading
# When the agent touches a file, the shim transparently surfaces the
# directory's `AGENTS.md` / `CLAUDE.md` the first time it's relevant (walking
# up to the workspace root). Per-run state ensures each file is shown at most
# once. Held in a `ContextVar` so each `run()` (and each test) gets its own
# set without test-ordering leaks, while `Task` sub-agents naturally inherit
# the parent's set via the async context tree (mutations to the shared object
# propagate both ways).
# --------------------------------------------------------------------------- #
_seen_context_files_var: contextvars.ContextVar[set[pathlib.Path] | None] = contextvars.ContextVar(
'_seen_context_files', default=None
)
def _seen_context_files() -> set[pathlib.Path]:
"""Return the current run's dedupe set, installing one lazily if a tool ran outside `run()`."""
s = _seen_context_files_var.get()
if s is None:
s = set[pathlib.Path]()
_seen_context_files_var.set(s)
return s
def reset_context_state() -> None:
"""Install a fresh dedupe set in the current context.
Called once at the
start of each `run()` (and from tests that want a clean slate).
"""
_seen_context_files_var.set(set())
def attach_context(path_arg: str | None) -> str:
"""Return any newly-discovered `AGENTS.md` / `CLAUDE.md` blocks to prepend to a path-taking tool result.
Walks up from the target's directory to the
workspace root, surfacing each file at most once per run.
"""
if not path_arg:
return ''
try:
ws = pathlib.Path(workspace()).resolve()
target = resolve(path_arg).resolve()
except OSError:
return ''
seen = _seen_context_files()
cur = target if target.is_dir() else target.parent
blocks: list[str] = []
while cur.is_relative_to(ws):
for name in CONTEXT_FILE_NAMES:
candidate = cur / name
if not candidate.is_file() or candidate in seen:
continue
seen.add(candidate)
try:
content = candidate.read_text(encoding='utf-8', errors='replace')
except OSError:
continue
rel = candidate.relative_to(ws)
blocks.append(
f'--- context: {rel} (auto-loaded; shown once per run) ---\n{content[:MAX_CONTEXT_FILE_CHARS]}\n'
)
if cur == ws or cur.parent == cur:
break
cur = cur.parent
if not blocks:
return ''
return '\n'.join(blocks) + '\n'
@@ -0,0 +1,52 @@
"""Claude's `TodoWrite` tool -- record the agent's task checklist.
Backed by pydantic-ai-harness's `experimental.planning` capability: the adapter
maps Claude's todo schema onto the harness `PlanItem` list and calls the same
`write_plan` the capability exposes, so the checklist is rendered (with the
harness's advisory note when more than one step is `in_progress`) instead of a
hand-rolled ack.
`planning` is experimental -- its API may change without a deprecation period --
which is acceptable here; the warning is silenced at import so it doesn't leak
into the agent's stdout. The Claude `TodoWrite` signature (`content` / `status` /
`activeForm` items) is preserved; `activeForm` is the present-tense label Claude
shows while a step runs and has no harness equivalent, so it's dropped (the
headless shim renders nothing live anyway).
"""
import warnings
from pydantic_ai_harness.experimental import HarnessExperimentalWarning
from typing_extensions import TypedDict
with warnings.catch_warnings():
warnings.simplefilter('ignore', HarnessExperimentalWarning)
from pydantic_ai_harness.experimental.planning import PlanItem, Planning, PlanningToolset, TaskStatus
class TodoItem(TypedDict):
"""One entry for `TodoWrite` (Claude's todo schema)."""
content: str
status: str
activeForm: str
def _to_status(value: str) -> TaskStatus:
"""Map a Claude todo status onto a harness `TaskStatus`, defaulting to `pending`."""
try:
return TaskStatus(value)
except ValueError:
return TaskStatus.pending
async def todo_write(todos: list[TodoItem]) -> str:
"""Record the agent's task checklist."""
items = [PlanItem(content=t.get('content', ''), status=_to_status(t.get('status', ''))) for t in todos]
# A fresh capability per call gives a fresh plan state; Claude resends the
# full list every time, so no cross-call state needs to be retained.
# `get_toolset()` is typed `AgentToolset | None` but always returns the
# planning toolset, so narrow to reach `write_plan` without a private import.
toolset = Planning[None]().get_toolset()
assert isinstance(toolset, PlanningToolset)
return await toolset.write_plan(items)
@@ -0,0 +1,32 @@
"""Claude's `Write` tool -- create or overwrite a workspace text file.
Backed by pydantic-ai-harness's `FileSystemToolset.write_file` (path
containment, symlink resolution). Claude's `Write` creates missing parent
directories, and the harness `write_file` requires the parent to exist, so the
adapter calls `create_directory` first to keep that behavior.
"""
import os
from pydantic_ai.exceptions import ModelRetry
from ._backends import filesystem
from .shared import attach_context
async def write_file(file_path: str, content: str) -> str:
"""Create or overwrite a UTF-8 text file under the workspace."""
fs = filesystem()
parent = os.path.dirname(file_path)
try:
if parent:
await fs.create_directory(parent)
result = await fs.write_file(file_path, content)
except (ModelRetry, OSError) as exc:
# The harness converts its own recoverable errors to `ModelRetry`, but
# `create_directory` -> `Path.mkdir(exist_ok=True)` still raises a bare
# `FileExistsError` when a path segment is an existing file. The old
# hand-rolled `Write` caught `OSError`, so keep doing that: a bad path is
# a returned error, not a run-aborting exception.
return f'error: {exc}'
return attach_context(file_path) + result
+179
View File
@@ -0,0 +1,179 @@
from __future__ import annotations
import sys
import urllib.error
from pathlib import Path
sys.path.insert(0, str(Path(__file__).parent))
import ci_duration
def test_normalize_matrix_job_signature():
job = ci_duration.normalize_job(
{
'id': 123,
'name': 'test on 3.10 (all-extras)',
'status': 'completed',
'conclusion': 'success',
'started_at': '2026-06-13T17:15:03Z',
'completed_at': '2026-06-13T17:24:05Z',
'runner_name': 'GitHub Actions 1001364942',
'runner_group_name': 'GitHub Actions',
'html_url': 'https://github.com/pydantic/pydantic-ai/actions/runs/1/job/123',
'steps': [],
}
)
assert job.job_family == 'test'
assert job.matrix_python == '3.10'
assert job.matrix_extra == 'all-extras'
assert job.runner_class == 'github-hosted'
assert job.job_signature == 'job=test / runner=github-hosted / py=3.10 / extra=all-extras'
assert job.duration_seconds == 542
assert ci_duration.is_tracked_test_job(job)
def test_non_test_jobs_are_not_tracked():
jobs = [
ci_duration.normalize_job(
{
'id': 123,
'name': name,
'status': 'completed',
'conclusion': 'success',
'started_at': '2026-06-13T17:15:03Z',
'completed_at': '2026-06-13T17:16:03Z',
'runner_name': 'GitHub Actions 1001364942',
'runner_group_name': 'GitHub Actions',
'html_url': 'https://github.com/pydantic/pydantic-ai/actions/runs/1/job/123',
'steps': [],
}
)
for name in ['lint', 'test examples on 3.13']
]
assert [ci_duration.is_tracked_test_job(job) for job in jobs] == [False, False]
def test_classify_slow_job_requires_relative_and_absolute_delta():
baseline = ci_duration.compute_baseline([360, 370, 380, 390, 400, 410, 420, 430, 440, 450])
job = ci_duration.JobRecord(
job_id=123,
raw_name='test on 3.10 (all-extras)',
job_family='test',
job_signature='job=test / runner=github-hosted / py=3.10 / extra=all-extras',
matrix_python='3.10',
matrix_extra='all-extras',
conclusion='success',
status='completed',
started_at='2026-06-13T17:15:03Z',
completed_at='2026-06-13T17:24:05Z',
duration_seconds=600,
runner_name='GitHub Actions 1001364942',
runner_group_name='GitHub Actions',
runner_class='github-hosted',
html_url='https://github.com/pydantic/pydantic-ai/actions/runs/1/job/123',
steps=[],
)
row = ci_duration.classify_job(job, baseline)
assert row.status == 'slow'
assert row.delta_seconds == 195
def test_render_report_uses_sticky_marker_and_threshold_context():
workflow: ci_duration.JsonObject = {
'duration_seconds': 840,
'html_url': 'https://github.com/pydantic/pydantic-ai/actions/runs/1',
}
row = ci_duration.ReportRow(
job_name='test on 3.10 (all-extras)',
job_signature='job=test / runner=github-hosted / py=3.10 / extra=all-extras',
duration_seconds=600,
baseline=ci_duration.compute_baseline([360, 370, 380, 390, 400, 410, 420, 430, 440, 450]),
delta_seconds=195,
delta_percent=48,
status='slow',
)
report = ci_duration.render_report(123, 'abcdef1234567890', workflow, [row])
assert report.startswith('<!-- ci-duration-report -->\n## CI Duration Report')
assert 'Tracked test jobs: 1' in report
assert 'Total tracked test job duration: 10m 00s' in report
assert 'Baseline: up to 30 successful `main` CI runs and 60 successful PR CI runs' in report
assert 'Minimum baseline sample: 10 successful matching jobs' in report
assert '| test on 3.10 (all-extras) | 10m 00s | 6m 45s | 7m 08s | +3m 15s (+48%) | slow |' in report
assert 'trigger:ci-duration-report' in report
def test_collect_baselines_skips_unavailable_historical_run():
class StubGitHubClient(ci_duration.GitHubClient):
def request_paginated(self, path: str, *, max_items: int | None = None) -> list[ci_duration.JsonObject]:
if path == 'actions/workflows/ci.yml/runs?branch=main&event=push&status=success':
return [
{
'id': run_id,
'run_attempt': 1,
'head_sha': f'baseline-{run_id}',
}
for run_id in range(11)
]
if path == 'actions/workflows/ci.yml/runs?event=pull_request&status=success':
return []
if path == 'actions/runs/0/attempts/1/jobs':
raise urllib.error.URLError('timed out')
if path.startswith('actions/runs/') and path.endswith('/attempts/1/jobs'):
return [
{
'id': 123,
'name': 'test on 3.10 (all-extras)',
'status': 'completed',
'conclusion': 'success',
'started_at': '2026-06-13T17:15:03Z',
'completed_at': '2026-06-13T17:24:05Z',
'runner_name': 'GitHub Actions 1001364942',
'runner_group_name': 'GitHub Actions',
'html_url': 'https://github.com/pydantic/pydantic-ai/actions/runs/1/job/123',
'steps': [],
}
]
raise RuntimeError(f'Unexpected path: {path}')
baselines = ci_duration.collect_baselines(StubGitHubClient('pydantic/pydantic-ai', 'token'), 'current-sha')
assert baselines['job=test / runner=github-hosted / py=3.10 / extra=all-extras'].sample_size == 10
def test_collect_baselines_stops_after_time_budget():
class StubGitHubClient(ci_duration.GitHubClient):
def request_paginated(self, path: str, *, max_items: int | None = None) -> list[ci_duration.JsonObject]:
if path == 'actions/workflows/ci.yml/runs?branch=main&event=push&status=success':
return [
{
'id': 1,
'run_attempt': 1,
'head_sha': 'baseline-1',
}
]
if path == 'actions/workflows/ci.yml/runs?event=pull_request&status=success':
return []
raise RuntimeError(f'Unexpected path: {path}')
monotonic_values = [0.0, ci_duration.BASELINE_COLLECTION_MAX_SECONDS]
original_monotonic = ci_duration.time.monotonic
def monotonic() -> float:
if monotonic_values:
return monotonic_values.pop(0)
return ci_duration.BASELINE_COLLECTION_MAX_SECONDS
ci_duration.time.monotonic = monotonic
try:
baselines = ci_duration.collect_baselines(StubGitHubClient('pydantic/pydantic-ai', 'token'), 'current-sha')
finally:
ci_duration.time.monotonic = original_monotonic
assert baselines == {}
File diff suppressed because it is too large Load Diff
+57
View File
@@ -0,0 +1,57 @@
import json
import os
import re
import typing
import httpx
DEPLOY_OUTPUT = os.environ['DEPLOY_OUTPUT']
GITHUB_TOKEN = os.environ['GITHUB_TOKEN']
REPOSITORY = os.environ['REPOSITORY']
REF = os.environ['REF']
ENVIRONMENT = 'deploy-docs-preview'
m = re.search(r'https://(\S+)\.workers\.dev', DEPLOY_OUTPUT)
assert m, f'Could not find worker URL in {DEPLOY_OUTPUT!r}'
worker_name = m.group(1)
m = re.search(r'Current Version ID: ([^-]+)', DEPLOY_OUTPUT)
assert m, f'Could not find version ID in {DEPLOY_OUTPUT!r}'
version_id = m.group(1)
preview_url = f'https://{version_id}-{worker_name}.workers.dev'
print('CloudFlare worker preview URL:', preview_url, flush=True)
gh_headers = {
'Accept': 'application/vnd.github+json',
'Authorization': f'Bearer {GITHUB_TOKEN}',
'X-GitHub-Api-Version': '2022-11-28',
}
deployment_url = f'https://api.github.com/repos/{REPOSITORY}/deployments'
deployment_data: dict[str, typing.Any] = {
'ref': REF,
'task': 'docs preview',
'environment': ENVIRONMENT,
'auto_merge': False,
'required_contexts': [],
'payload': json.dumps({
'preview_url': preview_url,
'worker_name': worker_name,
'version_id': version_id,
})
}
r = httpx.post(deployment_url, headers=gh_headers, json=deployment_data)
print(f'POST {deployment_url} {r.status_code} {r.text}', flush=True)
r.raise_for_status()
deployment_id = r.json()['id']
status_url = f'https://api.github.com/repos/{REPOSITORY}/deployments/{deployment_id}/statuses'
status_data = {
'environment': ENVIRONMENT,
'environment_url': preview_url,
'state': 'success',
}
r = httpx.post(status_url, headers=gh_headers, json=status_data)
print(f'POST {status_url} {r.status_code} {r.text}', flush=True)
r.raise_for_status()
+72
View File
@@ -0,0 +1,72 @@
import os
import re
import httpx
DEPLOY_OUTPUT = os.environ['DEPLOY_OUTPUT']
GITHUB_TOKEN = os.environ['GITHUB_TOKEN']
REPOSITORY = os.environ['REPOSITORY']
PULL_REQUEST_NUMBER = os.environ['PULL_REQUEST_NUMBER']
REF = os.environ['REF']
m = re.search(r'https://(\S+)\.workers\.dev', DEPLOY_OUTPUT)
assert m, f'Could not find worker URL in {DEPLOY_OUTPUT!r}'
worker_name = m.group(1)
m = re.search(r'Current Version ID: ([^-]+)', DEPLOY_OUTPUT)
assert m, f'Could not find version ID in {DEPLOY_OUTPUT!r}'
version_id = m.group(1)
preview_url = f'https://{version_id}-{worker_name}.workers.dev'
print('Docs preview URL:', preview_url, flush=True)
gh_headers = {
'Accept': 'application/vnd.github+json',
'Authorization': f'Bearer {GITHUB_TOKEN}',
'X-GitHub-Api-Version': '2022-11-28',
}
# now create or update a comment on the PR with the preview URL
if not PULL_REQUEST_NUMBER:
print('Pull request number not set', flush=True)
exit(1)
comments_url = f'https://api.github.com/repos/{REPOSITORY}/issues/{PULL_REQUEST_NUMBER}/comments'
r = httpx.get(comments_url, headers=gh_headers)
print(f'{r.request.method} {r.request.url} {r.status_code}', flush=True)
if r.status_code != 200:
print(f'Failed to get comments, status {r.status_code}, response:\n{r.text}', flush=True)
exit(1)
comment_update_url = None
for comment in r.json():
if comment['user']['login'] == 'github-actions[bot]' and comment['body'].startswith('## Docs Preview'):
comment_update_url = comment['url']
break
body = f"""\
## Docs Preview
<table>
<tr>
<td><strong>commit:</strong></td>
<td><code>{REF:.7}</code></td>
</tr>
<tr>
<td><strong>Preview URL:</strong></td>
<td><a href="{preview_url}">{preview_url}</a></td>
</tr>
</table>
"""
comment_data = {'body': body}
if comment_update_url:
print('Updating existing comment...', flush=True)
r = httpx.patch(comment_update_url, headers=gh_headers, json=comment_data)
else:
print('Creating new comment...', flush=True)
r = httpx.post(comments_url, headers=gh_headers, json=comment_data)
print(f'{r.request.method} {r.request.url} {r.status_code}', flush=True)
r.raise_for_status()
+13
View File
@@ -0,0 +1,13 @@
# Agentic workflows (`gh-aw`)
The `pydantic-ai-*` workflows in this directory are [agentic workflows](https://github.com/githubnext/gh-aw) authored as human-editable `<name>.md` sources (frontmatter + prompt) that **compile** to a generated `<name>.lock.yml`. GitHub Actions runs the `.lock.yml`, never the `.md`.
- **Never hand-edit a `*.lock.yml`.** It is generated — the header says so. Manual edits are silently overwritten on the next recompile, and until then the running workflow diverges from its source.
- **After editing a workflow `*.md` source, recompile and commit the regenerated `*.lock.yml` in the same change** — a `*.md` edit without its recompiled lock is incomplete, and source and lock drift apart:
```
gh aw compile
```
- **Recompilation is required for anything the lock bakes in:** a source's frontmatter (`on:` triggers, `permissions`, `tools`, `safe-outputs`, jobs, path/`detect` filters) and its `imports:` shared fragments (`shared/*.md`) are inlined into the lock at compile time.
- **Exception — runtime-resolved prompts need no recompile.** Agent prompts under `shared/prompts/` are fetched at run time (via the `fetch-dynamic-prompt` action / a Logfire-managed variable), not baked into the lock, so editing one takes effect on the next run without recompiling.
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+137
View File
@@ -0,0 +1,137 @@
name: After CI
on:
# zizmor: ignore[dangerous-triggers] -- workflow_run is needed to access secrets after CI completes on fork PRs
workflow_run:
workflows: [CI]
types: [completed]
permissions: {}
jobs:
ci-duration-collect:
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
python-version: "3.12"
- name: Collect CI duration telemetry
continue-on-error: true
run: uv run --no-project --with certifi --with logfire python .github/scripts/ci_duration.py collect --output ci-duration-record.json
env:
GITHUB_TOKEN: ${{ github.token }}
GITHUB_REPOSITORY: ${{ github.repository }}
CI_RUN_ID: ${{ github.event.workflow_run.id }}
CI_RUN_ATTEMPT: ${{ github.event.workflow_run.run_attempt }}
LOGFIRE_WRITE_TOKEN: ${{ secrets.LOGFIRE_WRITE_TOKEN }}
LOGFIRE_TOKEN: ${{ secrets.LOGFIRE_TOKEN }}
LOGFIRE_URL: ${{ vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
OTEL_EXPORTER_OTLP_ENDPOINT: ${{ vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
OTEL_EXPORTER_OTLP_HEADERS: Authorization=${{ secrets.LOGFIRE_TOKEN }}
- name: Store CI duration record
if: hashFiles('ci-duration-record.json') != ''
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ci-duration-${{ github.event.workflow_run.id }}-${{ github.event.workflow_run.run_attempt }}
path: ci-duration-record.json
retention-days: 7
smokeshow:
runs-on: ubuntu-latest
permissions:
statuses: write
steps:
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
python-version: "3.12"
- uses: dawidd6/action-download-artifact@b6e2e70617bc3265edd6dab6c906732b2f1ae151 # v21
with:
workflow: ci.yml
name: "(diff-)?coverage-html.*"
name_is_regexp: true
commit: ${{ github.event.workflow_run.head_sha }}
allow_forks: true
workflow_conclusion: completed
if_no_artifact_found: warn
- run: uvx smokeshow upload coverage-html
if: hashFiles('coverage-html/*.html') != ''
env:
SMOKESHOW_GITHUB_STATUS_DESCRIPTION: Coverage {coverage-percentage}
SMOKESHOW_GITHUB_COVERAGE_THRESHOLD: 95
SMOKESHOW_GITHUB_CONTEXT: coverage
SMOKESHOW_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SMOKESHOW_GITHUB_PR_HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
SMOKESHOW_AUTH_KEY: ${{ secrets.SMOKESHOW_AUTH_KEY }}
deploy-docs-preview:
runs-on: ubuntu-latest
if: github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.pull_requests[0] != null
permissions:
pull-requests: write
environment:
name: deploy-docs-preview
steps:
- run: echo "$GITHUB_EVENT_JSON"
env:
GITHUB_EVENT_JSON: ${{ toJSON(github.event) }}
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
- run: npm install
working-directory: docs-site
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
python-version: "3.12"
enable-cache: true
cache-suffix: deploy-docs-preview
- id: download-artifact
uses: dawidd6/action-download-artifact@b6e2e70617bc3265edd6dab6c906732b2f1ae151 # v21
with:
workflow: ci.yml
name: site
path: site
commit: ${{ github.event.workflow_run.head_sha }}
allow_forks: true
workflow_conclusion: completed
if_no_artifact_found: warn
- uses: cloudflare/wrangler-action@ebbaa1584979971c8614a24965b4405ff95890e0 # v4.0.0
id: deploy
if: steps.download-artifact.outputs.found_artifact == 'true'
with:
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
environment: previews
workingDirectory: docs-site
command: >
deploy
--var GIT_COMMIT_SHA:${{ github.event.workflow_run.head_sha }}
--var GIT_BRANCH:${{ github.event.workflow_run.head_branch }}
- name: Set preview URL
run: uv run --no-project --with httpx .github/set_docs_pr_preview_url.py
if: steps.deploy.outcome == 'success'
env:
DEPLOY_OUTPUT: ${{ steps.deploy.outputs.command-output }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPOSITORY: ${{ github.repository }}
PULL_REQUEST_NUMBER: ${{ github.event.workflow_run.pull_requests[0].number }}
REF: ${{ github.event.workflow_run.head_sha }}
+582
View File
@@ -0,0 +1,582 @@
# ___ _ _
# / _ \ | | (_)
# | |_| | __ _ ___ _ __ | |_ _ ___
# | _ |/ _` |/ _ \ '_ \| __| |/ __|
# | | | | (_| | __/ | | | |_| | (__
# \_| |_/\__, |\___|_| |_|\__|_|\___|
# __/ |
# _ _ |___/
# | | | | / _| |
# | | | | ___ _ __ _ __| |_| | _____ ____
# | |/\| |/ _ \ '__| |/ /| _| |/ _ \ \ /\ / / ___|
# \ /\ / (_) | | | | ( | | | | (_) \ V V /\__ \
# \/ \/ \___/|_| |_|\_\|_| |_|\___/ \_/\_/ |___/
#
# This file was automatically generated by pkg/workflow/maintenance_workflow.go (v0.74.8). DO NOT EDIT.
#
# To regenerate this workflow, run:
# gh aw compile
# Not all edits will cause changes to this file.
#
# For more information: https://github.github.com/gh-aw/introduction/overview/
#
# Alternative regeneration methods:
# make recompile
#
# Or use the gh-aw CLI directly:
# ./gh-aw compile --validate --verbose
#
# The workflow is generated when any workflow uses the 'expires' field
# in create-discussions, create-issues, or create-pull-request safe-outputs configuration.
# Schedule frequency is automatically determined by the shortest expiration time.
#
name: Agentic Maintenance
on:
schedule:
- cron: "37 0 * * *" # Daily (based on minimum expires: 7 days)
workflow_dispatch:
inputs:
operation:
description: 'Optional maintenance operation to run'
required: false
type: choice
default: ''
options:
- ''
- 'disable'
- 'enable'
- 'update'
- 'upgrade'
- 'safe_outputs'
- 'create_labels'
- 'activity_report'
- 'close_agentic_workflows_issues'
- 'clean_cache_memories'
- 'update_pull_request_branches'
- 'validate'
- 'forecast'
run_url:
description: 'Run URL or run ID to replay safe outputs from (e.g. https://github.com/owner/repo/actions/runs/12345 or 12345). Required when operation is safe_outputs.'
required: false
type: string
default: ''
workflow_call:
inputs:
operation:
description: 'Optional maintenance operation to run (disable, enable, update, upgrade, safe_outputs, create_labels, activity_report, close_agentic_workflows_issues, clean_cache_memories, update_pull_request_branches, validate, forecast)'
required: false
type: string
default: ''
run_url:
description: 'Run URL or run ID to replay safe outputs from (e.g. https://github.com/owner/repo/actions/runs/12345 or 12345). Required when operation is safe_outputs.'
required: false
type: string
default: ''
outputs:
operation_completed:
description: 'The maintenance operation that was completed (empty when none ran or a scheduled job ran)'
value: ${{ jobs.run_operation.outputs.operation || inputs.operation }}
applied_run_url:
description: 'The run URL that safe outputs were applied from'
value: ${{ jobs.apply_safe_outputs.outputs.run_url }}
permissions: {}
jobs:
close-expired-entities:
if: ${{ (!(github.event.repository.fork)) && github.event_name != 'push' && (github.event_name != 'workflow_dispatch' && github.event_name != 'workflow_call' || inputs.operation == '') }}
runs-on: ubuntu-slim
permissions:
discussions: write
issues: write
pull-requests: write
steps:
- name: Setup Scripts
uses: github/gh-aw-actions/setup@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Close expired discussions
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/close_expired_discussions.cjs');
await main();
- name: Close expired issues
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/close_expired_issues.cjs');
await main();
- name: Close expired pull requests
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/close_expired_pull_requests.cjs');
await main();
cleanup-cache-memory:
if: ${{ (!(github.event.repository.fork)) && github.event_name != 'push' && (github.event_name != 'workflow_dispatch' && github.event_name != 'workflow_call' || inputs.operation == '' || inputs.operation == 'clean_cache_memories') }}
runs-on: ubuntu-slim
permissions:
actions: write
steps:
- name: Setup Scripts
uses: github/gh-aw-actions/setup@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Cleanup outdated cache-memory entries
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/cleanup_cache_memory.cjs');
await main();
run_operation:
if: ${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.operation != '' && inputs.operation != 'safe_outputs' && inputs.operation != 'create_labels' && inputs.operation != 'activity_report' && inputs.operation != 'close_agentic_workflows_issues' && inputs.operation != 'clean_cache_memories' && inputs.operation != 'update_pull_request_branches' && inputs.operation != 'validate' && inputs.operation != 'forecast' && (!(github.event.repository.fork)) }}
runs-on: ubuntu-slim
permissions:
actions: write
contents: write
pull-requests: write
outputs:
operation: ${{ steps.record.outputs.operation }}
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Setup Scripts
uses: github/gh-aw-actions/setup@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Check admin/maintainer permissions
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/check_team_member.cjs');
await main();
- name: Install gh-aw
uses: github/gh-aw-actions/setup-cli@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
version: v0.74.8
- name: Run operation
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_AW_OPERATION: ${{ inputs.operation }}
GH_AW_CMD_PREFIX: gh aw
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/run_operation_update_upgrade.cjs');
await main();
- name: Record outputs
id: record
run: echo "operation=${{ inputs.operation }}" >> "$GITHUB_OUTPUT"
update_pull_request_branches:
if: ${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.operation == 'update_pull_request_branches' && (!(github.event.repository.fork)) }}
runs-on: ubuntu-slim
permissions:
contents: write
pull-requests: write
steps:
- name: Setup Scripts
uses: github/gh-aw-actions/setup@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Check admin/maintainer permissions
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/check_team_member.cjs');
await main();
- name: Update pull request branches
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/update_pull_request_branches.cjs');
await main();
apply_safe_outputs:
if: ${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.operation == 'safe_outputs' && (!(github.event.repository.fork)) }}
runs-on: ubuntu-slim
permissions:
actions: read
contents: write
discussions: write
issues: write
pull-requests: write
outputs:
run_url: ${{ steps.record.outputs.run_url }}
steps:
- name: Checkout actions folder
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
sparse-checkout: |
actions
persist-credentials: false
- name: Setup Scripts
uses: github/gh-aw-actions/setup@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Check admin/maintainer permissions
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/check_team_member.cjs');
await main();
- name: Apply Safe Outputs
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_AW_RUN_URL: ${{ inputs.run_url }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/apply_safe_outputs_replay.cjs');
await main();
- name: Record outputs
id: record
run: echo "run_url=${{ inputs.run_url }}" >> "$GITHUB_OUTPUT"
create_labels:
if: ${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.operation == 'create_labels' && (!(github.event.repository.fork)) }}
runs-on: ubuntu-slim
permissions:
contents: read
issues: write
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Setup Scripts
uses: github/gh-aw-actions/setup@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Check admin/maintainer permissions
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/check_team_member.cjs');
await main();
- name: Install gh-aw
uses: github/gh-aw-actions/setup-cli@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
version: v0.74.8
- name: Create missing labels
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
GH_AW_CMD_PREFIX: gh aw
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/create_labels.cjs');
await main();
activity_report:
if: ${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.operation == 'activity_report' && (!(github.event.repository.fork)) }}
runs-on: ubuntu-slim
timeout-minutes: 120
permissions:
actions: read
contents: read
issues: write
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Setup Scripts
uses: github/gh-aw-actions/setup@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Check admin/maintainer permissions
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/check_team_member.cjs');
await main();
- name: Install gh-aw
uses: github/gh-aw-actions/setup-cli@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
version: v0.74.8
- name: Restore activity report logs cache
id: activity_report_logs_cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ./.cache/gh-aw/activity-report-logs
key: ${{ runner.os }}-activity-report-logs-${{ github.repository }}-${{ github.ref_name }}-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-activity-report-logs-${{ github.repository }}-
${{ runner.os }}-activity-report-logs-
- name: Download activity report logs
timeout-minutes: 20
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_AW_CMD_PREFIX: gh aw
run: |
${GH_AW_CMD_PREFIX} logs \
--repo "${{ github.repository }}" \
--start-date -1w \
--count 100 \
--output ./.cache/gh-aw/activity-report-logs \
--format markdown \
> ./.cache/gh-aw/activity-report-logs/report.md
- name: Save activity report logs cache
if: ${{ always() }}
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ./.cache/gh-aw/activity-report-logs
key: ${{ steps.activity_report_logs_cache.outputs.cache-primary-key }}
- name: Generate activity report issue
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require('node:fs');
const reportPath = './.cache/gh-aw/activity-report-logs/report.md';
if (!fs.existsSync(reportPath)) {
core.warning('Activity report markdown not found at ' + reportPath + '; skipping issue creation.');
return;
}
let reportBody = '';
try {
reportBody = fs.readFileSync(reportPath, 'utf8').trim();
} catch (error) {
core.warning('Failed to read activity report markdown at ' + reportPath + ': ' + error.message);
return;
}
if (!reportBody) {
core.warning('Activity report markdown is empty at ' + reportPath + '; skipping issue creation.');
return;
}
const repoSlug = context.repo.owner + '/' + context.repo.repo;
const body = [
'### Agentic workflow activity report',
'',
'Repository: ' + repoSlug,
'Generated at: ' + new Date().toISOString(),
'',
reportBody,
].join('\n');
const createdIssue = await github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: '[aw] agentic status report',
body,
labels: ['agentic-workflows'],
});
core.info('Created issue #' + createdIssue.data.number + ': ' + createdIssue.data.html_url);
forecast_report:
if: ${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.operation == 'forecast' && (!(github.event.repository.fork)) }}
runs-on: ubuntu-slim
timeout-minutes: 60
permissions:
actions: read
contents: read
issues: write
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Setup Scripts
uses: github/gh-aw-actions/setup@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Check admin/maintainer permissions
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/check_team_member.cjs');
await main();
- name: Install gh-aw
uses: github/gh-aw-actions/setup-cli@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
version: v0.74.8
- name: Restore forecast report logs cache
id: forecast_report_logs_cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .github/aw/logs
key: ${{ runner.os }}-forecast-report-logs-${{ github.repository }}-${{ github.ref_name }}-${{ github.run_id }}
restore-keys: |
${{ runner.os }}-forecast-report-logs-${{ github.repository }}-
${{ runner.os }}-forecast-report-logs-
- name: Generate forecast report
shell: bash
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GH_AW_CMD_PREFIX: gh aw
run: |
mkdir -p ./.cache/gh-aw/forecast
${GH_AW_CMD_PREFIX} logs --repo "${{ github.repository }}" --start-date -30d --count 1500 > /dev/null
if ! compgen -G ".github/aw/logs/run-*/run_summary.json" > /dev/null; then
echo "::error::Missing run summary cache in .github/aw/logs after gh aw logs warm-up; cannot run forecast."
exit 1
fi
${GH_AW_CMD_PREFIX} forecast --repo "${{ github.repository }}" --json 2> >(grep -Fv "forecast is an experimental command and may change without notice" >&2) > ./.cache/gh-aw/forecast/report.json
- name: Save forecast report logs cache
if: ${{ always() }}
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .github/aw/logs
key: ${{ steps.forecast_report_logs_cache.outputs.cache-primary-key }}
- name: Generate forecast issue
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/create_forecast_issue.cjs');
await main();
close_agentic_workflows_issues:
if: ${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.operation == 'close_agentic_workflows_issues' && (!(github.event.repository.fork)) }}
runs-on: ubuntu-slim
permissions:
issues: write
steps:
- name: Setup Scripts
uses: github/gh-aw-actions/setup@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Check admin/maintainer permissions
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/check_team_member.cjs');
await main();
- name: Close no-repro agentic-workflows issues
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/close_agentic_workflows_issues.cjs');
await main();
validate_workflows:
if: ${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'workflow_call') && inputs.operation == 'validate' && (!(github.event.repository.fork)) }}
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Setup Scripts
uses: github/gh-aw-actions/setup@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
destination: ${{ runner.temp }}/gh-aw/actions
- name: Check admin/maintainer permissions
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/check_team_member.cjs');
await main();
- name: Install gh-aw
uses: github/gh-aw-actions/setup-cli@efa55847f72aadb03490d955263ff911bf758700 # v0.74.8
with:
version: v0.74.8
- name: Validate workflows and file issue on findings
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
GH_AW_CMD_PREFIX: gh aw
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');
setupGlobals(core, github, context, exec, io, getOctokit);
const { main } = require('${{ runner.temp }}/gh-aw/actions/run_validate_workflows.cjs');
await main();
+147
View File
@@ -0,0 +1,147 @@
name: '@claude'
on:
issue_comment:
types: [created]
pull_request_review_comment:
types: [created]
issues:
types: [opened, assigned]
pull_request_review:
types: [submitted]
permissions: {}
env:
UV_PYTHON: 3.13
UV_FROZEN: "1"
jobs:
get-pr-info:
permissions:
contents: read
if: |
(
(github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
) && contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'),
github.event.comment.author_association ||
github.event.review.author_association ||
github.event.issue.author_association)
runs-on: ubuntu-latest
outputs:
is_fork: ${{ steps.pr-info.outputs.is_fork }}
pr_head_repo: ${{ steps.pr-info.outputs.pr_head_repo }}
pr_head_sha: ${{ steps.pr-info.outputs.pr_head_sha }}
pr_base_sha: ${{ steps.pr-info.outputs.pr_base_sha }}
steps:
- name: Get PR info
if: github.event.issue.pull_request || github.event.pull_request
id: pr-info
run: |
PR_NUMBER=${{ github.event.pull_request.number || github.event.issue.number }}
PR_DATA=$(gh api repos/${{ github.repository }}/pulls/${PR_NUMBER})
echo "pr_head_repo=$(echo "$PR_DATA" | jq -r '.head.repo.full_name')" >> $GITHUB_OUTPUT
echo "pr_head_sha=$(echo "$PR_DATA" | jq -r '.head.sha')" >> $GITHUB_OUTPUT
echo "pr_base_sha=$(echo "$PR_DATA" | jq -r '.base.sha')" >> $GITHUB_OUTPUT
echo "is_fork=$(echo "$PR_DATA" | jq -r '.head.repo.fork')" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
at-claude:
needs: get-pr-info
if: needs.get-pr-info.outputs.is_fork != 'true'
runs-on: ubuntu-latest
timeout-minutes: 60
permissions:
contents: write
pull-requests: write
issues: write
id-token: write
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 1
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
cache-suffix: claude-code
- run: uv tool install pre-commit
- run: make install
- name: Install bubblewrap
run: sudo apt-get install -y bubblewrap
- uses: anthropics/claude-code-action@428971d2ecd6e3a7cb0ee0da2a3a8b33fdb3678d # v1.0.157
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
additional_permissions: |
actions: read
claude_args: |
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr checks:*),Bash(gh pr list:*),Bash(gh pr create:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh run view:*),Bash(gh run list:*),Bash(git log:*),Bash(git diff:*),Bash(git grep:*),Bash(git show:*),Bash(git status:*),Bash(git add:*),Bash(git checkout:*),Bash(git commit:*),Bash(git push:*),Bash(rg:*),Bash(ls:*),Bash(tree:*),Bash(grep:*),Bash(uv run:*),Bash(make:*)"
at-claude-fork:
needs: get-pr-info
if: needs.get-pr-info.outputs.is_fork == 'true'
runs-on: ubuntu-latest
timeout-minutes: 60
# Least privilege: on forks Claude may only read code and comment (see the
# restricted --allowedTools below — no push / pr create), so this job runs
# with read-only `contents`. It never gets write access to repository code.
permissions:
contents: read
pull-requests: write
issues: write
id-token: write
actions: read
steps:
- name: Checkout fork repository
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
repository: ${{ needs.get-pr-info.outputs.pr_head_repo }}
ref: ${{ needs.get-pr-info.outputs.pr_head_sha }}
fetch-depth: 1
persist-credentials: false
- name: Check for modified config files
# Compare the exact immutable SHAs captured in get-pr-info (and checked
# out above) rather than the live PR head. Using `gh pr diff` would read
# whatever the PR points at now, so a force-push after get-pr-info could
# slip a modified CLAUDE.md/AGENTS.md past this guard while different
# code is what actually runs (TOCTOU). Pinning to BASE_SHA...HEAD_SHA
# ties the check to the code Claude will see.
#
# Use the diff media type, not the compare JSON: the JSON `.files` array
# is capped at 300 entries (and `--paginate` only pages commits, not
# files), so a >300-file PR could hide an agent-config edit. The diff
# emits a `diff --git a/<old> b/<new>` header for every changed file,
# including renames, so matching those lines cannot miss a path.
run: |
CHANGED=$(gh api "repos/${{ github.repository }}/compare/${BASE_SHA}...${HEAD_SHA}" -H 'Accept: application/vnd.github.v3.diff' | grep '^diff --git ')
if echo "$CHANGED" | grep -qiE '/(AGENTS\.md|CLAUDE\.md)( |$)|/\.claude/'; then
echo "::error::PR modifies agent config files (AGENTS.md, CLAUDE.md, or .claude/). Skipping for security."
exit 1
fi
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
BASE_SHA: ${{ needs.get-pr-info.outputs.pr_base_sha }}
HEAD_SHA: ${{ needs.get-pr-info.outputs.pr_head_sha }}
- name: Install bubblewrap
run: sudo apt-get install -y bubblewrap
- uses: anthropics/claude-code-action@428971d2ecd6e3a7cb0ee0da2a3a8b33fdb3678d # v1.0.157
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
additional_permissions: |
actions: read
claude_args: |
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr checks:*),Bash(gh pr list:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh run view:*),Bash(gh run list:*),Bash(git log:*),Bash(git diff:*),Bash(git grep:*),Bash(git show:*),Bash(git status:*),Bash(rg:*),Bash(ls:*),Bash(tree:*),Bash(grep:*)"
+375
View File
@@ -0,0 +1,375 @@
name: PR Bots
on:
# zizmor: ignore[dangerous-triggers] -- pull_request_target is needed for fork PRs; inputs are validated before use
pull_request_target:
types: [opened, synchronize, labeled]
permissions: {}
concurrency:
group: pr-bots-${{ github.event.pull_request.number }}
cancel-in-progress: ${{ github.event.action == 'synchronize' }}
jobs:
size-label:
name: Size Label
if: github.event.action != 'labeled'
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
pull-requests: write
steps:
- name: Calculate PR size and set label
run: |
# Fetch file changes for this PR
FILES=$(gh api repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/files --paginate | jq -s 'add')
# Calculate lines by category (excluding uv.lock and cassettes)
CODE=$(echo "$FILES" | jq '[.[] | select(
(.filename | (startswith("tests/") or startswith("docs/") or endswith(".md")) | not) and
(.filename != "uv.lock") and
(.filename | contains("/cassettes/") | not)
) | .additions + .deletions] | add // 0')
DOCS=$(echo "$FILES" | jq '[.[] | select(
(.filename | (startswith("docs/") or endswith(".md"))) and
(.filename != "uv.lock") and
(.filename | contains("/cassettes/") | not)
) | .additions + .deletions] | add // 0')
TESTS=$(echo "$FILES" | jq '[.[] | select(
(.filename | startswith("tests/")) and
(.filename | contains("/cassettes/") | not) and
(.filename | endswith(".md") | not)
) | .additions + .deletions] | add // 0')
# Calculate weighted score: code + 50% docs + 50% tests
SCORE=$((CODE + DOCS / 2 + TESTS / 2))
echo "Code: $CODE, Docs: $DOCS, Tests: $TESTS"
echo "Weighted score: $SCORE"
# Determine size label based on cutoffs
if [ $SCORE -le 100 ]; then
SIZE="size: S"
elif [ $SCORE -le 500 ]; then
SIZE="size: M"
elif [ $SCORE -le 1500 ]; then
SIZE="size: L"
else
SIZE="size: XL"
fi
echo "Size: $SIZE"
# Remove any existing size labels (except the one we're setting) via API
for label in "size: S" "size: M" "size: L" "size: XL"; do
if [ "$label" != "$SIZE" ]; then
gh api "repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/labels/${label}" --method DELETE 2>/dev/null || true
fi
done
# Add the new size label via API
gh api "repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/labels" --method POST -f "labels[]=$SIZE"
echo "Set label: $SIZE (score: $SCORE)"
env:
GH_TOKEN: ${{ github.token }}
# Security: The classify job runs the LLM with READ-ONLY permissions and no label API access.
# The LLM's output is validated against an allowlist before the apply job takes any write action.
# This prevents prompt injection from adding arbitrary labels (e.g. 'auto-review' to trigger the review job).
category-classify:
name: Category Classify
if: github.event.action != 'labeled'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
pull-requests: read
outputs:
category: ${{ steps.extract.outputs.category }}
skip: ${{ steps.check-label.outputs.has_label }}
steps:
- name: Check for modified config files
if: github.event.pull_request.head.repo.fork
run: |
CHANGED=$(gh pr diff ${{ github.event.pull_request.number }} --name-only --repo ${{ github.repository }})
if echo "$CHANGED" | grep -qiE '(^|/)AGENTS\.md$|(^|/)CLAUDE\.md$|(^|/)\.claude/'; then
echo "::error::PR modifies agent config files (AGENTS.md, CLAUDE.md, or .claude/). Skipping auto-labeling for security."
exit 1
fi
env:
GH_TOKEN: ${{ github.token }}
- name: Check if category label already exists
id: check-label
run: |
LABELS=$(gh pr view ${{ github.event.pull_request.number }} --repo ${{ github.repository }} --json labels --jq '.labels[].name')
CATEGORY_LABELS=("bug" "feature" "docs" "chore" "dependency")
for label in "${CATEGORY_LABELS[@]}"; do
if echo "$LABELS" | grep -q "^${label}$"; then
echo "has_label=true" >> $GITHUB_OUTPUT
echo "PR already has category label: $label"
exit 0
fi
done
echo "has_label=false" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ github.token }}
- name: Checkout repository
if: steps.check-label.outputs.has_label == 'false'
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Install bubblewrap
if: steps.check-label.outputs.has_label == 'false'
run: sudo apt-get install -y bubblewrap
- name: Classify PR with Claude Code
if: steps.check-label.outputs.has_label == 'false'
id: classify
uses: anthropics/claude-code-action@428971d2ecd6e3a7cb0ee0da2a3a8b33fdb3678d # v1.0.157
env:
ANTHROPIC_BASE_URL: ${{ secrets.CLAUDE_CODE_BASE_URL }}
with:
anthropic_api_key: ${{ secrets.CLAUDE_CODE_API_KEY || secrets.ANTHROPIC_API_KEY }}
github_token: ${{ github.token }}
allowed_non_write_users: "*"
allowed_bots: "pydanty"
claude_args: |
--allowedTools "Bash(gh pr view:*),Bash(gh pr diff:*)"
--json-schema '{"type":"object","properties":{"category":{"type":"string"}},"required":["category"]}'
prompt: |
Classify PR #${{ github.event.pull_request.number }} in ${{ github.repository }}.
Run `gh pr view ${{ github.event.pull_request.number }} --repo ${{ github.repository }}` for the title and description.
Run `gh pr diff ${{ github.event.pull_request.number }} --repo ${{ github.repository }} --name-only` for the changed files.
Categories:
- bug: Fixes broken behavior
- feature: New functionality
- docs: Documentation-only (no code changes)
- chore: CI, refactoring, dev dependencies, tests-only
- dependency: Production dependency updates
When both code and docs change, prefer `feature` or `bug` over `docs`.
If pyproject.toml files changed, run `gh pr diff ${{ github.event.pull_request.number }} --repo ${{ github.repository }}` to see the full diff and distinguish production deps (`dependency`) from dev-only deps in `[dependency-groups]` (`chore`).
Return your classification as JSON with a "category" field set to one of: bug, feature, docs, chore, dependency.
- name: Extract and validate category
if: steps.check-label.outputs.has_label == 'false'
id: extract
run: |
CATEGORY=$(echo "$STRUCTURED_OUTPUT" | jq -r .category | tr -d '[:space:]')
ALLOWED="bug feature docs chore dependency"
if ! echo "$ALLOWED" | grep -Fqw "$CATEGORY"; then
echo "::error::Invalid category '$CATEGORY' from Claude — must be one of: $ALLOWED"
exit 1
fi
echo "category=$CATEGORY" >> $GITHUB_OUTPUT
echo "Classified as: $CATEGORY"
env:
STRUCTURED_OUTPUT: ${{ steps.classify.outputs.structured_output }}
category-apply:
name: Category Apply
needs: category-classify
if: needs.category-classify.outputs.skip != 'true' && needs.category-classify.outputs.category != ''
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- name: Validate and apply category label
run: |
CATEGORY="${NEEDS_CATEGORY_CLASSIFY_OUTPUTS_CATEGORY}"
ALLOWED="bug feature docs chore dependency"
if ! echo "$ALLOWED" | grep -Fqw "$CATEGORY"; then
echo "::error::Invalid category '$CATEGORY' — must be one of: $ALLOWED"
exit 1
fi
gh api "repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/labels" \
--method POST -f "labels[]=$CATEGORY"
echo "Applied label: $CATEGORY"
env:
GH_TOKEN: ${{ github.token }}
NEEDS_CATEGORY_CLASSIFY_OUTPUTS_CATEGORY: ${{ needs.category-classify.outputs.category }}
review:
name: Review
needs: [size-label, category-apply]
if: >-
!failure() && !cancelled() &&
github.event.action == 'labeled' && github.event.label.name == 'auto-review'
runs-on: ubuntu-latest
timeout-minutes: 60
permissions:
contents: read
issues: write
pull-requests: write
actions: read
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v6
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
persist-credentials: false
- name: Check for modified config files
if: github.event.pull_request.head.repo.fork
run: |
CHANGED=$(gh pr diff ${{ github.event.pull_request.number }} --name-only --repo ${{ github.repository }})
if echo "$CHANGED" | grep -qiE '(^|/)AGENTS\.md$|(^|/)CLAUDE\.md$|(^|/)\.claude/'; then
echo "::error::PR modifies agent config files (AGENTS.md, CLAUDE.md, or .claude/). Skipping auto-review for security."
exit 1
fi
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Select review model
id: model
run: |
LABELS=$(gh pr view ${{ github.event.pull_request.number }} --repo ${{ github.repository }} --json labels --jq '.labels[].name')
echo "Labels: $LABELS"
CATEGORY=$(echo "$LABELS" | grep -E '^(bug|feature|docs|chore|dependency)$' | head -1)
SIZE=$(echo "$LABELS" | grep -E '^size: ' | head -1)
# Default to Opus for large/complex PRs
MODEL="claude-opus-4-6[1m]"
# Disabled for now because Sonnet reviews have been disappointing
# # Use Sonnet for docs, dependency, chore, or small/medium PRs
# if [ "$CATEGORY" = "docs" ] || [ "$CATEGORY" = "dependency" ] || [ "$CATEGORY" = "chore" ] || [ "$SIZE" = "size: S" ] || [ "$SIZE" = "size: M" ]; then
# MODEL="claude-sonnet-4-5[1m]"
# fi
echo "model=$MODEL" >> $GITHUB_OUTPUT
echo "Selected model: $MODEL (category: $CATEGORY, size: $SIZE)"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Gather PR context
run: |
# Use the script from the base repo (main branch), not the fork
gh api "repos/${REPO}/contents/scripts/gather-review-context.sh?ref=${BASE_REF}" --jq .content | base64 -d > /tmp/gather-review-context.sh
bash /tmp/gather-review-context.sh "$PR_NUMBER" "$REPO"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO: ${{ github.repository }}
BASE_REF: ${{ github.event.pull_request.base.ref }}
PR_NUMBER: ${{ github.event.pull_request.number }}
- name: Install bubblewrap
run: sudo apt-get install -y bubblewrap
- uses: anthropics/claude-code-action@428971d2ecd6e3a7cb0ee0da2a3a8b33fdb3678d # v1.0.157
env:
ANTHROPIC_BASE_URL: ${{ secrets.CLAUDE_CODE_BASE_URL }}
with:
anthropic_api_key: ${{ secrets.CLAUDE_CODE_API_KEY || secrets.ANTHROPIC_API_KEY }}
github_token: ${{ secrets.GITHUB_TOKEN }}
allowed_non_write_users: "*"
display_report: 'true'
additional_permissions: |
actions: read
claude_args: |
--model ${{ steps.model.outputs.model }}
--allowedTools "mcp__github_inline_comment__create_inline_comment,Bash(gh pr comment:*),Bash(gh pr diff:*),Bash(gh pr view:*),Bash(gh pr checks:*),Bash(gh pr list:*),Bash(gh issue view:*),Bash(gh issue list:*),Bash(gh run view:*),Bash(gh run list:*),Bash(gh api repos/${{ github.repository }}/pulls/comments/:*),Bash(gh api repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/comments:*),Bash(gh api repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/reviews:*),Bash(gh api repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments:*),Bash(git log:*),Bash(git diff:*),Bash(git grep:*),Bash(git show:*),Bash(git status:*),Bash(jq:*),Bash(cat:*),Bash(rg:*),Bash(ls:*),Bash(tree:*),Bash(grep:*),WebSearch,WebFetch"
prompt: |
REPO: ${{ github.repository }}
PR NUMBER: ${{ github.event.pull_request.number }}
PR AUTHOR: ${{ github.event.pull_request.user.login }} (${{ github.event.pull_request.author_association }})
Review this pull request. The PR branch is already checked out in the current working directory and the `CLAUDE.md` at the root (symlinked to `AGENTS.md`) is already loaded in your system prompt.
# What to look for
- If the PR should not have been created yet (e.g. no issue, insufficiently defined scope, not ready for implementation, duplicate of existing open PR), just leave a comment informing the user and maintainer of this and don't bother doing a thorough review.
- Any change that does not align with the project's standards, philosophy, or requirements for every contribution as stated in `AGENTS.md` (symlinked from `CLAUDE.md`).
- Any change that does not match maintainer guidance in the issue or earlier PR comments on what an acceptable solution would look like.
- Any change or design decision or tradeoff (in both behavior and API) that needs explicit consideration, discussion, or maintainer awareness and approval.
- Any line of code that violates the concrete guidelines/rules laid out in the relevant `AGENTS.md` file(s): the top-level guidelines apply to all changes, while directory-specific guidelines affect only the changes in that directory.
- Anything else that the responsibilities you are assigned in `AGENTS.md` suggest that you should be calling out: use your best judgment.
Generally, the priority in terms of "crucial to get right" and "what to focus on first in a new PR" is public API > concepts and behavior > documentation > tests > code style.
If the PR has high level problems that will likely require significant changes at lower levels, hold off on looking for or commenting on lower level problems until the higher level problems are addressed, so that the PR author (and your context window) don't get overwhelmed.
Note that while another agent (Devin) is responsible for thoroughly reviewing the implementation for bugs, security issues, and edge cases,
_you_ are responsible for catching every violation of the repository's standards and guidelines listed in the `AGENTS.md` and `agent_docs/*.md` files.
Do not focus exclusively on high-level concerns: by the time the author has addressed every comment you've left over multiple rounds of review, the PR should be ready to merge.
# Gathering context
Before doing anything else, read ALL of the following pre-gathered context files in a single parallel tool call:
- `.github/.review-context/pr-details.json` — PR title, body, author, branch info, labels, state, review decision, timestamps
- `.github/.review-context/pr-comments.txt` — existing top-level PR comments
- `.github/.review-context/review-comments.txt` — existing inline review comments (resolved+outdated threads and threads predating the last auto-review are collapsed to one-liners with comment IDs so you can fetch full details if needed)
- `.github/.review-context/related-issues.txt` — linked issues and their comments
- `.github/.review-context/changed-files.txt` — changed files with per-file addition/deletion counts (tab-separated; non-generated files include a third column with the path to their per-file diff)
- `.github/.review-context/agents-md.txt` — directory-specific `AGENTS.md` files for changed directories
- `agent_docs/index.md` - repo-wide coding guidelines
The diff is split into per-file diffs under `.github/.review-context/diff/` (excluding `uv.lock` and cassettes), that you can read on demand and in parallel.
The diffs include function-level context (`git diff -W`), so you can see the full function/method being modified without needing to read the source file separately.
Each commentable line in the diff is prefixed with its source line number: `NL:<number>` for new or context lines, `OL:<number>` for deleted lines.
For newly added files, the diff contains the complete file contents — do not re-read these from disk.
The diff file paths are listed in the third column of `changed-files.txt`.
The pre-gathered diffs are the source of truth for what this PR changes. Do not re-fetch diffs or file lists using `gh pr diff` or `gh api`.
When you need code context beyond what the diffs provide, use the `Read` tool on the checked-out source files.
Use the `gh` CLI only when you need additional information not already in these files (e.g. to read other referenced PRs or issues, check CI status, or read files excluded from the gathered diff).
Use specific `gh` subcommands (`gh pr view`, `gh issue view`, `gh run view`, etc.) rather than `gh api` for most queries.
`gh api` is scoped to comment and review endpoints on this PR only:
- `gh api repos/${{ github.repository }}/pulls/comments/<id>` — individual review comment by ID
- `gh api repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/comments` — list review comments
- `gh api repos/${{ github.repository }}/pulls/${{ github.event.pull_request.number }}/reviews` — list reviews
- `gh api repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/comments` — list issue comments
Be careful about loading large diffs if you're unlikely to need them yet, like massive test files when there's plenty of more interesting code to comment on first, as you don't want to blow your context window too early.
You will usually want to read all the "core implementation" and docs diffs in one go, though, so you have the full context of the PR as you identify problems, instead of going file by file.
# Posting comments
While gathering context and learning about the PR, keep track of problems/points of discussion as you find them, and wait to post comments until the end,
as the comments you write will be better, less duplicative and more focused on the changes that really matter if you have the full set of problems as context.
For each identified issue that is determined to be worth a new comment, use `mcp__github_inline_comment__create_inline_comment` to attach the feedback to a specific line of code.
- Only lines with an `NL:` or `OL:` prefix in the diff are commentable. For OL lines, use `side: LEFT`.
- Include the reasoning, but don't quote specific rules from the `AGENTS.md` files.
- Include a concrete suggestion if appropriate (but to not use ` ```suggestion ` blocks as they can render incorrectly when the line numbers are off)
- Include a ping to the maintainer (`@DouweM`) on any change that requires maintainer input before the PR author can move forward.
- If the same issue shows up in multiple places, post a comment on each instance but have later comments refer to the first comment using a link.
- Use `gh pr comment` only for important feedback that doesn't relate to a specific line or file, not for a summary of feedback you've already posted inline.
Your comments should be:
- actionable: they should request a change, flag a concern that needs discussion, and/or suggest an improvement; don't comment on positive aspects of the PR like "excellent design choices".
- concise and to the point: don't use unnecessary emojis, lists, or subheadings, but do link to code if appropriate; 1 to 3 paragraphs are pretty much always enough.
- friendly without being sycophantic: use the tone and language of a helpful and encouraging project maintainer, but no need to compliment the author on positive aspects of the PR or point out changes that are good.
- non-repetitive: don't repeat things pointed out in earlier review comments, unless it looks like they'll be forgotten if you don't point them out; e.g. when they're marked as resolved/outdated but the problem persists without a satisfactory resolution (like a maintainer comment saying the comment does not need to be addressed).
You are meant to be helpful to the contributor and the maintainer, so your comments should never add noise to the conversation:
- Do not post a final summary comment; inline comments are sufficient.
- Do not comment on lines that do not need improvement, maintainer awareness, or discussion; comments pointing out a good choice are just noise.
- Do not post multiple comments for the same exact issue unless it shows up in different places.
It bears repeating that you are the first line of defense against low-quality contributions and maintainer headaches, and you have a big role in ensuring that every contribution to this project meets or exceeds the high standards that the Pydantic brand is known and loved for.
- name: Remove auto-review label
if: always()
run: gh api "repos/${{ github.repository }}/issues/${{ github.event.pull_request.number }}/labels/auto-review" --method DELETE 2>/dev/null || true
env:
GH_TOKEN: ${{ github.token }}
+55
View File
@@ -0,0 +1,55 @@
name: CI Duration Report
# Label-triggered CI duration report. Adding `trigger:ci-duration-report` to a
# PR compares that PR's latest CI run against recent successful `main` and PR
# runs, then updates one sticky comment on the PR. This workflow never checks
# out or executes PR head code.
on:
# zizmor: ignore[dangerous-triggers] -- pull_request_target is required so fork PRs can
# receive comments from a base-repo token. The workflow is gated on a maintainer-applied
# label and only reads GitHub Actions metadata.
pull_request_target:
types: [labeled]
permissions: {}
concurrency:
group: ci-duration-report-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
report:
if: github.event.label.name == 'trigger:ci-duration-report'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
actions: read
contents: read
issues: write
pull-requests: read
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
python-version: "3.12"
- name: Create or update CI duration report
run: uv run --no-project --with certifi python .github/scripts/ci_duration.py report --poll-seconds 600
env:
GITHUB_TOKEN: ${{ github.token }}
GITHUB_REPOSITORY: ${{ github.repository }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
- name: Remove trigger label
if: always()
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: gh api -X DELETE "repos/${REPO}/issues/${PR_NUMBER}/labels/trigger:ci-duration-report" || true
+522
View File
@@ -0,0 +1,522 @@
name: CI
on:
push:
branches:
- main
tags:
- "**"
pull_request: {}
env:
COLUMNS: 150
UV_PYTHON: 3.12
UV_FROZEN: "1"
permissions:
contents: read
jobs:
lint:
name: lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
python-version: "3.13"
enable-cache: true
cache-suffix: lint
- name: Install dependencies
run: uv sync --all-extras --all-packages --group lint
# pyright typechecks the gh-aw shim (.github/scripts/pydantic_ai_gh_aw_shim),
# which imports pydantic-ai-harness. The harness depends on pydantic-ai-slim,
# whose name is shadowed by the workspace member, so keeping it in the
# universal lock breaks the lowest-direct re-resolution. Install it into the
# synced venv out-of-band instead (--no-deps: pydantic-ai-slim is already
# present from the workspace). Keep the pin in sync with the runner's
# (.github/scripts/pydantic-ai-runner).
- run: uv pip install --no-deps "pydantic-ai-harness>=0.4.0"
- uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # v3.0.1
with:
extra_args: --all-files --verbose
env:
SKIP: no-commit-to-branch
# pre-commit hooks shell out via `uv run`; without this they would
# re-sync the venv and drop the out-of-band harness install above.
UV_NO_SYNC: "1"
- run: uv build --all-packages
- run: ls -lh dist/
# mypy and lint are a bit slower than other jobs, so we run them separately
mypy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
cache-suffix: mypy
- name: Install dependencies
run: uv sync --no-dev --group lint
- run: make typecheck-mypy
docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
cache-suffix: docs
- run: uv sync --group docs
- run: make docs
- run: tree -sh site
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
- run: npm install
working-directory: docs-site
- run: npm run typecheck
working-directory: docs-site
- name: Store docs
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: site
path: site
# check all docs images are tinified, You'll need an API key from https://tinify.com/ to fix this if it fails
- run: uvx tinicly docs --check
test:
name: test on ${{ matrix.python-version }} (${{ matrix.install.name }})
# Use Ubicloud 4-core runners for all-extras (the slowest variant) when run by maintainers or opted in via 'ci:fast' label
# Use 'ci:slow' label to force standard GitHub runners (e.g. during Ubicloud outages)
runs-on: >-
${{
!contains(github.event.pull_request.labels.*.name, 'ci:slow')
&& matrix.install.name == 'all-extras'
&& (
github.event.pull_request.head.repo.full_name == github.repository
|| contains(github.event.pull_request.labels.*.name, 'ci:fast')
)
&& 'ubicloud-premium-4'
|| 'ubuntu-latest'
}}
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
install:
- name: pydantic-ai-slim
command: "--package pydantic-ai-slim"
- name: pydantic-evals
command: "--package pydantic-evals"
- name: standard
command: ""
- name: all-extras
command: "--all-extras"
env:
CI: true
COVERAGE_PROCESS_START: ./pyproject.toml
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-suffix: ${{ matrix.install.name }}
- run: mkdir .coverage
- run: uv sync --only-dev
- name: cache HuggingFace models
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.cache/huggingface
key: hf-${{ runner.os }}-${{ hashFiles('**/uv.lock') }}
restore-keys: |
hf-${{ runner.os }}-
# Warm the HF cache out-of-band (retried, non-fatal) so the sentence-transformers
# test model is on disk when the ST tests load it offline (see the
# `stsb_bert_tiny_model` fixture). On a sustained HF outage the cache stays
# cold and those tests skip instead of erroring the whole job.
- name: pre-download sentence-transformers test model
if: matrix.install.name == 'all-extras'
continue-on-error: true
run: |
for i in 1 2 3; do
if uv run --all-extras python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('sentence-transformers-testing/stsb-bert-tiny-safetensors')"; then
exit 0
fi
echo "HF model download attempt $i failed; retrying in 15s..."
sleep 15
done
echo "sentence-transformers test model unavailable after retries; ST tests will skip"
exit 1
# `-n logical`, not `-n auto`: xdist `auto` counts *physical* cores, which
# under-subscribes Ubicloud's hyperthreaded vCPUs (premium-4 -> 2 workers
# instead of 4). `logical` uses all vCPUs; ~39% faster on premium-4, no-op
# on ubuntu-latest (already 4). See PR benchmark.
- run: uv run ${{ matrix.install.command }} coverage run -m pytest --durations=100 -n logical --dist=loadgroup
env:
COVERAGE_FILE: .coverage/.coverage.${{ matrix.python-version }}-${{ matrix.install.name }}
- name: store coverage files
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: coverage-${{ matrix.python-version }}-${{ matrix.install.name }}
path: .coverage
include-hidden-files: true
test-lowest-versions:
name: test on ${{ matrix.python-version }} (lowest-versions)
# Use Ubicloud 4-core runners for maintainers or opted in via 'ci:fast' label
# Use 'ci:slow' label to force standard GitHub runners (e.g. during Ubicloud outages)
runs-on: >-
${{
!contains(github.event.pull_request.labels.*.name, 'ci:slow')
&& (
github.event.pull_request.head.repo.full_name == github.repository
|| contains(github.event.pull_request.labels.*.name, 'ci:fast')
)
&& 'ubicloud-premium-4'
|| 'ubuntu-latest'
}}
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
env:
CI: true
COVERAGE_PROCESS_START: ./pyproject.toml
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-suffix: lowest-versions
- run: mkdir .coverage
- name: cache HuggingFace models
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: ~/.cache/huggingface
key: hf-${{ runner.os }}-${{ hashFiles('**/uv.lock') }}
restore-keys: |
hf-${{ runner.os }}-
- run: uv sync --all-extras --resolution lowest-direct
env:
UV_FROZEN: "0"
# Warm the HF cache out-of-band (see the `test` job) so the ST tests load
# the model offline from disk instead of revalidating it against HF Hub.
- name: pre-download sentence-transformers test model
continue-on-error: true
run: |
for i in 1 2 3; do
if uv run --no-sync python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('sentence-transformers-testing/stsb-bert-tiny-safetensors')"; then
exit 0
fi
echo "HF model download attempt $i failed; retrying in 15s..."
sleep 15
done
echo "sentence-transformers test model unavailable after retries; ST tests will skip"
exit 1
# `-n logical` (see note on the `test` job): use all vCPUs on Ubicloud.
- run: uv run --no-sync coverage run -m pytest --durations=100 -n logical --dist=loadgroup
env:
COVERAGE_FILE: .coverage/.coverage.${{matrix.python-version}}-lowest-versions
- name: store coverage files
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: coverage-${{ matrix.python-version }}-lowest-versions
path: .coverage
include-hidden-files: true
test-examples:
name: test examples on ${{ matrix.python-version }}
runs-on: ubuntu-latest
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
python-version: ["3.11", "3.12", "3.13", "3.14"]
env:
CI: true
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-suffix: examples
- run: uv run --all-extras python tests/import_examples.py
coverage:
runs-on: ubuntu-latest
needs: [test, test-lowest-versions]
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
# needed for diff-cover
fetch-depth: 0
persist-credentials: false
- name: get coverage files
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
merge-multiple: true
path: .coverage
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
cache-suffix: dev
- run: uv sync --group dev
- run: uv run coverage combine
- run: uv run coverage report
- run: uv run strict-no-cover
env:
COVERAGE_FILE: .coverage/.coverage
- run: uv run coverage html --show-contexts --title "Pydantic AI coverage for ${{ github.sha }}"
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: coverage-html
path: htmlcov
include-hidden-files: true
# https://github.com/marketplace/actions/alls-green#why used for branch protection checks
check:
if: always()
needs:
- lint
- mypy
- docs
- test
- test-lowest-versions
- test-examples
- coverage
runs-on: ubuntu-latest
steps:
- name: Decide whether the needed jobs succeeded or failed
uses: re-actors/alls-green@05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe # release/v1
with:
jobs: ${{ toJSON(needs) }}
# Note: this should match the `deploy-docs-manual` job in manually-deploy-docs.yml
deploy-docs:
needs: [check]
if: success() && startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
steps:
- name: Generate app token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
id: app-token
with:
app-id: ${{ vars.DOCS_APP_ID }}
private-key: ${{ secrets.DOCS_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: unified-docs
- name: Trigger unified-docs deployment
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
run: |
gh api repos/pydantic/unified-docs/dispatches \
--method POST \
-f event_type=docs-update \
-f "client_payload[source]=pydantic/pydantic-ai@${{ github.sha }}"
deploy-docs-preview:
needs: [check]
if: success() && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
environment:
name: deploy-docs-preview
permissions:
deployments: write
statuses: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
- run: npm install
working-directory: docs-site
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: true
cache-suffix: deploy-docs-preview
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: site
path: site
- uses: cloudflare/wrangler-action@ebbaa1584979971c8614a24965b4405ff95890e0 # v4.0.0
id: deploy
with:
apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
environment: previews
workingDirectory: docs-site
command: >
deploy
--var GIT_COMMIT_SHA:${{ github.sha }}
--var GIT_BRANCH:main
- name: Set preview URL
run: uv run --no-project --with httpx .github/set_docs_main_preview_url.py
env:
DEPLOY_OUTPUT: ${{ steps.deploy.outputs.command-output }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPOSITORY: ${{ github.repository }}
REF: ${{ github.sha }}
# Build wheels in an isolated job with no publish credentials. If anything in
# `uv build` (build backend, transitive build deps, restored cache) is
# compromised, the OIDC token for PyPI is in a separate job that only runs
# `pypi-publish` against the already-built artifacts.
release-build:
name: build release artifacts
needs: [check]
if: success() && startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
outputs:
package-version: ${{ steps.inspect_package.outputs.version }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
enable-cache: false
- run: uv build --all-packages
- name: Inspect package version
id: inspect_package
run: |
uv tool install --with uv-dynamic-versioning hatchling
version=$(uvx hatchling version)
echo "version=$version" >> "$GITHUB_OUTPUT"
- name: Upload distribution artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-dist
path: dist/
release:
name: publish to PyPI
needs: [release-build]
if: success() && startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
environment:
name: release
url: https://pypi.org/project/pydantic-ai/${{ needs.release-build.outputs.package-version }}
permissions:
id-token: write
outputs:
package-version: ${{ needs.release-build.outputs.package-version }}
steps:
- name: Download distribution artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: release-dist
path: dist/
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # release/v1
with:
skip-existing: true
send-tweet:
name: Send tweet
needs: [release]
if: needs.release.result == 'success'
runs-on: ubuntu-latest
steps:
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: "3.12"
- name: Install dependencies
run: pip install tweepy==4.14.0
- name: Send tweet
shell: python
run: |
import os
import tweepy
client = tweepy.Client(
access_token=os.getenv("TWITTER_ACCESS_TOKEN"),
access_token_secret=os.getenv("TWITTER_ACCESS_TOKEN_SECRET"),
consumer_key=os.getenv("TWITTER_CONSUMER_KEY"),
consumer_secret=os.getenv("TWITTER_CONSUMER_SECRET"),
)
version = os.getenv("VERSION").strip('"')
tweet = os.getenv("TWEET").format(version=version)
client.create_tweet(text=tweet)
env:
VERSION: ${{ needs.release.outputs.package-version }}
TWEET: |
Pydantic AI version {version} is out! 🎉
https://github.com/pydantic/pydantic-ai/releases/tag/v{version}
TWITTER_CONSUMER_KEY: ${{ secrets.TWITTER_CONSUMER_KEY }}
TWITTER_CONSUMER_SECRET: ${{ secrets.TWITTER_CONSUMER_SECRET }}
TWITTER_ACCESS_TOKEN: ${{ secrets.TWITTER_ACCESS_TOKEN }}
TWITTER_ACCESS_TOKEN_SECRET: ${{ secrets.TWITTER_ACCESS_TOKEN_SECRET }}
+106
View File
@@ -0,0 +1,106 @@
name: Docs Preview
# Label-triggered docs preview. Adding `trigger:docs` to a PR dispatches a
# preview build to pydantic/unified-docs; that repo builds the unified docs
# against this PR's head SHA, deploys to a shared Cloudflare Worker, and
# updates a comment on this PR with the preview URL.
#
# This workflow is purely a dispatcher — the build, deploy, and final
# URL-comment all happen in the unified-docs repo. The `trigger:docs` label
# is removed at the end so re-adding it re-runs the preview.
on:
# zizmor: ignore[dangerous-triggers] -- pull_request_target is required so fork PRs can
# access the DOCS_APP credentials. The dispatch is gated on a maintainer adding the
# `trigger:docs` label, and this workflow does not check out or execute PR code.
pull_request_target:
types: [labeled]
permissions: {}
concurrency:
group: docs-preview-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
dispatch:
if: github.event.label.name == 'trigger:docs'
runs-on: ubuntu-latest
permissions:
# Needed to post the "queued" comment and to remove the trigger:docs label.
pull-requests: write
steps:
- name: Generate app token (for dispatching to unified-docs)
id: app-token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
with:
app-id: ${{ vars.DOCS_APP_ID }}
private-key: ${{ secrets.DOCS_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: unified-docs
- name: Dispatch preview build
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
REPO: ${{ github.repository }}
run: |
gh api repos/pydantic/unified-docs/dispatches \
--method POST \
-f event_type=docs-preview \
-f "client_payload[library]=ai" \
-f "client_payload[source_repo]=${REPO}" \
-f "client_payload[source_pr]=${PR_NUMBER}" \
-f "client_payload[source_sha]=${HEAD_SHA}"
- name: Acknowledge on PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
REPO: ${{ github.repository }}
run: |
sha7=$(echo "$HEAD_SHA" | cut -c1-7)
body=$(cat <<EOF
## Docs Preview — queued
The preview build for commit \`${sha7}\` has been dispatched to [pydantic/unified-docs](https://github.com/pydantic/unified-docs/actions/workflows/docs-preview.yml). This comment will be updated with the preview URL once the build completes.
EOF
)
# Update an existing "Docs Preview" comment if present (covers re-runs); otherwise post a new one.
existing=$(gh api "repos/${REPO}/issues/${PR_NUMBER}/comments" \
--jq '[.[] | select(.user.login == "github-actions[bot]" and (.body | startswith("## Docs Preview")))][0].url // empty')
if [ -n "$existing" ]; then
gh api -X PATCH "$existing" -f body="$body"
else
gh api "repos/${REPO}/issues/${PR_NUMBER}/comments" -f body="$body"
fi
- name: Comment failure on PR
if: failure()
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
body=$(cat <<EOF
## Docs Preview — dispatch failed
The dispatch to pydantic/unified-docs failed. See [the workflow run](${RUN_URL}) for details.
<sub>Re-add the \`trigger:docs\` label to retry.</sub>
EOF
)
gh api "repos/${REPO}/issues/${PR_NUMBER}/comments" -f body="$body"
- name: Remove trigger:docs label
if: always()
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: gh api -X DELETE "repos/${REPO}/issues/${PR_NUMBER}/labels/trigger:docs" || true
@@ -0,0 +1,66 @@
name: Gateway Model Health
on:
schedule:
# Run weekly on Monday at 10:00 UTC.
- cron: '0 10 * * 1'
workflow_dispatch: {}
env:
COLUMNS: 150
UV_PYTHON: 3.12
UV_FROZEN: "1"
permissions:
contents: read
jobs:
gateway-model-health:
environment:
name: gateway-model-health
runs-on: ubuntu-latest
timeout-minutes: 30
env:
CI: true
PYDANTIC_AI_GATEWAY_API_KEY: ${{ secrets.PYDANTIC_AI_GATEWAY_API_KEY }}
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- uses: astral-sh/setup-uv@fac544c07dec837d0ccb6301d7b5580bf5edae39 # v8.2.0
with:
python-version: "3.12"
enable-cache: true
cache-suffix: gateway-model-health
- name: Install dependencies
run: uv sync --all-extras
- name: Run gateway model catalog checks
run: uv run pytest tests/providers/test_gateway_catalog.py -q --run-gateway-live
- name: Notify Slack on failure
if: failure() && !cancelled()
uses: slackapi/slack-github-action@45a88b9581bfab2566dc881e2cd66d334e621e2c # v3.0.3
with:
errors: true
webhook: ${{ secrets.SLACK_WEBHOOK_URL }}
webhook-type: incoming-webhook
payload: |
text: ":x: Gateway model health failed in ${{ github.repository }} on ${{ github.ref_name }}."
blocks:
- type: section
text:
type: mrkdwn
text: ":x: *Gateway model health failed*"
- type: section
fields:
- type: mrkdwn
text: "*Repository*\n<${{ github.server_url }}/${{ github.repository }}|${{ github.repository }}>"
- type: mrkdwn
text: "*Trigger*\n${{ github.event_name }}"
- type: mrkdwn
text: "*Ref*\n`${{ github.ref_name }}`"
- type: mrkdwn
text: "*Run*\n<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View workflow run>"
+39
View File
@@ -0,0 +1,39 @@
name: Harness Compat
# Verifies that pydantic-ai changes don't break the pydantic-ai-harness lint /
# typecheck / test suite. Calls the harness's `compat-test.yml` reusable
# workflow with the change's HEAD SHA + repo (so fork PRs are tested against
# the code in the fork, not main).
#
# Triggers:
# - PRs touching `pydantic_ai_slim/**`: required to merge.
# - Pushes to `main`: confirms the merged commit still works.
# - Push of `v*` tags: gates a release on harness compatibility.
#
# Fork PRs run with `contents: read` only and no secrets passed to the called
# workflow — same exposure as the regular test jobs in `ci.yml`. If the called
# workflow ever grows steps that need elevated permissions or secrets, add a
# fork gate (e.g. require a `safe-for-ci` label) before merging that change.
on:
pull_request:
types: [opened, synchronize, reopened]
paths:
- 'pydantic_ai_slim/**'
- '.github/workflows/harness-compat.yml'
push:
branches: [main]
tags: ['v*']
permissions:
contents: read
jobs:
harness-compat:
name: harness compat
# Same-org reusable workflow under shared maintenance; SHA pinning would force a coordination bump
# on every harness change without security benefit. Ignore configured in `.github/zizmor.yml`.
uses: pydantic/pydantic-ai-harness/.github/workflows/compat-test.yml@main
with:
pydantic-ai-ref: ${{ github.event.pull_request.head.sha || github.sha }}
pydantic-ai-repo: ${{ github.event.pull_request.head.repo.full_name || github.repository }}
+34
View File
@@ -0,0 +1,34 @@
name: Link check
on:
schedule:
- cron: '0 0 * * 1' # weekly on Monday
workflow_dispatch:
permissions: {}
jobs:
link-check:
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false
- name: Check links
id: lychee
uses: lycheeverse/lychee-action@8646ba30535128ac92d33dfc9133794bfdd9b411 # v2.8.0
with:
args: --no-progress --config .github/workflows/lychee.toml './docs/**/*.md' './*.md'
fail: false
- name: Open issue on broken links
if: steps.lychee.outputs.exit_code != 0
uses: peter-evans/create-issue-from-file@fca9117c27cdc29c6c4db3b86c48e4115a786710 # v6.0.0
with:
title: Broken links found in docs
content-filepath: ./lychee/out.md
labels: docs
+33
View File
@@ -0,0 +1,33 @@
# Configuration for the scheduled docs link check (see link-check.yml).
# Tuned to avoid false positives from sites that block bots or rate-limit, so the
# issue it opens only lists genuinely broken links.
# Some sites return 403/429 to non-browser clients even though the link is fine.
accept = ["200..=299", "403", "429"]
# Browser-like UA reduces bot-blocking 403s.
user_agent = "Mozilla/5.0 (compatible; lychee link checker)"
max_redirects = 10
max_retries = 3
retry_wait_time = 2
timeout = 30
# Don't check localhost example URLs or form-submission endpoints.
exclude_all_private = true
exclude_loopback = true
# Relative doc links are resolved by mkdocs, not as filesystem paths; skip the
# `file://` links lychee derives from them.
scheme = ["https", "http"]
# Login-gated or bot-blocking hosts that return non-2xx to automated checks even
# though the links are valid in a browser; not worth failing the run over.
exclude = [
"platform\\.openai\\.com",
"openai\\.com",
"console\\.mistral\\.ai",
"dash\\.voyageai\\.com",
"spec\\.modelcontextprotocol\\.io",
"customerioforms\\.com",
]
@@ -0,0 +1,30 @@
name: Manual Docs Deploy
on:
workflow_dispatch:
permissions:
contents: read
jobs:
# Note: this should match the `deploy-docs` job in ci.yml
deploy-docs-manual:
runs-on: ubuntu-latest
steps:
- name: Generate app token
uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
id: app-token
with:
app-id: ${{ vars.DOCS_APP_ID }}
private-key: ${{ secrets.DOCS_APP_PRIVATE_KEY }}
owner: ${{ github.repository_owner }}
repositories: unified-docs
- name: Trigger unified-docs deployment
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
run: |
gh api repos/pydantic/unified-docs/dispatches \
--method POST \
-f event_type=docs-update \
-f "client_payload[source]=pydantic/pydantic-ai (manual)"
+421
View File
@@ -0,0 +1,421 @@
name: PR Guard
on:
# zizmor: ignore[dangerous-triggers] -- pull_request_target is needed to close duplicate/unlinked PRs and manage issues; no fork code is checked out
pull_request_target:
types: [opened, reopened, ready_for_review, edited]
permissions: {}
concurrency:
group: pr-guard-${{ github.event.pull_request.number }}
jobs:
guard:
name: Guard
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
issues: write
pull-requests: write
steps:
- name: Check linked issues and guard against duplicates
run: |
set -euo pipefail
PR_NUMBER="${GITHUB_EVENT_PULL_REQUEST_NUMBER}"
PR_AUTHOR="${GITHUB_EVENT_PULL_REQUEST_USER_LOGIN}"
AUTHOR_ASSOCIATION="${GITHUB_EVENT_PULL_REQUEST_AUTHOR_ASSOCIATION}"
echo "PR #${PR_NUMBER} by ${PR_AUTHOR} (${AUTHOR_ASSOCIATION})"
# `edited` also fires on closed and merged PRs (e.g. an author updating
# the description before reopening, as the close comment suggests, or a
# bot rewriting the description). Nothing to check, and `gh pr close`
# on an already-closed PR would fail the job.
if [ "${GITHUB_EVENT_PULL_REQUEST_STATE}" != "open" ]; then
echo "PR is ${GITHUB_EVENT_PULL_REQUEST_STATE}, skipping checks."
exit 0
fi
# Skip draft PRs — authors may create drafts to save progress
if [ "${GITHUB_EVENT_PULL_REQUEST_DRAFT}" = "true" ]; then
echo "PR is a draft, skipping checks."
exit 0
fi
# Only check PRs targeting the default branch. GitHub interprets closing
# keywords ("Fixes #123") *only* on default-branch PRs; on any other base
# branch they're ignored and closingIssuesReferences is always empty, so
# enforcing the issue-link policy there would close validly-linked PRs (e.g.
# a fix targeting a maintenance branch). Fail open if either value is empty.
if [ -z "${DEFAULT_BRANCH}" ] || [ -z "${GITHUB_EVENT_PULL_REQUEST_BASE_REF}" ] || [ "${GITHUB_EVENT_PULL_REQUEST_BASE_REF}" != "${DEFAULT_BRANCH}" ]; then
echo "PR base '${GITHUB_EVENT_PULL_REQUEST_BASE_REF}' is not the default branch '${DEFAULT_BRANCH}'; skipping checks."
exit 0
fi
# Maintainer bypass
if [[ "$AUTHOR_ASSOCIATION" == "MEMBER" || "$AUTHOR_ASSOCIATION" == "OWNER" || "$AUTHOR_ASSOCIATION" == "COLLABORATOR" ]]; then
echo "Author is a maintainer/collaborator, skipping checks."
exit 0
fi
# Author repo-role bypass. The webhook's `author_association` is unreliable — it
# can report CONTRIBUTOR for a genuine org member/collaborator (observed on #6359:
# `dsfaccini`, a public org member with `maintain` role, came through as
# CONTRIBUTOR, so the bypass above missed and the PR was wrongly closed). Confirm
# the author's actual role on THIS repo via the API — the same `role_name` signal
# the trusted-actor bypass below uses for the sender — and skip for triage-or-higher.
# Fails open toward *checking* (an unreadable/unknown role continues the checks), so
# an external contributor can never gain a bypass this way.
AUTHOR_ROLE=$(gh api "repos/${REPO}/collaborators/${PR_AUTHOR}/permission" --jq '.role_name' 2>/dev/null || echo "unknown")
case "$AUTHOR_ROLE" in
triage | write | maintain | admin)
echo "Author ${PR_AUTHOR} has ${AUTHOR_ROLE} access to this repo; skipping checks."
exit 0
;;
*)
echo "Author ${PR_AUTHOR} repo role: ${AUTHOR_ROLE}; continuing checks."
;;
esac
# Private-membership bypass. The `author_association` in the webhook only
# reflects *public* org membership, so a private member of the org that
# owns this repo is reported as CONTRIBUTOR and misses the bypass above.
# Confirm real membership via the API, which sees private members when the
# token has `read:org` — the default `github.token` cannot, so this uses
# `ORG_READ_TOKEN` (an existing org-scoped secret). Fails open: if that
# token is unset or lacks the scope the API call errors and the existing
# policy applies unchanged, so no unlinked PR is ever wrongly kept open.
if [ -n "${ORG_READ_TOKEN:-}" ] && GH_TOKEN="$ORG_READ_TOKEN" gh api "orgs/${REPO%/*}/members/${PR_AUTHOR}" >/dev/null 2>&1; then
echo "Author ${PR_AUTHOR} is an org member (confirmed via API, incl. private membership); skipping checks."
exit 0
fi
# Trusted-actor bypass. AUTHOR_ASSOCIATION above is the PR *author's*
# relationship to the repo, so without this the guard re-closes a
# contributor PR every time a maintainer acts on it: reopening it, or
# editing its title/body (`reopened` and `edited` both re-run this
# workflow, so a reopen-only bypass gets undone by the next edit). When
# the person who triggered this run isn't the author, confirm they have
# triage-or-higher access on *this* repo and, if so, treat their action
# as a deliberate decision to keep the PR open and skip the checks.
# - We read `role_name`, not the legacy `permission` (which collapses
# triage into read), so a triager — who can legitimately reopen —
# still counts.
# - We require `sender != author` and a base-repo role because GitHub
# also lets a collaborator on the contributor's *fork* reopen the PR;
# `sender != author` alone isn't proof of base-repo trust.
# - If the role can't be determined, err toward respecting the action:
# this is a courtesy gate (trivially satisfied by linking any open
# issue), not a security boundary, and a deliberate maintainer action
# shouldn't be undone by a transient API hiccup.
# The author acting on their own PR (e.g. the `opened` event, where the
# sender is always the author) still gets checked, so the issue-link
# policy can't be bypassed this way.
if [ "${GITHUB_EVENT_SENDER_LOGIN}" != "$PR_AUTHOR" ]; then
SENDER_ROLE=$(gh api "repos/${REPO}/collaborators/${GITHUB_EVENT_SENDER_LOGIN}/permission" --jq '.role_name' 2>/dev/null || echo "unknown")
case "$SENDER_ROLE" in
read | none)
echo "Event triggered by ${GITHUB_EVENT_SENDER_LOGIN} (role: ${SENDER_ROLE}, no write access); continuing checks."
;;
*)
echo "Event triggered by ${GITHUB_EVENT_SENDER_LOGIN} (role: ${SENDER_ROLE}); a trusted collaborator acted on this PR, skipping checks."
exit 0
;;
esac
fi
# pydanty is the autonomous agent that opens PRs from issues. Its PRs are
# never auto-closed for duplication; instead a pydanty PR supersedes a
# competing PR opened within WINDOW_SECONDS before it (see the loop below).
PYDANTY_LOGIN="pydanty[bot]"
WINDOW_SECONDS=600
IS_PYDANTY_PR="false"
if [ "$PR_AUTHOR" = "$PYDANTY_LOGIN" ]; then
IS_PYDANTY_PR="true"
fi
PR_CREATED_EPOCH=$(date -u -d "$GITHUB_EVENT_PULL_REQUEST_CREATED_AT" +%s)
# Helper: close a contributor PR that doesn't link to any existing issue.
# Only external contributors reach this point (maintainers/collaborators
# bypass above). Exemptions:
# - bot authors (pydanty, dependabot, etc.) — they never file issues first
# - docs-only changes — small doc fixes are welcome without an issue
close_for_missing_issue() {
case "$PR_AUTHOR" in
*"[bot]")
echo "Author ${PR_AUTHOR} is a bot; not closing for missing issue link."
return 0
;;
esac
# If the changed files can't be determined, err on the side of leaving
# the PR open: closing is destructive and the docs-only exemption
# can't be ruled out.
if ! CHANGED_FILES=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}/files?per_page=100" --paginate --jq '.[].filename') || [ -z "$CHANGED_FILES" ]; then
echo "Could not determine changed files; not closing for missing issue link."
return 0
fi
DOCS_ONLY="true"
while IFS= read -r CHANGED_FILE; do
case "$CHANGED_FILE" in
docs/*|*.md|mkdocs.yml) ;;
*) DOCS_ONLY="false"; break ;;
esac
done <<< "$CHANGED_FILES"
if [ "$DOCS_ONLY" = "true" ]; then
echo "PR only touches documentation; not closing for missing issue link."
return 0
fi
echo "Closing PR #${PR_NUMBER} — no linked issue."
COMMENT=$(printf '%s\n\n%s\n\n%s\n\n%s' \
"Thanks for the contribution! To make sure changes are discussed before code is written, we ask that every PR references an existing issue with a closing keyword (e.g. \`Fixes #1234\`) in its description, so this PR has been closed automatically." \
"If there's no issue for this yet, please open one first (e.g. a bug report with a reproducible example), wait for a maintainer to confirm it, then update this PR's description to reference it and reopen the PR." \
"Documentation-only fixes are exempt and don't need an issue." \
"Feel free to ping our team if you think this was closed in error.")
# Comment before closing so the PR can never end up closed without an
# explanation; if commenting fails, set -e stops us and the PR stays open.
gh pr comment "$PR_NUMBER" --repo "$REPO" --body "$COMMENT"
gh pr close "$PR_NUMBER" --repo "$REPO"
}
# Ask GitHub which issues this PR would close, rather than parsing the
# body ourselves: GitHub's own parser is the authority on closing-keyword
# syntax ("Fixes #123", "Fixes: #123", "Fixes owner/repo#123", full issue
# URLs) and only counts references that resolve to real issues. Closed
# issues are included (with state CLOSED), so the checks below still see
# them. References to issues in other repos don't count: the policy
# requires an issue in this repo. If the query fails, set -e fails the
# job and the PR is left open.
fetch_linked_issues() {
gh api graphql \
-f owner="${REPO%/*}" -f name="${REPO#*/}" -F number="$PR_NUMBER" \
-f query='
query($owner: String!, $name: String!, $number: Int!) {
repository(owner: $owner, name: $name) {
pullRequest(number: $number) {
closingIssuesReferences(first: 100) {
nodes { number repository { nameWithOwner } }
}
}
}
}' \
--jq ".data.repository.pullRequest.closingIssuesReferences.nodes[] | select(.repository.nameWithOwner == \"${REPO}\") | .number"
}
ISSUE_NUMBERS=$(fetch_linked_issues)
if [ -z "$ISSUE_NUMBERS" ]; then
# GitHub resolves closing references asynchronously, so a query right
# after a PR is opened or edited can transiently come back empty.
# Closing is destructive — wait and confirm before acting on it.
echo "No linked issues found; re-checking in 30s in case GitHub hasn't resolved references yet."
sleep 30
ISSUE_NUMBERS=$(fetch_linked_issues)
fi
if [ -z "$ISSUE_NUMBERS" ]; then
echo "No linked issues found in PR body."
close_for_missing_issue
exit 0
fi
echo "Found linked issues: $ISSUE_NUMBERS"
# Helper: attempt to assign a user to an issue.
# GitHub silently ignores assignees who lack push access (returns 201 but doesn't add them).
# We check assignability first via GET /repos/{owner}/{repo}/assignees/{user} (204 = yes, 404 = no).
try_assign() {
local issue_num="$1"
local user="$2"
if gh api "repos/${REPO}/assignees/${user}" > /dev/null 2>&1; then
gh api "repos/${REPO}/issues/${issue_num}/assignees" --method POST -f "assignees[]=${user}" > /dev/null
echo "Assigned ${user} to issue #${issue_num}."
return 0
else
echo "Cannot assign ${user} to issue #${issue_num} (user is not assignable to this repo)."
return 1
fi
}
FOUND_OPEN_ISSUE="false"
FOUND_CLOSED_ISSUE="false"
for ISSUE_NUM in $ISSUE_NUMBERS; do
echo ""
echo "--- Checking issue #${ISSUE_NUM} ---"
# Issues come from closingIssuesReferences, so they are known to exist;
# a fetch failure here is transient. Let set -e fail the job (leaving
# the PR open) rather than treating the reference as unresolved, which
# could get a validly-linked PR closed.
ISSUE_JSON=$(gh api "repos/${REPO}/issues/${ISSUE_NUM}")
# Skip if this is actually a pull request
IS_PR=$(echo "$ISSUE_JSON" | jq 'has("pull_request")')
if [ "$IS_PR" = "true" ]; then
echo "#${ISSUE_NUM} is a pull request, not an issue. Skipping."
continue
fi
# Skip closed issues — duplicate check only applies to open issues
ISSUE_STATE=$(echo "$ISSUE_JSON" | jq -r '.state')
if [ "$ISSUE_STATE" != "open" ]; then
echo "Issue #${ISSUE_NUM} is ${ISSUE_STATE}, skipping."
FOUND_CLOSED_ISSUE="true"
continue
fi
FOUND_OPEN_ISSUE="true"
ISSUE_AUTHOR=$(echo "$ISSUE_JSON" | jq -r '.user.login')
# Check for duplicate/blocking PRs first — this must run regardless of assignment outcome.
# As with the current PR's linked issues, ask GitHub which open PRs
# would close this issue instead of regex-matching PR bodies, so all
# closing syntaxes (and manually linked PRs) are recognized. Bot
# logins get their "[bot]" suffix restored to match the REST-style
# logins used elsewhere (PYDANTY_LOGIN, ISSUE_AUTHOR).
# Only PRs created before this one can block it: two PRs racing each
# other's guard runs would otherwise both see the other as open and
# both get closed, and a newer PR (e.g. pydanty's, deliberately left
# alongside an older human PR) must not close the older one when an
# edit re-triggers the guard. The pydanty supersede logic below is
# unaffected: it only ever targets PRs opened before pydanty's.
echo "Checking for existing PRs targeting issue #${ISSUE_NUM}..."
BLOCKING_PRS=""
FOUND_STALE_PR="false"
MATCHING_PRS=$(gh api graphql \
-f owner="${REPO%/*}" -f name="${REPO#*/}" -F number="$ISSUE_NUM" \
-f query='
query($owner: String!, $name: String!, $number: Int!) {
repository(owner: $owner, name: $name) {
issue(number: $number) {
closedByPullRequestsReferences(first: 100) {
nodes {
number
state
createdAt
repository { nameWithOwner }
author { __typename login }
labels(first: 100) { nodes { name } }
}
}
}
}
}' \
--jq ".data.repository.issue.closedByPullRequestsReferences.nodes[]
| select(.state == \"OPEN\" and .repository.nameWithOwner == \"${REPO}\" and .number != ${PR_NUMBER} and .createdAt < \"${GITHUB_EVENT_PULL_REQUEST_CREATED_AT}\")
| {number, created_at: .createdAt, user: {login: (if .author.__typename == \"Bot\" then .author.login + \"[bot]\" else .author.login end)}, labels: [.labels.nodes[]]}")
while IFS= read -r PR_JSON; do
[ -z "$PR_JSON" ] && continue
EXISTING_PR_NUM=$(echo "$PR_JSON" | jq -r '.number')
EXISTING_PR_AUTHOR=$(echo "$PR_JSON" | jq -r '.user.login')
echo "Found PR #${EXISTING_PR_NUM} by ${EXISTING_PR_AUTHOR} that references issue #${ISSUE_NUM}."
# Check if the existing PR has the Stale label
HAS_STALE=$(echo "$PR_JSON" | jq '[.labels[].name] | any(. == "Stale")')
if [ "$HAS_STALE" = "true" ]; then
echo "PR #${EXISTING_PR_NUM} is stale. Allowing new PR to supersede."
FOUND_STALE_PR="true"
continue
fi
# pydanty's own PR is never closed here. Instead it supersedes a
# competing PR that opened within WINDOW_SECONDS before it, unless
# that PR belongs to the issue author (who keeps priority).
if [ "$IS_PYDANTY_PR" = "true" ]; then
if [ "$EXISTING_PR_AUTHOR" = "$PYDANTY_LOGIN" ]; then
echo "PR #${EXISTING_PR_NUM} is also pydanty's; leaving it."
continue
fi
if [ "$EXISTING_PR_AUTHOR" = "$ISSUE_AUTHOR" ]; then
echo "PR #${EXISTING_PR_NUM} is by the issue author (${ISSUE_AUTHOR}); leaving both open for a maintainer."
continue
fi
EXISTING_EPOCH=$(date -u -d "$(echo "$PR_JSON" | jq -r '.created_at')" +%s)
DELTA=$(( PR_CREATED_EPOCH - EXISTING_EPOCH ))
if [ "$DELTA" -ge 0 ] && [ "$DELTA" -le "$WINDOW_SECONDS" ]; then
echo "pydanty PR #${PR_NUMBER} opened ${DELTA}s after PR #${EXISTING_PR_NUM} (<= ${WINDOW_SECONDS}s); superseding it."
SUPERSEDE_COMMENT=$(printf '%s\n\n%s' \
"An automated pull request from @${PYDANTY_LOGIN} addressing issue #${ISSUE_NUM} opened within ~10 minutes of this one, so it is taking precedence and this PR has been closed to avoid duplicate effort." \
"Thanks for contributing — if you'd like to keep working on this, please comment on [issue #${ISSUE_NUM}](https://github.com/${REPO}/issues/${ISSUE_NUM}) and a maintainer can help coordinate.")
gh pr comment "$EXISTING_PR_NUM" --repo "$REPO" --body "$SUPERSEDE_COMMENT"
gh pr close "$EXISTING_PR_NUM" --repo "$REPO"
else
echo "PR #${EXISTING_PR_NUM} opened more than ${WINDOW_SECONDS}s before pydanty's; leaving both open for a maintainer."
fi
continue
fi
# Collect all non-stale blocking PRs
if [ -z "$BLOCKING_PRS" ]; then
BLOCKING_PRS="#${EXISTING_PR_NUM}"
else
BLOCKING_PRS="${BLOCKING_PRS}, #${EXISTING_PR_NUM}"
fi
done <<< "$MATCHING_PRS"
if [ "$IS_PYDANTY_PR" != "true" ] && [ -n "$BLOCKING_PRS" ]; then
echo "Closing PR #${PR_NUMBER} — issue #${ISSUE_NUM} already has active PRs: ${BLOCKING_PRS}"
COMMENT=$(printf '%s\n\n%s\n\n%s' \
"Thanks for your interest in this issue! However, there are already open PRs addressing issue #${ISSUE_NUM}: ${BLOCKING_PRS}." \
"To avoid duplicate efforts, this PR has been closed. If you'd like to contribute, you can review the existing PRs or share your thoughts on [issue #${ISSUE_NUM}](https://github.com/${REPO}/issues/${ISSUE_NUM})." \
"If you believe the existing PRs are inactive, please comment on the issue and a maintainer can reassess.")
gh pr comment "$PR_NUMBER" --repo "$REPO" --body "$COMMENT"
gh pr close "$PR_NUMBER" --repo "$REPO"
exit 0
fi
if [ "$FOUND_STALE_PR" = "true" ]; then
echo "All existing PRs for issue #${ISSUE_NUM} are stale. Allowing new PR."
fi
# Now handle assignment (best-effort, does not gate duplicate check above)
ASSIGNEES=$(echo "$ISSUE_JSON" | jq -r '[.assignees[].login] | join(",")')
echo "Current assignees: ${ASSIGNEES:-none}"
if [ -z "$ASSIGNEES" ]; then
# No assignee — attempt to assign PR author (may fail if user lacks push access)
try_assign "$ISSUE_NUM" "$PR_AUTHOR" || true
elif echo ",$ASSIGNEES," | grep -qF ",${PR_AUTHOR},"; then
echo "Issue #${ISSUE_NUM} is already assigned to ${PR_AUTHOR}."
else
echo "Issue #${ISSUE_NUM} is assigned to someone else. Leaving assignment as-is for maintainers to review."
fi
done
# If none of the referenced numbers resolved to an actual issue (open or
# closed), the PR is effectively unlinked — same policy as no keywords at all.
if [ "$FOUND_OPEN_ISSUE" = "false" ] && [ "$FOUND_CLOSED_ISSUE" = "false" ]; then
echo "No referenced numbers resolved to actual issues."
close_for_missing_issue
exit 0
fi
# If we found closed issues but no open ones, close the PR (pydanty PRs are exempt).
if [ "$IS_PYDANTY_PR" != "true" ] && [ "$FOUND_CLOSED_ISSUE" = "true" ] && [ "$FOUND_OPEN_ISSUE" = "false" ]; then
echo "All referenced issues are closed. Closing PR."
gh pr comment "$PR_NUMBER" --repo "$REPO" --body "All issues referenced by this PR are already closed. If you believe an issue should be reopened, please comment on it first."
gh pr close "$PR_NUMBER" --repo "$REPO"
fi
env:
GH_TOKEN: ${{ github.token }}
# Read-only org-scoped token, used solely to confirm private org
# membership (see the private-membership bypass). Falls back to empty
# when unset, which the bypass handles by failing open.
ORG_READ_TOKEN: ${{ secrets.GH_AW_GITHUB_TOKEN }}
REPO: ${{ github.repository }}
GITHUB_EVENT_SENDER_LOGIN: ${{ github.event.sender.login }}
GITHUB_EVENT_PULL_REQUEST_NUMBER: ${{ github.event.pull_request.number }}
GITHUB_EVENT_PULL_REQUEST_USER_LOGIN: ${{ github.event.pull_request.user.login }}
GITHUB_EVENT_PULL_REQUEST_AUTHOR_ASSOCIATION: ${{ github.event.pull_request.author_association }}
GITHUB_EVENT_PULL_REQUEST_CREATED_AT: ${{ github.event.pull_request.created_at }}
GITHUB_EVENT_PULL_REQUEST_DRAFT: ${{ github.event.pull_request.draft }}
GITHUB_EVENT_PULL_REQUEST_STATE: ${{ github.event.pull_request.state }}
GITHUB_EVENT_PULL_REQUEST_BASE_REF: ${{ github.event.pull_request.base.ref }}
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,67 @@
---
emoji: "🐛"
name: "Pydantic AI Bug Hunter"
description: "Find a reproducible, user-impacting bug in pydantic-ai and file a report issue. Runs on the Pydantic AI gh-aw shim; the task prompt is iterable from a Logfire managed variable."
on: weekly on thursday
permissions:
contents: read
issues: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-bug-hunter
cancel-in-progress: true
tools:
github:
mode: gh-proxy
toolsets: [default]
safe-outputs:
footer: false
activation-comments: false
noop:
create-issue:
max: 1
title-prefix: "[bug-hunter] "
close-older-key: "[bug-hunter]"
close-older-issues: false
expires: 7d
timeout-minutes: 30
imports:
- shared/network-vendor-domains.md
- shared/otel-logfire.md
- shared/tool-hints.md
- shared/repo-context.md
- shared/rigor.md
- shared/adversarial-review.md
- shared/checkout.md
- shared/engine-minimax.md
- shared/pre-steps.md
- shared/pre-agent-steps.md
jobs:
fetch_dynamic_prompt:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
dynamic_prompt: ${{ steps.resolve.outputs.dynamic_prompt }}
steps:
- name: Check out the prompt resolver action and default prompt
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github/actions/fetch-dynamic-prompt
.github/workflows/shared/prompts/pydantic-ai-bug-hunter.md
sparse-checkout-cone-mode: false
- name: Resolve agent prompt (Logfire managed variable, else committed default)
id: resolve
uses: ./.github/actions/fetch-dynamic-prompt
with:
logfire-variable-key: gh_aw_pydantic_ai_bug_hunter_prompt
default-prompt-file: .github/workflows/shared/prompts/pydantic-ai-bug-hunter.md
logfire-read-key: ${{ secrets.LOGFIRE_READ_EXTERNAL_VARIABLES }}
logfire-base-url: ${{ secrets.LOGFIRE_URL || vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
---
${{ needs.fetch_dynamic_prompt.outputs.dynamic_prompt }}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,67 @@
---
emoji: "📚"
name: "Pydantic AI Docs Drift"
description: "Detect negative docs drift where existing documentation no longer matches the codebase, and file an issue. Runs on the Pydantic AI gh-aw shim; the task prompt is iterable from a Logfire managed variable."
on: weekly on monday
permissions:
contents: read
issues: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-docs-drift
cancel-in-progress: true
tools:
github:
mode: gh-proxy
toolsets: [default]
safe-outputs:
footer: false
activation-comments: false
noop:
create-issue:
max: 1
title-prefix: "[docs-drift] "
labels: [docs-drift]
close-older-key: "[docs-drift]"
close-older-issues: false
expires: 7d
timeout-minutes: 30
imports:
- shared/network-vendor-domains.md
- shared/otel-logfire.md
- shared/tool-hints.md
- shared/repo-context.md
- shared/rigor.md
- shared/checkout.md
- shared/engine-minimax.md
- shared/pre-steps.md
- shared/pre-agent-steps.md
jobs:
fetch_dynamic_prompt:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
dynamic_prompt: ${{ steps.resolve.outputs.dynamic_prompt }}
steps:
- name: Check out the prompt resolver action and default prompt
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github/actions/fetch-dynamic-prompt
.github/workflows/shared/prompts/pydantic-ai-docs-drift.md
sparse-checkout-cone-mode: false
- name: Resolve agent prompt (Logfire managed variable, else committed default)
id: resolve
uses: ./.github/actions/fetch-dynamic-prompt
with:
logfire-variable-key: gh_aw_pydantic_ai_docs_drift_prompt
default-prompt-file: .github/workflows/shared/prompts/pydantic-ai-docs-drift.md
logfire-read-key: ${{ secrets.LOGFIRE_READ_EXTERNAL_VARIABLES }}
logfire-base-url: ${{ secrets.LOGFIRE_URL || vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
---
${{ needs.fetch_dynamic_prompt.outputs.dynamic_prompt }}
File diff suppressed because one or more lines are too long
+137
View File
@@ -0,0 +1,137 @@
---
emoji: "🔎"
name: "Pydantic AI PR Review"
description: "AI-driven PR review on the Pydantic AI gh-aw shim: inline comments + a single review verdict. Prompt iterable from a Logfire managed variable; read-only via gh-aw safe-outputs."
on:
pull_request:
types: [opened, synchronize, ready_for_review]
workflow_dispatch:
# Fork-PR safety: only trigger when the actor has admin/maintainer/write
# access. Without this, any established external contributor's PR would
# consume the configured Anthropic key and a model run.
roles: [admin, maintainer, write]
# Skip if PR has the `auto-review` label (opts into the legacy bots.yml reviewer).
if: ${{ !contains(github.event.pull_request.labels.*.name, 'auto-review') }}
permissions:
contents: read
# safe-outputs perform the actual writes in a separate conclusion job; the
# agent job stays read-only (gh-aw strict mode requires this).
pull-requests: read
issues: read
concurrency:
# One review per PR; newer pushes supersede in-flight reviews.
group: ${{ github.workflow }}-pr-review-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
tools:
github:
mode: gh-proxy
# PR-scoped surface: read the PR, related issues, repo, and search.
toolsets: [pull_requests, repos, search, issues]
safe-outputs:
footer: false
activation-comments: false
noop:
create-pull-request-review-comment:
max: 30
submit-pull-request-review:
supersede-older-reviews: true
max: 1
timeout-minutes: 30
imports:
- shared/network-vendor-domains.md
- shared/otel-logfire.md
- shared/tool-hints.md
- shared/repo-context.md
- shared/rigor.md
- shared/review-context.md
- shared/checkout.md
- shared/engine-minimax.md
- shared/pre-steps.md
- shared/pre-agent-steps.md
pre-steps:
# Setting engine.command makes gh-aw skip ALL engine installation steps,
# which also drops the bundled AWF firewall binary install. Re-run gh-aw's
# own installer (the same call it makes for non-custom-command jobs).
- name: Install AWF firewall binary (skipped by custom engine.command)
run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.46
pre-agent-steps:
# Stage the committed launcher script at gh-aw's exec-able
# /tmp/gh-aw/bin/ path. Runs in pre-agent-steps (not pre-steps) because
# gh-aw's repository checkout happens between pre-steps and
# pre-agent-steps, and this step reads from .github/scripts/ in the
# workspace.
- name: Stage Pydantic AI gh-aw shim launcher
run: |
mkdir -p /tmp/gh-aw/bin
install -m 755 .github/scripts/pydantic-ai-runner-launch.sh /tmp/gh-aw/bin/pydantic-ai-runner-launch
# Warm the harness's uv script environment on the OPEN network so the
# firewalled agent reuses a warm cache (non-fatal on failure).
- name: Pre-warm Pydantic AI gh-aw shim uv environment
run: bash .github/scripts/prewarm-pydantic-ai-runner.sh
# Pre-fetch PR context into `/tmp/gh-aw/.review-context/`: pr-details, PR
# comments, review threads (with annotated diff hunks + resolved/outdated
# state), annotated per-file diffs, related issues, AGENTS.md excerpts for
# changed dirs, file orderings for sub-agent fan-out, and a PR-size summary.
# The agent reads these files instead of calling the GitHub API at run time.
# Non-fatal: missing context just reduces signal.
#
# The script lives at scripts/ (NOT .github/scripts/) because gh-aw's
# "Save/Restore agent config folders from base branch" step snapshots and
# restores `.github/` (and other managed agent-config folders) from the
# BASE branch — making any new file added under those folders unreliable
# for steps that run after the restore. `scripts/` is outside that set,
# matching where the legacy reviewer's gather-review-context.sh already
# lives. The script is a fork of scripts/gather-review-context.sh — see
# the TODO at the top of the fork.
- name: Gather PR review context
if: ${{ github.event.pull_request.number }}
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: |
set -uo pipefail
# Diagnostic: log the workspace state of both scripts/ and .github/scripts/
# so we can verify the gh-aw restore-from-base step left them untouched.
echo "::group::workspace inventory at gather time"
ls -la scripts/ 2>&1 | head -50 || true
echo "---"
ls -la .github/scripts/ 2>&1 | head -50 || true
echo "::endgroup::"
script=scripts/gather-pydantic-ai-review-context.sh
if [ -x "$script" ]; then
"$script" "$PR_NUMBER" "$REPO" \
|| echo "::warning::${script} failed; reviewer will run with less context"
else
echo "::warning::${script} not present; reviewer will run with less context"
fi
jobs:
fetch_dynamic_prompt:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
dynamic_prompt: ${{ steps.resolve.outputs.dynamic_prompt }}
steps:
- name: Check out the prompt resolver action and default prompt
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github/actions/fetch-dynamic-prompt
.github/workflows/shared/prompts/pydantic-ai-pr-review.md
sparse-checkout-cone-mode: false
- name: Resolve agent prompt (Logfire managed variable, else committed default)
id: resolve
uses: ./.github/actions/fetch-dynamic-prompt
with:
logfire-variable-key: gh_aw_pydantic_ai_pr_review_prompt
default-prompt-file: .github/workflows/shared/prompts/pydantic-ai-pr-review.md
logfire-read-key: ${{ secrets.LOGFIRE_READ_EXTERNAL_VARIABLES }}
logfire-base-url: ${{ secrets.LOGFIRE_URL || vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
---
${{ needs.fetch_dynamic_prompt.outputs.dynamic_prompt }}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,68 @@
---
emoji: "🔌"
name: "Pydantic AI Provider Mapping Sweep"
description: "Audit one model provider's request/response mapping against its SDK and file a reproducible bug. Rotates providers; runs on the Pydantic AI gh-aw shim; the prompt is iterable from a Logfire managed variable."
on: weekly on monday
permissions:
contents: read
issues: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-provider-mapping-sweep
cancel-in-progress: true
tools:
github:
mode: gh-proxy
toolsets: [default]
safe-outputs:
footer: false
activation-comments: false
noop:
create-issue:
max: 1
title-prefix: "[provider-mapping-sweep] "
labels: [provider-mapping-sweep]
close-older-key: "[provider-mapping-sweep]"
close-older-issues: false
expires: 7d
timeout-minutes: 30
imports:
- shared/network-vendor-domains.md
- shared/otel-logfire.md
- shared/tool-hints.md
- shared/repo-context.md
- shared/rigor.md
- shared/adversarial-review.md
- shared/checkout.md
- shared/engine-minimax.md
- shared/pre-steps.md
- shared/pre-agent-steps.md
jobs:
fetch_dynamic_prompt:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
dynamic_prompt: ${{ steps.resolve.outputs.dynamic_prompt }}
steps:
- name: Check out the prompt resolver action and default prompt
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github/actions/fetch-dynamic-prompt
.github/workflows/shared/prompts/pydantic-ai-provider-mapping-sweep.md
sparse-checkout-cone-mode: false
- name: Resolve agent prompt (Logfire managed variable, else committed default)
id: resolve
uses: ./.github/actions/fetch-dynamic-prompt
with:
logfire-variable-key: gh_aw_pydantic_ai_provider_mapping_sweep_prompt
default-prompt-file: .github/workflows/shared/prompts/pydantic-ai-provider-mapping-sweep.md
logfire-read-key: ${{ secrets.LOGFIRE_READ_EXTERNAL_VARIABLES }}
logfire-base-url: ${{ secrets.LOGFIRE_URL || vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
---
${{ needs.fetch_dynamic_prompt.outputs.dynamic_prompt }}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,68 @@
---
emoji: "🧭"
name: "Pydantic AI Provider Parity Explore"
description: "Explore one cross-cutting capability's support across all providers and file an issue for concrete parity gaps. Runs on the Pydantic AI gh-aw shim; the prompt is iterable from a Logfire managed variable."
on: weekly on tuesday
permissions:
contents: read
issues: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-provider-parity-explore
cancel-in-progress: true
tools:
github:
mode: gh-proxy
toolsets: [default]
safe-outputs:
footer: false
activation-comments: false
noop:
create-issue:
max: 1
title-prefix: "[provider-parity-explore] "
labels: [provider-parity-explore]
close-older-key: "[provider-parity-explore]"
close-older-issues: false
expires: 7d
timeout-minutes: 30
imports:
- shared/network-vendor-domains.md
- shared/otel-logfire.md
- shared/tool-hints.md
- shared/repo-context.md
- shared/rigor.md
- shared/adversarial-review.md
- shared/checkout.md
- shared/engine-minimax.md
- shared/pre-steps.md
- shared/pre-agent-steps.md
jobs:
fetch_dynamic_prompt:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
dynamic_prompt: ${{ steps.resolve.outputs.dynamic_prompt }}
steps:
- name: Check out the prompt resolver action and default prompt
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github/actions/fetch-dynamic-prompt
.github/workflows/shared/prompts/pydantic-ai-provider-parity-explore.md
sparse-checkout-cone-mode: false
- name: Resolve agent prompt (Logfire managed variable, else committed default)
id: resolve
uses: ./.github/actions/fetch-dynamic-prompt
with:
logfire-variable-key: gh_aw_pydantic_ai_provider_parity_explore_prompt
default-prompt-file: .github/workflows/shared/prompts/pydantic-ai-provider-parity-explore.md
logfire-read-key: ${{ secrets.LOGFIRE_READ_EXTERNAL_VARIABLES }}
logfire-base-url: ${{ secrets.LOGFIRE_URL || vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
---
${{ needs.fetch_dynamic_prompt.outputs.dynamic_prompt }}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,68 @@
---
emoji: "⏪"
name: "Pydantic AI Regression Detector"
description: "Detect behavioral regressions between the two most recent releases and file a reproducible report. Runs on the Pydantic AI gh-aw shim; the prompt is iterable from a Logfire managed variable."
on: weekly on wednesday
permissions:
contents: read
issues: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-regression-detector
cancel-in-progress: true
tools:
github:
mode: gh-proxy
toolsets: [default]
safe-outputs:
footer: false
activation-comments: false
noop:
create-issue:
max: 1
title-prefix: "[regression-detector] "
labels: [regression]
close-older-key: "[regression-detector]"
close-older-issues: false
expires: 7d
timeout-minutes: 30
imports:
- shared/network-vendor-domains.md
- shared/otel-logfire.md
- shared/tool-hints.md
- shared/repo-context.md
- shared/rigor.md
- shared/adversarial-review.md
- shared/checkout.md
- shared/engine-minimax.md
- shared/pre-steps.md
- shared/pre-agent-steps.md
jobs:
fetch_dynamic_prompt:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
dynamic_prompt: ${{ steps.resolve.outputs.dynamic_prompt }}
steps:
- name: Check out the prompt resolver action and default prompt
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github/actions/fetch-dynamic-prompt
.github/workflows/shared/prompts/pydantic-ai-regression-detector.md
sparse-checkout-cone-mode: false
- name: Resolve agent prompt (Logfire managed variable, else committed default)
id: resolve
uses: ./.github/actions/fetch-dynamic-prompt
with:
logfire-variable-key: gh_aw_pydantic_ai_regression_detector_prompt
default-prompt-file: .github/workflows/shared/prompts/pydantic-ai-regression-detector.md
logfire-read-key: ${{ secrets.LOGFIRE_READ_EXTERNAL_VARIABLES }}
logfire-base-url: ${{ secrets.LOGFIRE_URL || vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
---
${{ needs.fetch_dynamic_prompt.outputs.dynamic_prompt }}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,72 @@
---
emoji: "♻️"
name: "Pydantic AI Round-Trip Sweep"
description: "Find serialize/deserialize state-loss bugs across a message round-trip boundary and file a reproducible report. Runs on the Pydantic AI gh-aw shim; the prompt is iterable from a Logfire managed variable."
on: daily
permissions:
contents: read
issues: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-roundtrip-sweep
cancel-in-progress: true
tools:
github:
mode: gh-proxy
toolsets: [default]
safe-outputs:
footer: false
activation-comments: false
# Engine/model failures are tracked as ERROR spans in Logfire (service_name
# `gh-aw.pydantic-ai-roundtrip-sweep`) via the otel-logfire import + the shim's
# `instrument_pydantic_ai`, so we don't also file an auto-generated failure issue.
report-failure-as-issue: false
noop:
create-issue:
max: 1
title-prefix: "[roundtrip-sweep] "
labels: [roundtrip-sweep]
close-older-key: "[roundtrip-sweep]"
close-older-issues: false
expires: 7d
timeout-minutes: 30
imports:
- shared/network-vendor-domains.md
- shared/otel-logfire.md
- shared/tool-hints.md
- shared/repo-context.md
- shared/rigor.md
- shared/adversarial-review.md
- shared/checkout.md
- shared/engine-minimax.md
- shared/pre-steps.md
- shared/pre-agent-steps.md
jobs:
fetch_dynamic_prompt:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
dynamic_prompt: ${{ steps.resolve.outputs.dynamic_prompt }}
steps:
- name: Check out the prompt resolver action and default prompt
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github/actions/fetch-dynamic-prompt
.github/workflows/shared/prompts/pydantic-ai-roundtrip-sweep.md
sparse-checkout-cone-mode: false
- name: Resolve agent prompt (Logfire managed variable, else committed default)
id: resolve
uses: ./.github/actions/fetch-dynamic-prompt
with:
logfire-variable-key: gh_aw_pydantic_ai_roundtrip_sweep_prompt
default-prompt-file: .github/workflows/shared/prompts/pydantic-ai-roundtrip-sweep.md
logfire-read-key: ${{ secrets.LOGFIRE_READ_EXTERNAL_VARIABLES }}
logfire-base-url: ${{ secrets.LOGFIRE_URL || vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
---
${{ needs.fetch_dynamic_prompt.outputs.dynamic_prompt }}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,145 @@
---
emoji: "🔍"
name: "Pydantic AI Stale Issues Finder"
description: "Find open issues that are likely already resolved, obsolete, or tied to deprecated features, and file a report issue. Runs on the Pydantic AI gh-aw shim; the task prompt is iterable from a Logfire managed variable."
# Weekly on Monday: gh-aw scatters the run and auto-adds workflow_dispatch.
# Adjust to 'daily' or another weekly schedule to change frequency.
on: weekly on monday
permissions:
contents: read
issues: read
pull-requests: read
# Full git history: the agent needs `git log` to detect removed/renamed APIs
# referenced in open issues. fetch-depth: 0 gives the full commit history.
checkout:
fetch-depth: 0
concurrency:
group: ${{ github.workflow }}-stale-issues-finder
cancel-in-progress: true
network:
allowed:
- defaults
# Python/PyPI ecosystem — the harness installs its deps via `uv` at agent
# time; allow them through the AWF firewall.
- python
# ANTHROPIC_BASE_URL is a compile-time literal (below) so gh-aw already
# auto-allowlists the host; this explicit entry is a harmless safety net.
- api.minimax.io
# We register as the built-in `claude` engine and only override `command`, so
# gh-aw runs its full Claude proxy + credential-injection machinery for us.
# ANTHROPIC_BASE_URL MUST be a compile-time literal (not a ${{ vars.* }}
# expression): gh-aw derives the api-proxy target host AND the
# `--anthropic-api-base-path` from its parsed URL path at compile time. With a
# vars expression the path can't be parsed, so the proxy drops the `/anthropic`
# prefix and the upstream returns 404. Only ANTHROPIC_API_KEY stays a secret
# (injected by the AWF api-proxy, excluded from the agent container). MiniMax
# exposes an Anthropic-compatible API at https://api.minimax.io/anthropic.
runtimes:
uv: {}
engine:
id: claude
# Pulled from the repo's `vars.GH_AW_MODEL` (set out-of-band).
# gh-aw compiles this into the engine command's `--model <name>` argv,
# which the harness reads via `args.model`.
model: ${{ vars.GH_AW_MODEL }}
# The checked-out workspace is mounted no-exec in the AWF sandbox, so a
# pre-step stages a launcher in gh-aw's exec-able /tmp/gh-aw/bin that runs
# `uv run --script` against the workspace harness.
command: /tmp/gh-aw/bin/pydantic-ai-runner-launch
env:
ANTHROPIC_BASE_URL: https://api.minimax.io/anthropic
ANTHROPIC_API_KEY: ${{ secrets.MINIMAX_API_KEY }}
tools:
github:
mode: gh-proxy
toolsets: [default]
safe-outputs:
# Hide gh-aw's "Generated by …" footer on every safe-output;
# hidden gh-aw-workflow-id / gh-aw-tracker-id markers still get emitted
# for search-ability.
footer: false
activation-comments: false
noop:
create-issue:
max: 1
title-prefix: "[stale-finder] "
labels: [stale-issues]
close-older-key: "[stale-finder]"
close-older-issues: false
expires: 7d
# Note: elastic uses 2d with twice-weekly schedule. Adjust 'expires'
# and the schedule together if you change run frequency.
timeout-minutes: 60
imports:
- shared/network-vendor-domains.md
- shared/otel-logfire.md
- shared/tool-hints.md
- shared/repo-context.md
- shared/rigor.md
pre-steps:
# Setting engine.command makes gh-aw skip ALL engine installation steps,
# which also drops the bundled AWF firewall binary install. Re-run gh-aw's
# own installer (the same call it makes for non-custom-command jobs).
- name: Install AWF firewall binary (skipped by custom engine.command)
run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.46
pre-agent-steps:
# Stage the committed launcher script at gh-aw's exec-able
# /tmp/gh-aw/bin/ path. Runs in pre-agent-steps (not pre-steps) because
# gh-aw's repository checkout happens between pre-steps and
# pre-agent-steps, and this step reads from .github/scripts/ in the
# workspace.
- name: Stage Pydantic AI gh-aw shim launcher
run: |
mkdir -p /tmp/gh-aw/bin
install -m 755 .github/scripts/pydantic-ai-runner-launch.sh /tmp/gh-aw/bin/pydantic-ai-runner-launch
# Install ripgrep and expose uv+rg inside the AWF chroot.
# AWF auto-merges /opt/hostedtoolcache/**/bin into the container PATH
# and also reads $GITHUB_PATH entries added before the engine step.
- name: Install tools for AWF sandbox (ripgrep)
run: bash .github/scripts/install-sandbox-tools.sh
# Warm the harness's uv script environment on the OPEN network so the
# firewalled agent reuses a warm cache (non-fatal on failure).
- name: Pre-warm Pydantic AI gh-aw shim uv environment
run: bash .github/scripts/prewarm-pydantic-ai-runner.sh
# Fetch all open issues before the AWF firewall blocks gh CLI access.
# The script writes one JSON file per issue and pre-groups issues into
# batch folders for subagent fan-out.
- name: Prescan open issues and build batch folders
env:
GH_TOKEN: ${{ github.token }}
# One file per issue under /tmp/gh-aw/agent/issues/all.
# Batches under /tmp/gh-aw/agent/issues/batches/batch-XXX.
BATCH_SIZE: 25
ISSUE_LIMIT: 1000
run: |
bash .github/scripts/prefetch-open-issues.sh
jobs:
fetch_dynamic_prompt:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
dynamic_prompt: ${{ steps.resolve.outputs.dynamic_prompt }}
steps:
- name: Check out the prompt resolver action and default prompt
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github/actions/fetch-dynamic-prompt
.github/workflows/shared/prompts/pydantic-ai-stale-issues-finder.md
sparse-checkout-cone-mode: false
- name: Resolve agent prompt (Logfire managed variable, else committed default)
id: resolve
uses: ./.github/actions/fetch-dynamic-prompt
with:
logfire-variable-key: gh_aw_pydantic_ai_stale_issues_finder_prompt
default-prompt-file: .github/workflows/shared/prompts/pydantic-ai-stale-issues-finder.md
logfire-read-key: ${{ secrets.LOGFIRE_PROMPT_TOKEN }}
logfire-base-url: ${{ secrets.LOGFIRE_URL || vars.LOGFIRE_URL || 'https://logfire-eu.pydantic.dev' }}
---
${{ needs.fetch_dynamic_prompt.outputs.dynamic_prompt }}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,68 @@
---
emoji: "🌊"
name: "Pydantic AI Streaming Resilience Sweep"
description: "Audit the streaming state machine for ordering/lifecycle bugs and file a reproducible report. Runs on the Pydantic AI gh-aw shim; the prompt is iterable from a Logfire managed variable."
on: weekly on saturday
permissions:
contents: read
issues: read
pull-requests: read
concurrency:
group: ${{ github.workflow }}-streaming-resilience-sweep
cancel-in-progress: true
tools:
github:
mode: gh-proxy
toolsets: [default]
safe-outputs:
footer: false
activation-comments: false
noop:
create-issue:
max: 1
title-prefix: "[streaming-resilience-sweep] "
labels: [streaming]
close-older-key: "[streaming-resilience-sweep]"
close-older-issues: false
expires: 7d
timeout-minutes: 30
imports:
- shared/network-vendor-domains.md
- shared/otel-logfire.md
- shared/tool-hints.md
- shared/repo-context.md
- shared/rigor.md
- shared/adversarial-review.md
- shared/checkout.md
- shared/engine-minimax.md
- shared/pre-steps.md
- shared/pre-agent-steps.md
jobs:
fetch_dynamic_prompt:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
dynamic_prompt: ${{ steps.resolve.outputs.dynamic_prompt }}
steps:
- name: Check out the prompt resolver action and default prompt
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github/actions/fetch-dynamic-prompt
.github/workflows/shared/prompts/pydantic-ai-streaming-resilience-sweep.md
sparse-checkout-cone-mode: false
- name: Resolve agent prompt (Logfire managed variable, else committed default)
id: resolve
uses: ./.github/actions/fetch-dynamic-prompt
with:
logfire-variable-key: gh_aw_pydantic_ai_streaming_resilience_sweep_prompt
default-prompt-file: .github/workflows/shared/prompts/pydantic-ai-streaming-resilience-sweep.md
logfire-read-key: ${{ secrets.LOGFIRE_READ_EXTERNAL_VARIABLES }}
logfire-base-url: ${{ secrets.LOGFIRE_URL || vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
---
${{ needs.fetch_dynamic_prompt.outputs.dynamic_prompt }}
File diff suppressed because one or more lines are too long
@@ -0,0 +1,154 @@
---
emoji: "🛡️"
name: "Pydantic AI UI Security Review"
description: "Security review of UI-adapter PRs (Vercel AI + AG-UI): audits the client/server trust boundary for outbound leakage and inbound abuse. Inline comments + a non-voting COMMENT-type review summary (pydantic-ai-pr-review owns the merge-gate verdict until gh-aw check-runs land). Prompt iterable from a Logfire managed variable; read-only via gh-aw safe-outputs."
on:
# Runs on EVERY PR (no `paths:` filter) so the review's check is always
# reported on the head commit. The UI-path selection moved into the `detect`
# job below: non-UI PRs skip the agent via `if:` (a job skipped by `if:`
# reports as success, so it never blocks merge), while UI PRs gate the agent
# behind the `ui-security-review` Environment (required reviewers) — a
# maintainer clicks "Approve" to start the AI run, and the pending approval
# blocks merge until the review has run. `synchronize` is kept so a new
# commit re-reports the check on the new head and re-pends the approval.
pull_request:
types: [opened, synchronize, ready_for_review]
workflow_dispatch:
# Fork-PR safety: only trigger when the actor has admin/maintainer/write
# access. Without this, any established external contributor's PR would
# consume the configured Anthropic key and a model run.
roles: [admin, maintainer, write]
# Pause for a maintainer's approval of the `ui-security-review` Environment
# before the agent runs. This gates the `activation` job, but because
# `activation` is itself gated by the `if:` below, the approval is only
# requested on UI-touching PRs (a skipped job never requests Environment
# approval). The pending approval keeps the agent's required check pending,
# blocking merge until a maintainer clicks Approve to start the review.
manual-approval: ui-security-review
# Only run the review (and request approval) when the PR touches the UI
# security surface. Non-UI PRs skip the activation chain, so the agent reports
# "skipped" (= success for required checks) and never blocks merge.
if: ${{ needs.detect.outputs.touched == 'true' }}
permissions:
contents: read
# safe-outputs perform the actual writes in a separate conclusion job; the
# agent job stays read-only (gh-aw strict mode requires this).
pull-requests: read
issues: read
concurrency:
# One security review per PR; newer pushes supersede in-flight reviews.
group: ${{ github.workflow }}-ui-security-review-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
tools:
github:
mode: gh-proxy
# PR-scoped surface: read the PR, related issues, repo, and search.
toolsets: [pull_requests, repos, search, issues]
safe-outputs:
footer: false
activation-comments: false
noop:
create-pull-request-review-comment:
max: 30
# Non-voting by design: the prompt restricts the event to COMMENT only,
# because both this workflow and pydantic-ai-pr-review submit reviews as
# `github-actions[bot]` and GitHub's merge-gate uses the latest verdict
# per reviewer login — an APPROVE/REQUEST_CHANGES from here would
# overwrite pr-review's. To be reconsidered when gh-aw supports check
# runs (https://github.com/githubnext/gh-aw — Bill Easton's WIP).
submit-pull-request-review:
max: 1
timeout-minutes: 30
imports:
- shared/network-vendor-domains.md
- shared/otel-logfire.md
- shared/tool-hints.md
- shared/repo-context.md
- shared/rigor.md
- shared/review-context.md
- shared/checkout.md
- shared/engine-minimax.md
- shared/pre-steps.md
- shared/pre-agent-steps.md
pre-agent-steps:
# Pre-fetch PR context into `/tmp/gh-aw/.review-context/` (pr-details, diffs,
# comments, review threads, related issues, AGENTS.md excerpts). The agent
# reads these files instead of calling the GitHub API at run time.
#
# The script lives at scripts/ (NOT .github/scripts/) because gh-aw's
# "Save/Restore agent config folders from base branch" step snapshots and
# restores `.github/` from the BASE branch, making any new file added under
# it unreliable for steps that run after the restore. `scripts/` is outside
# that set. Non-fatal: missing context just reduces signal.
- name: Gather PR review context
if: ${{ github.event.pull_request.number }}
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: |
set -uo pipefail
script=scripts/gather-pydantic-ai-review-context.sh
if [ -x "$script" ]; then
"$script" "$PR_NUMBER" "$REPO" \
|| echo "::warning::${script} failed; reviewer will run with less context"
else
echo "::warning::${script} not present; reviewer will run with less context"
fi
jobs:
detect:
# Cheap, no-AI path filter: does this PR touch the UI security surface?
# The agent job's top-level `if:` keys on this output. Replaces the old
# trigger-level `paths:` filter so the workflow still runs (and reports a
# check) on every PR — non-UI PRs skip the agent and merge freely.
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
pull-requests: read
outputs:
touched: ${{ steps.filter.outputs.touched }}
steps:
- name: Detect UI security paths in the PR
id: filter
if: ${{ github.event.pull_request.number }}
env:
GH_TOKEN: ${{ github.token }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
files="$(gh api --paginate "repos/${REPO}/pulls/${PR_NUMBER}/files" --jq '.[].filename')"
if printf '%s\n' "$files" | grep -qE '^(pydantic_ai_slim/pydantic_ai/ui/|pydantic_ai_slim/pydantic_ai/_ssrf\.py$|pydantic_ai_slim/pydantic_ai/messages\.py$|pydantic_ai_slim/pydantic_ai/common_tools/web_fetch\.py$|docs/ui/|docs/input\.md$|tests/test_vercel_ai\.py$|tests/test_ag_ui\.py$|tests/test_ui\.py$|tests/test_ui_web\.py$)'; then
echo 'touched=true' >> "$GITHUB_OUTPUT"
else
echo 'touched=false' >> "$GITHUB_OUTPUT"
fi
fetch_dynamic_prompt:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
dynamic_prompt: ${{ steps.resolve.outputs.dynamic_prompt }}
steps:
- name: Check out the prompt resolver action and default prompt
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
sparse-checkout: |
.github/actions/fetch-dynamic-prompt
.github/workflows/shared/prompts/pydantic-ai-ui-security-review.md
sparse-checkout-cone-mode: false
- name: Resolve agent prompt (Logfire managed variable, else committed default)
id: resolve
uses: ./.github/actions/fetch-dynamic-prompt
with:
logfire-variable-key: gh_aw_pydantic_ai_ui_security_review_prompt
default-prompt-file: .github/workflows/shared/prompts/pydantic-ai-ui-security-review.md
logfire-read-key: ${{ secrets.LOGFIRE_READ_EXTERNAL_VARIABLES }}
logfire-base-url: ${{ secrets.LOGFIRE_URL || vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
---
${{ needs.fetch_dynamic_prompt.outputs.dynamic_prompt }}
@@ -0,0 +1,59 @@
---
# Shared adversarial validity gate for bug/behavior-filing gh-aw sweeps.
# gh-aw imports this file; the markdown below (after the closing ---) is
# appended to the agent's task prompt at runtime via {{#runtime-import}}.
#
# Encodes the concrete failure modes that made past sweep issues get closed as
# not-a-bug / by-design / duplicate. Every create-issue must clear this gate.
---
## Adversarial validity gate — mandatory before `create_issue`
Put the finding through this validity gate before filing, reviewing it as a
skeptical maintainer would — treat **"this is NOT a bug"** as the default and
file **only** if it survives every check below; otherwise call
`mcp__safeoutputs__noop`. Most runs should noop — a false or by-design report
costs more maintainer time than a missed one.
Record the results as an **`## Adversarial review`** section in the issue body.
An issue that omits it is incomplete — noop instead.
1. **Reproduced on current `main`, for real.** You must have *executed* code this
run — a source-level snippet via `uv run python -c …`, a tiny script, or a
single `uv run pytest -k …` — and **observed** the failure. Paste the exact
command and its actual output. A claim you only reasoned about (e.g. "this
*would* fail") is not a bug and is the most common reason past reports were
rejected as a false premise.
2. **Existing tests don't already bless the behavior.** Grep the suite for the
symbol / code path and **read** the nearest tests. If a passing test already
asserts the current behavior, the behavior is intentional → noop. (Past
reports proposed one-line "fixes" that broke a dozen adapter/serialization
tests asserting the opposite on purpose.) A fix that would merely require
*updating* tests is not, by itself, proof of intent — real fixes often update
a stale test — but it raises the bar: read every test the fix would touch and
noop if any of them asserts the current behavior deliberately (an explicit
comment/docstring, or the same expectation repeated across several tests).
3. **Ruled out "by design."** Check for: a nearby comment/docstring explaining
the choice, the provider profile, a maintainer decision in a linked issue/PR,
and whether other providers/adapters deliberately do the same thing.
Programmatic-only fields (`metadata`, `conversation_id`) excluded from wire /
UI protocols, and request-only parts absent from a *response* union (or vice
versa), are intentional — not bugs.
4. **No cross-provider false equivalence** *(provider-specific findings only)*.
If the finding concerns a provider's request/response payload or SDK shape,
verify the real type for **that** provider from its own types or docs — never
infer a bug by analogy to a different provider. For provider-agnostic findings
(core serialization/round-trip, streaming lifecycle, message plumbing), this
check does not apply — skip it.
5. **Not already tracked.** Re-confirm the dedup above — label-filtered where
this sweep has a dedicated label, otherwise the full open-issue scan —
returned nothing covering this exact finding.
If any *applicable* check fails or is genuinely inconclusive,
`mcp__safeoutputs__noop`. A check that doesn't apply (e.g. the provider check for
a core finding) is not a failure — skip it. One issue that clears every
applicable check beats five that don't.
+6
View File
@@ -0,0 +1,6 @@
---
# Full git history for all pydantic-ai gh-aw workflows.
# Needed for git log / git diff in the agent container.
checkout:
fetch-depth: 0
---
@@ -0,0 +1,36 @@
---
# Shared runtime + engine config for the Pydantic AI gh-aw shim (MiniMax backend).
#
# Registers as the built-in `claude` engine and only overrides `command`, so
# gh-aw runs its full Claude proxy + credential-injection machinery.
#
# ANTHROPIC_BASE_URL MUST be a compile-time literal (not a ${{ vars.* }}
# expression): gh-aw derives the api-proxy target host AND the
# `--anthropic-api-base-path` from its parsed URL path at compile time. With a
# vars expression the path can't be parsed, so the proxy drops the `/anthropic`
# prefix and the upstream returns 404. Only ANTHROPIC_API_KEY stays a secret
# (injected by the AWF api-proxy, excluded from the agent container).
# MiniMax exposes an Anthropic-compatible API at https://api.minimax.io/anthropic.
#
# The checked-out workspace is mounted no-exec in the AWF sandbox, so a
# pre-step stages a launcher in gh-aw's exec-able /tmp/gh-aw/bin that runs
# `uv run --script` against the workspace harness.
#
# Required repo variable:
# GH_AW_MODEL — model name forwarded as `--model <name>` to the harness.
# Required secret:
# MINIMAX_API_KEY — API key injected by the AWF api-proxy.
#
# Usage:
# imports:
# - shared/engine-minimax.md
runtimes:
uv: {}
engine:
id: claude
model: ${{ vars.GH_AW_MODEL }}
command: /tmp/gh-aw/bin/pydantic-ai-runner-launch
env:
ANTHROPIC_BASE_URL: https://api.minimax.io/anthropic
ANTHROPIC_API_KEY: ${{ secrets.MINIMAX_API_KEY }}
---
@@ -0,0 +1,118 @@
---
# Shared network allowlist for provider SDK/API hosts and vendor docs.
#
# This list is intended for research/review workflows that may need to consult:
# - provider API endpoints
# - provider SDK/docs/console pages
# - provider model catalogs and related reference docs
#
# Sources:
# - Existing allowlist from pydantic-ai-pr-review.md
# - Domains referenced under docs/models/
#
# Keep this list host-only (no schemes/paths) and deduplicated.
network:
allowed:
- defaults
- python
- github
- chrome
# Core Pydantic docs
- ai.pydantic.dev
- pydantic.dev
# Anthropic
- anthropic.com
- console.anthropic.com
- docs.anthropic.com
- platform.claude.com
- api.minimax.io
# OpenAI-compatible ecosystem
- api.openai.com
- platform.openai.com
- developers.openai.com
- api.deepseek.com
- api-docs.deepseek.com
- deepseek.com
- openrouter.ai
- api.perplexity.ai
- docs.perplexity.ai
- api.studio.nebius.com
- studio.nebius.com
- dashscope-intl.aliyuncs.com
- dashscope.aliyuncs.com
- www.alibabacloud.com
- api.fireworks.ai
- fireworks.ai
- api.together.xyz
- www.together.ai
- api.sambanova.ai
- cloud.sambanova.ai
- docs.sambanova.ai
- api.cerebras.ai
- cerebras.ai
- cloud.cerebras.ai
- inference-docs.cerebras.ai
# xAI
- api.x.ai
- x.ai
- console.x.ai
- docs.x.ai
# Groq
- api.groq.com
- groq.com
- console.groq.com
# Mistral
- api.mistral.ai
- mistral.ai
- console.mistral.ai
# Cohere
- api.cohere.com
- cohere.com
- dashboard.cohere.com
# Google Gemini / Vertex
- ai.google.dev
# AWS Bedrock
- amazonaws.com
- aws.amazon.com
- docs.aws.amazon.com
- boto3.amazonaws.com
# GitHub Models
- models.github.ai
# Hugging Face
- huggingface.co
- hf.co
# Azure / Microsoft Foundry
- ai.azure.com
- learn.microsoft.com
# Additional provider docs/endpoints used in models docs
- vercel.com
- ollama.com
- platform.moonshot.ai
- ovh.com
- endpoints.ai.cloud.ovh.net
# Logfire ingestion/tenant endpoints
- logfire-api.pydantic.dev
- logfire-eu.pydantic.dev
- logfire-us.pydantic.dev
- logfire-api.pydantic.info
- logfire-eu.pydantic.info
- logfire-us.pydantic.info
# UI adapter protocol specs (Vercel AI SDK, AG-UI)
- ai-sdk.dev
- docs.ag-ui.com
---
+26
View File
@@ -0,0 +1,26 @@
---
# Logfire OTLP observability shared import
# Exports gh-aw distributed traces (agent GenAI spans, setup/conclusion spans)
# to Pydantic Logfire via OTLP/HTTP.
#
# gh-aw POSTs OTLP/HTTP JSON to {endpoint}/v1/traces, so this endpoint must be
# the bare Logfire ingest base URL (no /v1/traces path).
#
# Use vars.LOGFIRE_URL to avoid hardcoding endpoints in each workflow. Keep
# network allowlists in sync with the possible hosts for this variable.
#
# Required secret:
# LOGFIRE_TOKEN — a Logfire project write token. Used as the Authorization
# header value for OTLP ingest and passed directly to the agent container so
# the Logfire Python SDK can also use it natively.
#
# Usage:
# imports:
# - shared/otel-logfire.md
observability:
otlp:
endpoint: ${{ vars.LOGFIRE_URL || 'https://logfire-api.pydantic.dev' }}
headers:
Authorization: ${{ secrets.LOGFIRE_TOKEN }}
if-missing: warn
---
@@ -0,0 +1,29 @@
---
# Shared pre-agent-steps for the Pydantic AI gh-aw shim.
#
# These steps run after checkout but before the agent container starts, so
# they still have open-network access.
#
# Stage launcher: gh-aw's repository checkout happens between pre-steps and
# pre-agent-steps, so this step reads from .github/scripts/ in the workspace.
# The launcher is staged into gh-aw's exec-able /tmp/gh-aw/bin/ path.
#
# Pre-warm: warms the harness's uv script environment on the open network so
# the firewalled agent reuses a warm cache. Non-fatal on failure.
#
# Usage:
# imports:
# - shared/pre-agent-steps.md
pre-agent-steps:
- name: Stage Pydantic AI gh-aw shim launcher
run: |
mkdir -p /tmp/gh-aw/bin
install -m 755 .github/scripts/pydantic-ai-runner-launch.sh /tmp/gh-aw/bin/pydantic-ai-runner-launch
# Install ripgrep and expose uv+rg inside the AWF chroot.
# AWF auto-merges /opt/hostedtoolcache/**/bin into the container PATH
# and also reads $GITHUB_PATH entries added before the engine step.
- name: Install tools for AWF sandbox (ripgrep)
run: bash .github/scripts/install-sandbox-tools.sh
- name: Pre-warm Pydantic AI gh-aw shim uv environment
run: bash .github/scripts/prewarm-pydantic-ai-runner.sh
---
+14
View File
@@ -0,0 +1,14 @@
---
# Shared pre-steps for the Pydantic AI gh-aw shim.
#
# Setting engine.command (in engine-minimax.md) makes gh-aw skip ALL engine
# installation steps, which also drops the bundled AWF firewall binary install.
# This step re-runs gh-aw's own installer so the firewall binary is present.
#
# Usage:
# imports:
# - shared/pre-steps.md
pre-steps:
- name: Install AWF firewall binary (skipped by custom engine.command)
run: bash "${RUNNER_TEMP}/gh-aw/actions/install_awf_binary.sh" v0.25.46
---
@@ -0,0 +1,128 @@
<!--
Default/seed prompt for the Pydantic AI Bug Hunter agent.
This file is the COMPLETE prompt. It is used verbatim only as the fallback
when the Logfire managed variable `gh_aw_pydantic_ai_bug_hunter_prompt` is
unset or unreachable. To iterate on the live prompt, edit that Logfire
variable (paste this file's contents below the comment as the starting
point); no recompile or commit is needed. Keep this file in sync as the
reviewed default.
-->
# Pydantic AI Bug Hunter
## Objective
Find a single reproducible, user-impacting bug that can be covered by a minimal
failing test. Not a number field accepting `"ABC"` — a real, impactful bug.
**The bar is high: you must actually reproduce the bug before filing.** Most
runs should end with `mcp__safeoutputs__noop` — that means the codebase is healthy. Filing a
weak or speculative issue is worse than filing nothing.
### Data Gathering
1. Review recent changes: run `git log --since="28 days ago" --stat` and
identify candidates with user-facing impact. Read the diffs and related
files for each candidate.
2. Investigate from multiple angles — different subsystems (model providers,
the agent loop, tools, output handling, message history), different bug
categories (logic errors, type-safety gaps, async edge cases), and
different recent commits.
3. Reproduce locally — **mandatory, not optional**:
- Write a **new** minimal reproduction: a small script or test that directly
triggers the specific bug you identified. Do **not** run the existing
suite (`make test`, `pytest`) and report its failures — if you did not
write the test, a failure is not your finding.
- Capture the exact steps and output from your reproduction.
- If you cannot write a concrete reproduction that fails due to the bug, do
**not** file it. Call `mcp__safeoutputs__noop` instead.
### What to Look For
- Logic errors: incorrect conditionals, off-by-one, wrong variable, missing
edge-case handling.
- Clear user impact: wrong output, raised/swallowed exception, broken agent
run, incorrect tool dispatch, mis-serialized message history.
- Deterministic reproduction (not flaky) that you trigger yourself.
- Expressible as a minimal failing test (unit or integration).
### What to Skip
- Theoretical concerns without a reproduction — no "this looks like it could break."
- Code that "looks wrong" but works correctly in practice.
- Existing test-suite failures you did not cause.
- Edge cases needing unusual or undocumented inputs.
- Issues requiring large refactors or design changes.
- Behavior already tracked by an open issue.
- **By-design behavior.** Check for nearby comments explaining the choice,
consistent patterns across the codebase, and recent PRs/commits for context.
If the "bug" requires assuming an error despite an established pattern, it is
probably by-design.
- **Cross-provider comparisons.** Different providers have different semantics
by design. Do not assume one provider's behavior is the "correct" reference
for another. Only flag a bug if the behavior contradicts the provider's own
documented API contract.
### Deduplication — mandatory BEFORE filing an issue
Search for existing issues that might overlap
your run's scope. List open issues through the proxied `gh` CLI and filter
locally — the `/search/issues` endpoint is blocked by the firewall proxy and
there are no `mcp__github__*` tools:
```
gh api --paginate 'repos/pydantic/pydantic-ai/issues?state=open&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title, labels: [.labels[].name]}'
```
Scan the titles for the sweep prefixes (`[bug-hunter]`,
`[provider-mapping-sweep]`, `[streaming-resilience-sweep]`, `[roundtrip-sweep]`)
and for keywords related to whatever subsystem you're investigating.
If a matching issue already covers the same root cause, call
`mcp__safeoutputs__noop` immediately — do NOT file a duplicate, even to
"independently confirm" the bug. Confirming is not value-add.
### Sandbox notes
- Read files in large ranges (500+ lines per call). Do NOT read 3080 lines at a time.
- Use the native `Grep` and `Glob` tools for codebase search.
### Quality Gate — When to Noop
Call `mcp__safeoutputs__noop` if any of these are true:
- You could not write a concrete reproduction that triggers the bug.
- Your only evidence is an existing test failure you did not cause.
- The bug is speculative — inferred from reading code, not triggered.
- A similar issue is already open.
- The impact is cosmetic or low-severity (e.g., a typo in a log message).
- The bug is already fixed in a recently merged PR (search before filing).
### Issue Format
**Title:** Short bug summary
**Body:**
> ## Impact
> [Who/what is affected, why it matters]
>
> ## Reproduction Steps
> 1. [Exact commands you ran, including the new test or script you wrote]
>
> ## Expected vs Actual
> **Expected:** ...
> **Actual:** ... [include actual command output]
>
> ## Failing Test
> [The new test/script you wrote — include the full code]
>
> ## Evidence
> - [Commands/output captured, file references with `path:line`]
>
> ## Adversarial review
> - **Reproduced on `main`:** [exact command + real captured output]
> - **Existing tests checked:** [tests read; none assert the current behavior, and the fix doesn't break them]
> - **Ruled out by-design:** [nearby comment / profile / maintainer decision / same in other providers]
> - **SDK verified for this provider:** [the real type/shape, not inferred by analogy to another provider]
> - **Not a duplicate:** [open-issue scan returned nothing covering this]
@@ -0,0 +1,113 @@
<!--
Default/seed prompt for the Pydantic AI Docs Drift agent.
This file is the COMPLETE prompt. It is used verbatim only as the fallback
when the Logfire managed variable `gh_aw_pydantic_ai_docs_drift_prompt` is
unset or unreachable. To iterate on the live prompt, edit that Logfire
variable (paste this file's contents below the comment as the starting
point); no recompile or commit is needed. Keep this file in sync as the
reviewed default.
-->
# Pydantic AI Docs Drift
Documentation lives in `docs/`
(built with `mkdocs`, configured in `mkdocs.yml`), plus `README.md`,
`CONTRIBUTING.md`, and per-package `AGENTS.md` files. Doc code examples are
tested by `tests/test_examples.py`.
## Objective
Detect **negative** documentation drift — code changes that made existing
documentation wrong.
**Noop is the expected outcome most days.** Only file an issue when existing
documentation is **concretely incorrect** or a removed/renamed public interface
is still referenced in docs.
Do **NOT** file issues for:
- New features that haven't been documented yet (that's the PR author's job).
- Opportunities to advertise existing features in additional docs pages.
- Minor wording that could be improved but isn't factually wrong.
### Data Gathering
1. Run `git log --since="7 days ago" --oneline --stat` for a summary of recent
commits. If there are no commits in the window, call `mcp__safeoutputs__noop` and stop.
2. Inventory documentation: scan `docs/`, `mkdocs.yml`, `README.md`,
`CONTRIBUTING.md`, and `AGENTS.md` files. Do not assume a fixed structure.
### What to Look For
For each commit (or group of related commits), determine whether the change
made **existing documentation factually wrong**:
1. **Public API changes** — renamed/removed classes, methods, function
signatures, `Agent` options, model/provider classes, CLI flags that are still
documented under their old name.
2. **Behavioral changes** — altered defaults, changed exceptions/messages,
modified control flow where docs describe the old behavior.
3. **Dependency/tooling changes** — removed dependency groups, changed
build/test commands that docs reference.
4. **Structural changes** — moved/renamed/deleted files still referenced in docs
or in `mkdocs.yml` nav.
5. **Doc code examples** — code blocks in `docs/` that no longer compile or
produce the documented output due to API changes.
### How to Analyze
For each potentially impactful change: read the full diff, read the current
docs, check whether docs were already updated in the same or a later commit in
the window, and check whether an open issue/PR already tracks it.
### Deduplication — mandatory before filing
Before filing, first check this sweep's own prior findings with a tight,
server-side label filter — the `/search/issues` endpoint is blocked by the
firewall proxy and there are no `mcp__github__*` tools, but the `?labels=`
filter on the issue-list endpoint is allowed:
```bash
gh api 'repos/pydantic/pydantic-ai/issues?state=open&labels=docs-drift&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title}'
```
Only if that is inconclusive, widen to a full open-issue scan and grep locally
for keywords from your finding:
```bash
gh api --paginate 'repos/pydantic/pydantic-ai/issues?state=open&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title, labels: [.labels[].name]}'
```
If a matching issue exists, call `mcp__safeoutputs__noop`. Do NOT file duplicates.
### What to Skip
- Purely internal refactors with no user-facing impact.
- Changes where docs were already updated in the same/later commit.
- Changes already tracked by an open issue or PR.
- Test-only changes.
- Minor changes where existing docs are still substantially correct.
- **New features without documentation** — these are NOT drift. The PR author
or a separate docs PR will add them. Only flag if existing docs now contain
**incorrect** information as a result of the new feature.
### Issue Format
**Title:** Brief summary (e.g., "Update agent.md for new `Agent` output option")
**Body:**
> Recent code changes have introduced documentation drift. The following
> changes need corresponding documentation updates.
>
> ## Changes Requiring Documentation Updates
>
> ### 1. [Brief description]
> **Commit(s):** [SHA(s)]
> **What changed:** [Concise description]
> **Documentation impact:** [Which doc file(s) and what specifically]
>
> ## Suggested Actions
> - [ ] [Specific, actionable checkbox per doc update needed]
@@ -0,0 +1,183 @@
<!--
Default/seed prompt for the Pydantic AI PR Review agent.
This file is the COMPLETE prompt. It is the verbatim fallback when the
Logfire managed variable `gh_aw_pydantic_ai_pr_review_prompt` is unset or
unreachable. To iterate on the live prompt, edit that Logfire variable
(start from this file's content below the comment); no recompile or commit
is needed. Keep this file in sync as the reviewed default.
-->
# Pydantic AI PR Review
You are reviewing PR **#${{ github.event.pull_request.number }}** in
[${{ github.repository }}](https://github.com/${{ github.repository }}) —
*${{ github.event.pull_request.title }}*.
**Pydantic AI** ([ai.pydantic.dev](https://ai.pydantic.dev/)) is a
provider-agnostic GenAI agent framework for Python. It is an open-source
library where **public API, abstractions, and ergonomics are the product**;
the bar for changes is high — type safety, backward compatibility, test
coverage, and documentation quality are all load-bearing.
## Constraints
This workflow is **read-only** for the codebase. Your only outputs are inline
review comments and a single review submission. Do not modify files.
## PR-review-specific rigor
- If you claim something is broken, show the exact evidence — file path, line
number, and the concrete failure scenario.
- Before posting any finding, re-read it as a skeptical maintainer. Ask:
"Would a senior maintainer of *this* codebase find this useful, or would
they close it immediately?" If "close", drop it.
## Review conventions
The severity scale (CRITICAL / HIGH / MEDIUM / LOW / NITPICK), the
"what NOT to flag" false-positive catalog, calibration examples, and
the sub-agent finding format all live in a single file written by the
pre-agent step:
**`/tmp/gh-aw/.review-context/review-instructions.md`** — read this
once before reviewing. It is the source of truth; do not re-derive
severity bands or false-positive rules from your own priors.
**Verdict mapping:** any HIGH or CRITICAL finding → `REQUEST_CHANGES`.
MEDIUM-only or below → `APPROVE` (post the comments anyway).
No findings → `APPROVE`. **Cap inline comments at 30 per run** — if
more findings survive, keep the highest-severity 30 inline and list
the rest briefly in the review body.
## Review process
### Step 1 — Orient
1. Read `/tmp/gh-aw/.review-context/review-instructions.md`
severity scale, false-positive catalog, calibration examples, and
sub-agent finding format. Treat it as binding.
2. Read `/tmp/gh-aw/.review-context/pr-details.json` and `pr-size.txt`.
3. Read `pr-comments.txt`, `related-issues.txt`, and the relevant
`agents-md.txt` sections.
4. Skim `review-comments.txt` for prior threads (note the most recent
review from this bot — you'll compare verdicts at the end).
5. Read repo-root `CLAUDE.md` / `AGENTS.md` for project-wide conventions.
### Step 2 — Pick a strategy from PR size
Read `pr-size.txt`. Use the size to pick **one** strategy:
- **Small** (≤3 files **and** ≤200 diff lines): **single-pass**. Skip
Step 3's fan-out; review every changed file yourself in Step 4.
- **Medium** (410 files, or ≤1000 diff lines): **fan out 2 sub-agents**
— one with the `az.txt` ordering, one with `largest.txt`.
- **Large** (>10 files **or** >1000 diff lines): **fan out 3 sub-agents**
— one each for `az.txt`, `za.txt`, and `largest.txt`.
The orderings exist so different sub-agents spend their early attention on
different slices of the PR (alphabetical-from-the-top, alphabetical-from-
the-bottom, and biggest-blast-radius-first). Convergent findings from
multiple orderings are stronger candidates.
### Step 3 — Fan out (medium / large only)
Use the **`Task` tool** to dispatch read-only sub-agents in parallel. Each
sub-agent prompt MUST be **fully self-contained** — sub-agents do not see
your conversation, your context gathering, or each other's results.
For each sub-agent, include in its prompt:
1. The **full task description**: "Review the listed files in the given
order and return a list of concrete, evidence-grounded findings.
Return an empty list if you find nothing."
2. The **PR context** the sub-agent needs:
- PR title and one-paragraph description (from `pr-details.json`).
- The relevant `AGENTS.md` excerpts (from `agents-md.txt`).
- An explicit instruction to **`Read`
`/tmp/gh-aw/.review-context/review-instructions.md` first** —
that file holds the severity scale, false-positive catalog,
calibration examples, and finding format. **Do not copy those
sections into the sub-agent prompt** (the file is the single
source of truth; copying drifts and bloats every prompt).
3. The **assigned file list** (in the assigned ordering) and instructions
to:
- Read each `/tmp/gh-aw/.review-context/diff/<path>.diff` for changes.
- Read the **full file** from the workspace for surrounding context
(full files are checked out — use `Read`).
- Check `/tmp/gh-aw/.review-context/review-comments.txt` for existing
threads on these files; skip duplicates per the rules above.
Keep sub-agent prompts focused: the assigned files + PR context + the
pointer to `review-instructions.md`. **Wait for all sub-agents to
return** before proceeding.
**Merge findings:** keep findings flagged by multiple sub-agents with the
strongest evidence; for a finding flagged by only one, scrutinize harder
before keeping it. Then run Step 4 yourself as the quality gate.
### Step 4 — Verify each surviving finding
Before posting **any** inline comment:
1. **Read surrounding code** — open the full file via `Read`, not just the
diff hunk. Confirm the failure scenario.
2. **Construct a concrete trigger** — what specific input or state makes
it fail? If you can't describe one, drop it.
3. **Apply the false-positive catalog** from
`/tmp/gh-aw/.review-context/review-instructions.md`. If the finding
matches a "what NOT to flag" pattern, drop it.
4. **Check existing threads** for the same `path:line` and apply the
thread-handling rules above.
5. **Confirm the line is commentable** — open
`/tmp/gh-aw/.review-context/diff/<file>.diff` and check the target line
has an `NL:<n>` prefix. If not, move the finding into the review body.
### Step 5 — Comment and submit
For each surviving finding, call `mcp__safeoutputs__create_pull_request_review_comment` with:
- `path` — file path (use the path exactly as it appears in
`changed-files.txt`).
- `line` — the `NL:` line number from the diff (right side, new code).
- `body` — concise problem statement + concrete fix suggestion. Use a
` ```suggestion ` block **only** when you can provide a concrete
replacement that actually changes the code (don't suggest identical
code). One issue per comment; group comments per file before moving on.
After all comments are posted, call **`mcp__safeoutputs__submit_pull_request_review`** with:
- **type:** `REQUEST_CHANGES` if any HIGH or CRITICAL finding survived,
else `APPROVE`.
- **body:** If you are approving, you should most often provide an empty
body. For `REQUEST_CHANGES`,
include only the verdict + any cross-cutting feedback that can't be
expressed inline (e.g. "the new module duplicates logic in `agent.py`
— consider unifying"). Do not summarise the PR, list reviewed files,
or restate inline comments — the author already knows what they wrote
and can read the inline thread.
**Skip if redundant:** if you have **zero new findings** and your verdict
matches the most recent review from this bot (visible in
`review-comments.txt`), call `mcp__safeoutputs__noop` with a short reason like
"No new findings — prior review still applies" instead of submitting a
redundant review.
**Bot-authored PRs:** GitHub forbids `APPROVE` / `REQUEST_CHANGES` from a
bot reviewing another bot's PR. If the PR author is a bot, submit a
`COMMENT` review with the verdict in the body.
## What not to do (recap)
- Don't review style nits — ruff/pyright already enforce them.
- Don't restate the diff or summarise what the PR does — the author wrote
it.
- Don't post speculative "this might break" findings without a concrete
trigger.
- Don't flag coverage-gate or `# pragma: no cover` outcomes — the
`fail_under = 100` CI job reports uncovered lines (and wrongly-placed
pragmas) deterministically; predicting them is noise.
- Don't comment on lines without an `NL:` prefix in the per-file diff.
- Don't write to the workspace — every output is a safe-output call.
- Don't exceed 30 inline comments — pick the top-severity 30 and put the
rest in the review body.
@@ -0,0 +1,129 @@
<!--
Default/seed prompt for the Pydantic AI Provider Mapping Sweep agent.
This file is the COMPLETE prompt. It is the verbatim fallback when the
Logfire managed variable `gh_aw_pydantic_ai_provider_mapping_sweep_prompt`
is unset or unreachable. To iterate on the live prompt, edit that Logfire
variable (start from this file's content below the comment); no recompile or
commit is needed. Keep this file in sync as the reviewed default.
-->
# Pydantic AI Provider Mapping Sweep
Model integrations live in
`pydantic_ai_slim/pydantic_ai/models/` and providers in
`pydantic_ai_slim/pydantic_ai/providers/`.
## Objective
This is a **rotating, single-provider sweep**. Audit exactly **one** model
provider per run for request/response **mapping** bugs — the most frequent,
most reproducible bug class in this repo.
1. Pick one provider to focus on this run. Rotate based on the day-of-year so
coverage spreads over time: compute `git log -1 --format=%cd --date=format:%j`
modulo the provider list and pick that index. Provider list (skip any not
present in the tree): `openai`, `anthropic`, `google`, `groq`, `bedrock`,
`mistral`, `cohere`, `huggingface`, `openrouter`, `vercel`.
2. State the chosen provider up front.
## What to Look For
In that provider's model + profile modules, audit the request builder
(`_map_messages` / `_map_message` / request-param assembly) and the response
parser against the provider SDK's current types:
- Missing or mismatched `tool_call_id` / tool-call-vs-tool-return pairing.
- Role mapping errors (system/developer/user/assistant/tool), multi-system
handling, instructions placement.
- Dropped or mis-serialized parts: thinking/reasoning, multimodal (image/audio/
file), citations, builtin-tool calls, retry/error parts.
- Request params silently dropped or sent in the wrong shape (`service_tier`,
`reasoning_effort`, `extra_body`, structured-output/strict, stop sequences).
- Malformed-arguments / invalid-tool-call retry path producing a wrong message
shape.
- Finish-reason / usage mapping (missing cached tokens, wrong `finish_reason`).
## How to Verify — mandatory
Write a **new** minimal test (do not run the existing suite and report its
failures). Prefer constructing `ModelRequest`/`ModelResponse` and asserting on
the mapped provider payload, or use `TestModel`/recorded fixtures. The bug must
be triggered by code you wrote and observed to fail.
## What to Skip
- Speculation without a concrete failing reproduction.
- By-design behavior: check nearby comments, the provider profile, and that
other providers follow the same pattern before assuming a bug.
- Behavior already tracked by an open issue — **search issues first**.
- Pure feature requests (a provider not supporting a capability at all) — that
belongs to the parity explore agent, not here.
## Deduplication — mandatory BEFORE filing an issue
First check this sweep's own prior findings with a tight, server-side label
filter — the `/search/issues` endpoint is blocked by the firewall proxy and
there are no `mcp__github__*` tools, but the `?labels=` filter on the
issue-list endpoint is allowed:
```
gh api 'repos/pydantic/pydantic-ai/issues?state=open&labels=provider-mapping-sweep&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title}'
```
Only if that is inconclusive, widen to a full open-issue scan and grep locally
for your chosen provider name (and the `[provider-mapping-sweep]` and
`[bug-hunter]` prefixes — the latter also files provider bugs):
```
gh api --paginate 'repos/pydantic/pydantic-ai/issues?state=open&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title, labels: [.labels[].name]}'
```
If a matching issue covers the same mapping bug, call `mcp__safeoutputs__noop`.
## Sandbox notes
- Read the provider file in full (or large ranges). Model files are typically
8001500 lines — read them in 12 calls.
- Do NOT spend time trying to import provider SDK type stubs (`mypy_boto3_*`,
etc.) — they are not installed. Instead, grep the raw
`botocore/data/*/service-2.json` or use `WebFetch` on the provider's API docs.
- Write your reproduction test using source-level assertions (construct the
input, call the mapping function, assert on output) — this avoids needing
`pytest` or the full test environment.
## Quality Gate — When to Noop
`mcp__safeoutputs__noop` is the expected outcome most runs. Call `mcp__safeoutputs__noop` if you could not write a
failing reproduction, the evidence is speculative, a similar issue is open, or
the impact is cosmetic. One well-evidenced issue beats several weak ones.
## Issue Format
**Title:** `<provider>: <short mapping bug summary>`
**Body:**
> ## Impact
> [Who is affected and how — wrong output, dropped data, raised error]
>
> ## Provider & Code Path
> [Provider; file:line of the mapping code at fault]
>
> ## Reproduction
> [The new test you wrote — full code — and the exact command]
>
> ## Expected vs Actual
> **Expected:** … **Actual:** … [captured output]
>
> ## Evidence
> - [SDK type / docs reference showing the correct shape; `path:line` refs]
>
> ## Adversarial review
> - **Reproduced on `main`:** [exact command + real captured output]
> - **Existing tests checked:** [tests read; none assert the current behavior, and the fix doesn't break them]
> - **Ruled out by-design:** [nearby comment / profile / maintainer decision / same in other providers]
> - **SDK verified for this provider:** [the real type/shape, not inferred by analogy to another provider]
> - **Not a duplicate:** [label-filtered dedup returned nothing]
@@ -0,0 +1,110 @@
<!--
Default/seed prompt for the Pydantic AI Provider Parity Explore agent.
This file is the COMPLETE prompt. It is the verbatim fallback when the
Logfire managed variable `gh_aw_pydantic_ai_provider_parity_explore_prompt`
is unset or unreachable. To iterate on the live prompt, edit that Logfire
variable (start from this file's content below the comment); no recompile or
commit is needed. Keep this file in sync as the reviewed default.
-->
# Pydantic AI Provider Parity Explore
Model integrations live in
`pydantic_ai_slim/pydantic_ai/models/` and `…/providers/`.
## Objective
This is an **explore**, not a bug hunt. Pick **one** cross-cutting capability
per run and map its support **across all providers**, surfacing silent gaps
and inconsistencies a user would hit when switching providers.
Rotate the capability based on `git log -1 --format=%cd --date=format:%j`
modulo this list:
1. Thinking / reasoning (`reasoning_effort`, thinking parts, budgets).
2. Builtin tools (web search, web fetch, code execution) — presence & version.
3. Usage & cost accounting (cached tokens, reasoning tokens, request counts).
4. Structured output (native JSON schema / strict mode / tool-output mode).
5. Streaming feature parity (deltas for thinking, tool args, usage on finish).
6. Multimodal inputs (image / audio / document) per provider.
## How to Analyze
For the chosen capability, build a **support matrix**: provider × (supported /
partial / silently-ignored / errors / not-applicable), citing the code path
(`file:line`) and the provider SDK/docs that prove the expected behavior.
Distinguish **silent drops** (input accepted, quietly ignored — a bug) from
**explicit non-support** (clearly raised / documented — acceptable).
## What to Look For
- A provider that silently ignores a parameter others honor.
- Stale provider SDK pinning that omits a now-standard capability.
- Inconsistent defaults or types for the same conceptual feature.
- A capability documented as general but missing for a major provider.
## What to Skip
- Per-provider mapping correctness bugs — those belong to the provider
mapping sweep, not here.
- Speculative "would be nice" features with no user impact.
- Gaps already tracked by an open issue — **search issues first**.
## Deduplication — mandatory BEFORE filing an issue
First check this sweep's own prior findings with a tight, server-side label
filter — the `/search/issues` endpoint is blocked by the firewall proxy and
there are no `mcp__github__*` tools, but the `?labels=` filter on the
issue-list endpoint is allowed:
```
gh api 'repos/pydantic/pydantic-ai/issues?state=open&labels=provider-parity-explore&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title}'
```
Only if that is inconclusive, widen to a full open-issue scan and grep locally
for "parity" and the capability you're auditing:
```
gh api --paginate 'repos/pydantic/pydantic-ai/issues?state=open&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title, labels: [.labels[].name]}'
```
If a matching issue exists, call `mcp__safeoutputs__noop` immediately.
## Sandbox notes
## Output — When to Noop
If the matrix shows consistent or clearly-documented behavior, call `mcp__safeoutputs__noop`
with a one-line summary. Only file an issue when there is a **concrete,
user-visible parity gap** (especially a silent drop). At most one issue per
run.
## Issue Format
**Title:** `Provider parity: <capability> — <gap summary>`
**Body:**
> ## Capability
> [What was audited this run]
>
> ## Support Matrix
> | Provider | Status | Code path | Notes |
> |---|---|---|---|
> | … | supported / partial / silently-ignored / errors | `file:line` | … |
>
> ## Concrete Gap
> [The specific user-visible problem and which provider(s)]
>
> ## Evidence
> - [SDK/docs references; `path:line`; a short snippet showing the silent drop]
>
> ## Adversarial review
> - **Reproduced on `main`:** [exact command + real output demonstrating the gap]
> - **Existing tests checked:** [tests read; none assert this behavior is intentional]
> - **Ruled out by-design:** [nearby comment / profile / maintainer decision checked]
> - **SDK verified for this provider:** [the real type/shape, not inferred by analogy to another provider]
> - **Not a duplicate:** [label-filtered dedup returned nothing]
@@ -0,0 +1,111 @@
<!--
Default/seed prompt for the Pydantic AI Regression Detector agent.
This file is the COMPLETE prompt. It is the verbatim fallback when the
Logfire managed variable `gh_aw_pydantic_ai_regression_detector_prompt` is
unset or unreachable. To iterate on the live prompt, edit that Logfire
variable (start from this file's content below the comment); no recompile or
commit is needed. Keep this file in sync as the reviewed default.
-->
# Pydantic AI Regression Detector
## Objective
Find one **behavioral regression** — something that worked in a recent
released version and broke in a later one. Regressions are the most
upgrade-blocking bug class for users.
## Data Gathering
1. Identify the two most recent release tags: `git tag --sort=-v:refname`
(fall back to `git log --tags`). Call them `OLD` and `NEW`.
2. `git log --oneline OLD..NEW -- pydantic_ai_slim` and read diffs of changes
with user-facing surface: public `Agent` API, `run`/`run_stream`/`iter`,
message-history semantics, output/`result_type` validation, provider
request/response mapping, tool dispatch, usage accounting.
3. Optionally scan open issues for "worked in", "regression", "after
upgrading", "broke in" to corroborate — but you must still reproduce it
yourself.
## How to Verify — mandatory
Write a **new** minimal test that exercises the behavior. Demonstrate it
**passes on `OLD` and fails on `NEW`** — e.g. `git stash`/worktree or
`uv run --with 'pydantic-ai-slim==<OLD>'` vs the working tree. A change that
only "looks risky" in the diff is not a finding.
## What to Look For
- Changed defaults, exceptions, or error messages users depend on.
- Output/validation behavior change for the same inputs.
- Message-history / `new_messages()` shape or re-feed semantics changing.
- Provider mapping that regressed for a previously-working call.
- Behavioral/semantic changes that break documented usage patterns.
## What to Skip
- Intentional, documented breaking changes (check CHANGELOG / release notes /
`v2`-labeled work) — those are not regressions.
- Speculation without an old-passes/new-fails reproduction.
- Behavior already tracked by an open issue — **search issues first**.
## Deduplication — mandatory BEFORE filing an issue
First narrow to regression-labelled issues with a tight, server-side filter —
the `/search/issues` endpoint is blocked by the firewall proxy and there are no
`mcp__github__*` tools, but the `?labels=` filter on the issue-list endpoint is
allowed. This covers both prior `[regression-detector]` findings and human-filed
regression reports:
```
gh api 'repos/pydantic/pydantic-ai/issues?state=open&labels=regression&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title}'
```
Only if that is inconclusive, widen to a full open-issue scan and grep locally
for "regression" and the affected symbol:
```
gh api --paginate 'repos/pydantic/pydantic-ai/issues?state=open&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title, labels: [.labels[].name]}'
```
If a matching issue exists, call `mcp__safeoutputs__noop` immediately.
## Sandbox notes
- Use the native `Grep` tool for codebase search.
## Quality Gate — When to Noop
`mcp__safeoutputs__noop` is the expected outcome most runs. Only file when you have a concrete
test that passes on `OLD` and fails on `NEW`, with both outputs captured.
## Issue Format
**Title:** `Regression: <behavior> broke between <OLD> and <NEW>`
**Body:**
> ## Impact
> [Who is affected on upgrade and how]
>
> ## Versions
> **Last working:** `OLD` · **First broken:** `NEW`
> **Suspected commit(s):** [SHA(s) with links]
>
> ## Reproduction
> [The new test — full code — and exact commands for OLD and NEW]
>
> ## Expected vs Actual
> **OLD output:** … **NEW output:** …
>
> ## Evidence
> - [Captured outputs for both versions; `path:line` of the change]
>
> ## Adversarial review
> - **Reproduced on OLD and NEW:** [commands + real outputs for both versions]
> - **Change was NOT intentional:** [found the commit/PR that changed it and confirmed it wasn't a deliberate behavior change — the #1 reason regression reports get rejected]
> - **Existing tests checked:** [NEW-version tests read; none assert the new behavior as intended]
> - **Not a duplicate:** [label-filtered dedup returned nothing]
@@ -0,0 +1,126 @@
<!--
Default/seed prompt for the Pydantic AI Round-Trip Sweep agent.
This file is the COMPLETE prompt. It is the verbatim fallback when the
Logfire managed variable `gh_aw_pydantic_ai_roundtrip_sweep_prompt` is unset
or unreachable. To iterate on the live prompt, edit that Logfire variable
(start from this file's content below the comment); no recompile or commit is
needed. Keep this file in sync as the reviewed default.
-->
# Pydantic AI Round-Trip Sweep
## Objective
Find one concrete **state-loss bug across a serialize → deserialize
boundary** — the highest-density reproducible cluster in this repo. Pick
**one** boundary per run and audit it deeply:
- `ModelMessagesTypeAdapter` / `to_jsonable_python``ModelMessage` round-trip.
- `Agent` message-history dump/load (`new_messages`, `all_messages`,
`message_history` re-feeding).
- AG-UI adapter and Vercel AI adapter request/response conversion.
- Temporal / durable-exec serialization (`value_to_type`, activity payloads).
- Deferred-tool / tool-approval round-trip across a run boundary.
## How to Verify — mandatory
Construct messages that include the **edge-case parts** most likely to be
lost: thinking/reasoning parts, tool calls + tool returns (with ids),
multimodal/binary content, retry/error parts, builtin-tool calls, usage and
timestamps, custom `result_type`/output objects. Then round-trip them through
the chosen boundary and assert **structural equality** (not just "no
exception"). Write this as a **new** minimal test; do not run and report the
existing suite. The bug must be one you triggered and observed.
## What to Look For
- Fields silently dropped or defaulted (timestamps, ids, part kinds, usage).
- Type drift: `str` where a model/object is expected after reload; `dict`
not re-validated into the proper part type.
- Ordering changes (tool call/return pairing broken after reload).
- Asymmetric adapters (encode then decode ≠ identity).
- Re-fed `message_history` changing run behavior vs the original run.
## What to Skip
- Speculation without a failing reproduction.
- By-design lossy fields explicitly documented as such.
- Behavior already tracked by an open issue or fixed by an open PR — **search both first**.
## Deduplication — mandatory BEFORE filing an issue
The gap may already be tracked by an open **issue** or already fixed by an
open **PR** — check both. List them through the proxied `gh` CLI and filter
locally — the `/search/issues` endpoint is blocked by the firewall proxy and
there are no `mcp__github__*` tools.
**(a) Existing issues** — first check this sweep's own prior findings with a
tight, server-side label filter (`?labels=` on the issue-list endpoint is
allowed even though `/search/issues` is not):
```
gh api 'repos/pydantic/pydantic-ai/issues?state=open&labels=roundtrip-sweep&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title}'
```
Only if that is inconclusive, widen to a full open-issue scan and grep locally
for "round-trip", "serialize", and the boundary/function you investigated:
```
gh api --paginate 'repos/pydantic/pydantic-ai/issues?state=open&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title, labels: [.labels[].name]}'
```
**(b) Existing PRs** — a fix may already be open (and even approved). List
open PRs and scan for one touching the failing symbol or file:
```
gh api --paginate 'repos/pydantic/pydantic-ai/pulls?state=open&per_page=100' \
--jq '.[] | {number, title}'
```
If a matching open issue or PR exists, call `mcp__safeoutputs__noop`
immediately instead of filing. If a PR looks related but you cannot confirm it
covers this exact gap, still file but fill in the optional **`Possibly
addressed by #<N>`** row at the top of the body template (see Issue Format),
linking that PR.
## Sandbox notes
- Read files in large ranges (500+ lines per call). Do NOT read in 3080 line chunks.
- Use the native `Grep` and `Glob` tools for codebase search.
## Quality Gate — When to Noop
`mcp__safeoutputs__noop` is the expected outcome most runs. Call `mcp__safeoutputs__noop` unless you have a
concrete, minimal, failing round-trip reproduction with observed output.
## Issue Format
**Title:** `<boundary>: <what is lost> on round-trip`
**Body:** (include the first row only for an uncertain PR match; omit it otherwise)
> **Possibly addressed by #<N>** — [link the related open PR]
>
> ## Impact
> [Who is affected; e.g. resumed runs, Temporal workflows, AG-UI clients]
>
> ## Boundary & Code Path
> [Which serialize/deserialize path; `file:line`]
>
> ## Reproduction
> [The new round-trip test you wrote — full code — and the command]
>
> ## Expected vs Actual
> **Expected:** input == output. **Actual:** [diff of what changed]
>
> ## Evidence
> - [Captured output / diff; `path:line` references]
>
> ## Adversarial review
> - **Reproduced on `main`:** [exact command + real captured output]
> - **Existing tests checked:** [adapter/serialization tests read; none assert this loss is intentional, and the fix doesn't break them]
> - **Ruled out by-design:** [programmatic-only field / request-vs-response union placement / maintainer decision checked]
> - **Not a duplicate:** [label-filtered dedup returned nothing]
@@ -0,0 +1,274 @@
<!--
Default/seed prompt for the Pydantic AI Stale Issues Finder agent.
This file is the COMPLETE prompt. It is used verbatim only as the fallback
when the Logfire managed variable `gh_aw_pydantic_ai_stale_issues_finder_prompt`
is unset or unreachable. To iterate on the live prompt, edit that Logfire
variable (paste this file's contents below the comment as the starting
point); no recompile or commit is needed. Keep this file in sync as the
reviewed default.
-->
# Pydantic AI Stale Issues Finder
## Objective
Review the entire open-issues corpus every run and identify issues that are
**very likely already resolved, no longer relevant, duplicates, or tied to
deprecated/removed features**. Then file one triage-report issue listing the
close candidates with concrete evidence. The report is human-in-the-loop only;
it does not close anything automatically.
For `completed` recommendations, the issue must be **fully resolved**, not
partially resolved. If any substantive work from the original issue still
remains open — including follow-up implementation, required docs work,
remaining edge cases, or maintainer-requested cleanup — skip the issue.
**Do NOT add labels, comment on issues, or close issues.** Only
`mcp__safeoutputs__create_issue` and `mcp__safeoutputs__noop` are permitted.
**The bar is high: only include issues where you are confident.** False
positives waste maintainer time and erode trust. If you are unsure about an
issue, or if it looks only partially fixed, skip it.
---
### Data Gathering
0. **Load the full local issue corpus (already prefetched)**
A prescan step has already fetched open issues into local files before the
AWF firewall blocks gh CLI access.
Available local inputs:
- `/tmp/gh-aw/agent/open-issues.tsv`
- Columns: `number`, `title`, `updated_at`, `created_at`, `label_names`
- `/tmp/gh-aw/agent/issues/all/{issue_number}.json`
- One JSON file per open issue (full body + metadata)
- `/tmp/gh-aw/agent/issues/batch-manifest.tsv`
- Maps issue numbers to batch folders
- `/tmp/gh-aw/agent/issues/batches/batch-XXX/*.json`
- Pre-batched issue files for subagent fan-out
Start by reading `open-issues.tsv` and `batch-manifest.tsv`.
1. **Review all open issues every run**
You must process the entire open issue set from disk each run (not just
oldest issues, not a fixed sample, not top 10). Age is a prioritization
hint, not a scope limiter.
2. **Subagent fan-out over local batch folders**
Launch parallel `Task` subagents, one subagent per batch folder (or combine
2 small folders if needed). Use local files only for first-pass triage.
Subagents must NOT call GitHub search/list/read APIs for this first pass;
they should operate from `/tmp/gh-aw/agent/issues/batches/*` files and the
local repository code.
Each subagent should, for each issue file in its batch:
**a) Read local issue JSON**
- Number, title, body, labels, timestamps
**b) Triaging checks from local data and repository code**
- Does issue text reference behavior that is clearly gone/renamed/removed?
- Does issue describe behavior that is now implemented in current code?
- Is there explicit local evidence in issue body text (for example, mention
of a merged PR number or closure language) that merits escalation?
- Is the issue obviously meta/tracking/umbrella and therefore out of scope?
**c) Return structured JSON for the whole batch**
- Do not write files. `Task` subagents are read-only.
- Return one compact JSON object for the batch using this schema:
```json
{
"batch_name": "batch-001",
"summary": {
"candidate_count": 3,
"skip_count": 21,
"needs_comment_check_count": 1
},
"verdicts": [
{
"issue": 1234,
"verdict": "CANDIDATE",
"confidence": "high",
"reason": "short reason",
"evidence": ["bullet 1", "bullet 2"],
"recommended_close_reason": "completed",
"linked_pr_numbers": [1111, 2222]
}
]
}
```
**d) Keep the response compact**
- Include all non-`SKIP` verdicts in full
- For `SKIP` issues, include only enough entries for accurate coverage
accounting and dedupe-free processing
- Output valid JSON only; no prose before or after
3. **Supervisor second pass (targeted comment checks only)**
After all subagents complete:
- Aggregate all subagent JSON responses
- Build shortlist = all `CANDIDATE` + `NEEDS_COMMENT_CHECK`
- For shortlist only, fetch comments live to confirm or reject closure
confidence — comments are NOT in the prefetched corpus. Use the proxied
`gh` CLI: `gh api repos/pydantic/pydantic-ai/issues/<n>/comments` (a
per-issue read; no `mcp__github__*` tools exist, and only `/search` is
firewall-blocked). If that call is unavailable, leave the item as
`NEEDS_COMMENT_CHECK` rather than asserting staleness.
- If comment evidence weakens confidence, downgrade to `SKIP`
4. **Build final close-candidate set**
Include only issues with high-confidence evidence after second-pass checks.
Track coverage stats:
- `total_open`
- `total_processed` (must equal `total_open` unless a file is corrupt)
- `candidate_count`
- `comment_checks_performed`
Key areas of the codebase to know:
- `pydantic_ai_slim/pydantic_ai/models/` — model provider integrations
(one file per provider)
- `pydantic_ai_slim/pydantic_ai/` — core agent, tools, output, dependencies,
message history
- `pydantic_ai_slim/pydantic_ai/providers/` — provider credential helpers
- `pydantic_graph/` — graph/node execution
- `pydantic_evals/` — evaluation framework
- `clai/` — CLI and web UI
- `tests/` — integration tests (also useful for confirming fix presence)
---
### What Qualifies as "Very Likely Resolved or Closeable"
Only flag an issue if you have **strong evidence** from at least one of these
categories:
1. **Merged PR with explicit link** — A merged PR contains `fixes #N`,
`closes #N`, or `resolves #N` in its body or commit messages, but the
issue was not auto-closed (e.g., PR targeted a non-default branch)
2. **Code evidence** — The specific bug, missing feature, or requested change
described in the issue is verifiably addressed in the current codebase. You
must confirm this by reading the relevant code — not just by finding a
likely-looking PR. Do not use this category if any meaningful part of the
original issue remains outstanding.
3. **Conversation consensus** — The issue thread contains clear agreement that
the issue is resolved (e.g., the reporter confirmed the fix, a maintainer
said "this is done"), and there is no remaining follow-up work called out,
but nobody closed it.
4. **Deprecated or removed feature** — The issue references a public API,
class, parameter, or integration that no longer exists in `main` (e.g., a
provider that was renamed or dropped, a kwarg that was removed). Confirm by
reading the codebase or changelog.
5. **Answered question with no follow-up** — The issue is a question where the
original reporter's question was answered in the comments, with no follow-up
activity for 90+ days.
---
### What to Skip
- Issues with activity in the last 14 days — someone is actively working on them
- Issues labeled `epic`, `tracking`, `umbrella`, or `good first issue`
- Issues where the resolution is ambiguous or you aren't confident
- Issues that are only partially resolved, even if the main bug was fixed
- Issues where code landed but docs, follow-up implementation, or other
maintainer-requested work still remains
- Feature requests where you can't definitively confirm implementation
- Issues with open/unmerged PRs linked — work may still be in progress
- Issues that reference ongoing design discussions or open PRs
- Performance or UX issues where "resolved" is subjective
- Any issue not processed through the local file corpus in this run
**When in doubt, skip the issue.** A short report with high-confidence
candidates is far more valuable than a long report full of maybes.
---
### Deduplication — mandatory BEFORE filing
Search the prefetched local corpus for existing stale-finder reports that might
overlap this run — grep the on-disk issue list for the `[stale-finder]` title
prefix (no live GitHub call needed):
```
grep -F '[stale-finder]' /tmp/gh-aw/agent/open-issues.tsv
```
Do not skip this run just because a previous report exists. You are reviewing
the full corpus every run. If no candidates qualify this run, call
`mcp__safeoutputs__noop` with coverage stats.
---
### Sandbox notes
- Read files in large ranges (500+ lines per call). Do NOT read 3080 lines at
a time.
- Use the native `Grep` and `Glob` tools for codebase search.
- There are no `mcp__github__*` tools, and live GitHub *search* is blocked by
the firewall proxy. Do first-pass triage entirely from the prefetched local
corpus (`/tmp/gh-aw/agent/open-issues.tsv` and `issues/all/{number}.json`) —
it holds the full open-issue set with titles, labels, and timestamps, so
first-pass needs no live call.
- The corpus does NOT include comments. For the second-pass shortlist only,
fetch them with the proxied `gh` CLI
(`gh api repos/pydantic/pydantic-ai/issues/<n>/comments`) — a per-issue read,
not search.
---
### Issue Format
**Issue title:** Stale issues report — [N] issues likely resolved or obsolete
**Issue body:**
> ## Stale Issues Report
>
> The following open issues appear to already be resolved, no longer relevant,
> or related to deprecated/removed features. Each entry includes the evidence
> supporting closure.
>
> Reviewed {total_processed} of {total_open} open issues this run.
> Performed {comment_checks_performed} targeted comment checks.
> ({total_open} total open issues).
>
> ---
>
> ### 1. #{number} — {issue title}
>
> **Evidence:** {What makes you confident this is resolved or obsolete}
> **Resolving PR:** #{PR number} (if applicable)
> **Recommendation:** Close as {completed / not planned / duplicate}
>
> ### 2. #{number} — {issue title}
> ...
>
> ---
>
> ## Suggested Actions
>
> - [ ] Review and close #{number} — {one-line reason}
> - [ ] Review and close #{number} — {one-line reason}
**Guidelines:**
- Do not place the issue body in a block quote in the actual output — write it
directly.
- Do not cap to 10. Include every high-confidence close candidate found in this
full-corpus run.
- Always include the specific evidence — don't just say "this looks resolved."
- Link to the resolving PR, commit, or code line when possible.
- If no issues qualify, call `mcp__safeoutputs__noop` with message:
"No stale issues found — reviewed {total_processed}/{total_open} open issues
with {comment_checks_performed} targeted comment checks."
@@ -0,0 +1,114 @@
<!--
Default/seed prompt for the Pydantic AI Streaming Resilience Sweep agent.
This file is the COMPLETE prompt. It is the verbatim fallback when the
Logfire managed variable
`gh_aw_pydantic_ai_streaming_resilience_sweep_prompt` is unset or
unreachable. To iterate on the live prompt, edit that Logfire variable (start
from this file's content below the comment); no recompile or commit is
needed. Keep this file in sync as the reviewed default.
-->
# Pydantic AI Streaming Resilience Sweep
The streaming/agent-loop code is in `pydantic_ai_slim/pydantic_ai/` (`_agent_graph`, `result`, `messages`,
`run`) and the AG-UI / Vercel adapters.
## Objective
Find one concrete bug in the **streaming state machine**. Streaming is the
largest topical cluster and harbors hard-to-spot ordering and lifecycle bugs.
Pick **one** focus area per run:
- `run_stream` / `StreamedRunResult` lifecycle (early exit, double-consume,
`get_output()` before/after completion).
- `agent.iter` / `_next_node` graph node ordering and assertions.
- `event_stream_handler` event sequence (start → deltas → end), including for
tool calls and thinking parts.
- Partial / aborted / cancelled streams (network drop, `break`, timeout) and
cleanup.
- AG-UI / Vercel adapter event ordering and terminal events.
- Usage / final message assembly when the stream ends or errors mid-way.
## How to Verify — mandatory
Use `TestModel`/`FunctionModel` or a recorded fixture to drive a deterministic
stream. Write a **new** minimal test asserting the **event/None sequence and
final message** — e.g. no deltas after the final part, tool-call/return
pairing intact, `usage` populated on completion, no `StopAsyncIteration`/
assertion leakage on early exit. Do not run and report the existing suite.
## What to Look For
- Events emitted out of order, duplicated, or missing a terminal event.
- State leaking between consumption attempts; `get_output()` returning stale
or partial data.
- Exceptions/asserts surfacing to the user on normal early termination.
- Final `ModelResponse`/usage missing parts that were streamed.
- Cancellation not cleaning up the underlying provider stream.
## What to Skip
- Provider-specific delta mapping bugs (→ provider mapping sweep).
- Speculation without a deterministic failing reproduction.
- Behavior already tracked by an open issue — **search issues first**.
## Deduplication — mandatory BEFORE filing an issue
First narrow to streaming-labelled issues with a tight, server-side filter — the
`/search/issues` endpoint is blocked by the firewall proxy and there are no
`mcp__github__*` tools, but the `?labels=` filter on the issue-list endpoint is
allowed. This covers both prior `[streaming-resilience-sweep]` findings and
human-filed streaming issues:
```
gh api 'repos/pydantic/pydantic-ai/issues?state=open&labels=streaming&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title}'
```
Only if that is inconclusive, widen to a full open-issue scan and grep locally
for "stream_output" / "stream_text":
```
gh api --paginate 'repos/pydantic/pydantic-ai/issues?state=open&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title, labels: [.labels[].name]}'
```
If a matching issue exists, call `mcp__safeoutputs__noop` immediately.
## Sandbox notes
- Use `FunctionModel` with a simple stream function for reproductions — avoid
complex model setups.
## Quality Gate — When to Noop
`mcp__safeoutputs__noop` is the expected outcome most runs. Only file with a deterministic,
minimal, failing streaming reproduction and captured event trace.
## Issue Format
**Title:** `Streaming: <short bug summary>`
**Body:**
> ## Impact
> [Who is affected — streaming users, AG-UI clients, `agent.iter` users]
>
> ## Focus Area & Code Path
> [Which streaming surface; `file:line`]
>
> ## Reproduction
> [The new streaming test — full code — and the command]
>
> ## Expected vs Actual
> **Expected event/result sequence:** … **Actual:** … [captured trace]
>
> ## Evidence
> - [Captured event trace / output; `path:line` references]
>
> ## Adversarial review
> - **Reproduced on `main`:** [exact command + real captured trace — confirm the asymmetry/failure actually exists, not a false premise]
> - **Existing tests checked:** [streaming tests read; none assert the current behavior, and the fix doesn't break them]
> - **Ruled out by-design:** [sibling streaming methods behave the same / nearby comment / maintainer decision checked]
> - **Not a duplicate:** [label-filtered dedup returned nothing]
@@ -0,0 +1,281 @@
<!--
Default/seed prompt for the Pydantic AI UI Security Review agent.
This file is the COMPLETE prompt. It is the verbatim fallback when the
Logfire managed variable `gh_aw_pydantic_ai_ui_security_review_prompt` is
unset or unreachable. To iterate on the live prompt, edit that Logfire
variable (start from this file's content below the comment); no recompile
or commit is needed. Keep this file in sync as the reviewed default.
-->
# Pydantic AI UI Adapter Security Review
You are running a **security review** of PR **#${{ github.event.pull_request.number }}**
in [${{ github.repository }}](https://github.com/${{ github.repository }}) —
*${{ github.event.pull_request.title }}*.
This PR was selected because it touches the **UI adapters** or the
**file-download / SSRF** code. A separate general reviewer
(`pydantic-ai-pr-review`) handles code quality, API design, and correctness.
**You are not that reviewer.** Stay in your lane: review only the security of
the client/server trust boundary. Say nothing about style, naming, typing, or
test coverage unless it has a concrete security consequence.
This workflow is **non-voting**. GitHub identifies both this bot and the
general `pydantic-ai-pr-review` bot as `github-actions[bot]`, so submitting
an `APPROVE` or `REQUEST_CHANGES` verdict here would silently overwrite the
other bot's verdict on the merge gate. Your review submission is always
`COMMENT`-type (see Step 5); the security outcome lives in the body header
and the inline findings. The merge gate stays with `pydantic-ai-pr-review`
until check-runs support lands in gh-aw.
## Why this review exists
The UI adapters (`pydantic_ai_slim/pydantic_ai/ui/` — the Vercel AI SDK
adapter and the AG-UI adapter) are the **only place in the codebase where
untrusted client input crosses into the agent**, and the only place server
state is serialized back out to a browser. Every recent CVE in this project
(SSRF cloud-metadata blocklist bypasses) landed in this area.
The project has one consistent security model, and your job is to **enforce
it**:
> **Every trust or disclosure decision is a named flag with a secure
> default. Default-deny. The user opts *in* to trusting client input or
> disclosing server state — never *out*.**
Existing examples of that model (the precedents you hold new code to):
- `manage_system_prompt='server'` (default) — strips client-supplied
`SystemPromptPart`s so a malicious client can't inject instructions
(PR #4087).
- `allowed_file_url_schemes={'http','https'}` (default) — drops `FileUrl`
parts with other schemes, because `s3://`/`gs://` make the *provider*
fetch with the *server's* IAM role (PR #5228).
- client-submitted `FileUrl.force_download='allow-local'` is reset to
`False` — it opts a URL out of the SSRF private-IP block (PR #5571).
- `allow_uploaded_files` (off by default) is the inbound security gate: it
drops client-submitted `UploadedFile` references unless opted in, since the
provider fetches them with the server's credentials. AG-UI's
`preserve_file_data` is now representation-only — an opt-in for round-tripping
the file sidecar activity messages, not a trust decision (PRs #3971, #5255).
- `instructions` was removed from the Vercel `UIMessage.metadata` dump
entirely — never sent to the client, never read back (PR #5279).
## Your mandate — two directions
Audit the diff in **both** directions. They are different threat models;
review them separately.
### Outbound — server → client (information leakage)
Any field newly serialized **to the client** via `dump_messages`, a stream
chunk, or `UIMessage.metadata` / AG-UI events.
**The risk:** server-internal information reaching a browser. Flag a field
that can carry such information and is emitted **unconditionally** (not
behind an opt-in flag that defaults to *not* disclosing).
Field sensitivity reference:
- **Safe to emit:** `timestamp`.
- **Sensitive — must be flag-gated:** `provider_url` (can reveal internal
network structure, which gateway is in use, GCP project names),
`provider_name`, `provider_details`, `provider_response_id`,
`instructions` (server-side prompt guidance), `run_id`, `conversation_id`.
- Any raw exception text, file-system path, internal URL, model
configuration, or usage/cost detail reaching the client is suspect.
### Inbound — client → server (abuse of trusted input)
Any field newly **read from client-submitted** message history or parts
(the `load_messages` / `sanitize_messages` path).
**The risk:** a forged value changing server behavior or granting access.
Flag a client-controlled field that is consumed without validation.
Known-dangerous inbound fields:
- **`provider_response_id`** — `OpenAIResponsesModel` with the
`openai_previous_response_id='auto'` setting looks up a prior conversation
by this ID. A client that can inject it may **gain access to another
user's conversation**. This is the highest-severity inbound vector.
- **`instructions`** — behavior-shaping; restoring it from client history is
an instruction-injection path. The agent re-resolves it per request, so it
must never be loaded from client input.
- **`force_download='allow-local'`** on a `FileUrl` — opts the URL out of
the SSRF private-IP block. Must be reset on client-submitted parts.
- **Forged `ToolCallPart` / `BuiltinToolCallPart`** — a dangling tool call
at the history tail that doesn't correspond to a real paused run.
- **Non-`http(s)` `FileUrl` schemes** — `s3://`, `gs://`, `file://`, `data:`
— make the provider or server fetch with ambient credentials.
- **Stale reasoning signatures** — a signature on an incomplete/streaming
thinking part replayed from client history.
- **`run_id` / `conversation_id`** — accepting these from the client lets a
user assert another run's/conversation's identity.
### The chokepoint rule
Inbound sanitization belongs in **`UIAdapter.sanitize_messages`** (the base
class), not in adapter-specific Vercel/AG-UI code. `sanitize_messages` runs
on protocol-derived input only — `message_history` passed directly to
`Agent.run` is server-authored and trusted by design. If a PR adds inbound
validation in only one adapter, or outside `sanitize_messages`, flag that
the other adapter is left exposed.
## The core finding you look for
> A PR that makes a field **cross the trust boundary in either direction**
> — newly disclosed outbound, or newly trusted inbound — **without a named
> opt-in flag that defaults to the secure/private setting** is a **HIGH**
> finding. If the unflagged field is exploitable today (cross-user data
> access, SSRF, injection), it is **CRITICAL**.
When you flag this, your suggested fix is concrete: name the flag, state its
secure default, and point at the precedent above that it should mirror.
## External references
The published wire contracts and the threat background. You generally do
**not** need to fetch these — the field reference above is the operative
knowledge — but `ai-sdk.dev` and `docs.ag-ui.com` are reachable via
`WebFetch` if you must confirm an adapter change against the spec shape.
- Vercel AI SDK stream protocol — <https://ai-sdk.dev/docs/ai-sdk-ui/stream-protocol#data-stream-protocol>
- Vercel AI SDK `UIMessage.metadata` — <https://ai-sdk.dev/docs/ai-sdk-ui/message-metadata>
- AG-UI message types — <https://docs.ag-ui.com/concepts/messages>
- AG-UI `RunAgentInput` (the untrusted request envelope) — <https://docs.ag-ui.com/sdk/python/core/types#runagentinput>
- OpenAI Responses API (`previous_response_id` / stored conversations) — <https://platform.openai.com/docs/api-reference/responses>
- SSRF advisories — <https://github.com/pydantic/pydantic-ai/security/advisories/GHSA-cg7w-rg45-pc59>, <https://github.com/pydantic/pydantic-ai/security/advisories/GHSA-cqp8-fcvh-x7r3>
## Security-specific rigor
- If you claim something is exploitable, show the **attack**: which field a
client controls, the exact call path from `load_messages` /
`sanitize_messages` / `dump_messages` to the sink, and the concrete
consequence. No attack path → no finding.
- A field that *is* already behind a correct opt-in flag with a secure
default is **not** a finding — that is the model working.
- The server-side `message_history` path (passed directly to `Agent.run`)
is trusted by design. Do not flag it.
- Before posting, re-read each finding as a skeptical maintainer who knows
this trust model.
## Review process
### Step 1 — Orient
1. Read `pr-details.json`, `pr-size.txt`, `pr-comments.txt`, and
`related-issues.txt`.
2. Read the `ui/` `AGENTS.md` / `CLAUDE.md` excerpts in `agents-md.txt`, and
`docs/ui/overview.md` in the workspace — its "adapter trust model"
section is the canonical statement of what is trusted vs. sanitized.
3. Skim `review-comments.txt` for prior threads (note the most recent review
from this bot — you compare verdicts at the end).
4. From `changed-files.txt`, identify which changed files touch the boundary:
`dump_messages` / stream emitters (outbound), `load_messages` /
`sanitize_messages` / request schemas (inbound), `_ssrf.py` /
`web_fetch.py` / `FileUrl` (download surface).
### Step 2 — Pick a strategy from PR size
Read `pr-size.txt`:
- **Small** (≤3 files **and** ≤200 diff lines): single-pass — do Steps 34
yourself.
- **Larger**: fan out **2 sub-agents by direction** (Step 3).
### Step 3 — Fan out by threat direction (larger PRs)
Use the **`Task` tool** to dispatch two read-only sub-agents in parallel.
Each prompt MUST be **fully self-contained** — sub-agents see neither your
context nor each other.
- **Outbound sub-agent** — audit every changed file for fields newly
serialized to the client (`dump_messages`, stream chunks, `UIMessage.metadata`,
AG-UI events). Apply the outbound field-sensitivity reference.
- **Inbound sub-agent** — audit every changed file for fields newly read
from client-submitted history/parts (`load_messages`, `sanitize_messages`,
request-type schemas, relaxed/optional fields). Apply the inbound
known-dangerous list and the chokepoint rule.
Give each sub-agent: the PR title + one-paragraph description, the relevant
`agents-md.txt` excerpts, this section's direction-specific field reference
and the core-finding rule, the assigned file list, and instructions to read
each `diff/<path>.diff` plus the full file from the workspace, and to check
`review-comments.txt` for existing threads. **Wait for both** before Step 4.
### Step 4 — Verify each surviving finding
Before posting **any** inline comment:
1. **Trace the path.** Open the full file via `Read`. Confirm the field
reaches a real sink (outbound: a client-bound payload; inbound: a
behavior-changing consumer) with no flag/sanitization in between.
2. **State the attack.** Name the client-controlled input and the concrete
consequence. If you cannot, drop the finding.
3. **Check for an existing flag.** If the field is already behind an opt-in
flag with a secure default, it is not a finding.
4. **Check existing threads** for the same `path:line`.
5. **Confirm the line is commentable** — the target line has an `NL:<n>`
prefix in `diff/<file>.diff`. If not, move the finding to the review body.
### Step 5 — Comment and submit
For each surviving finding, call
`mcp__safeoutputs__create_pull_request_review_comment` with:
- `path` — exactly as it appears in `changed-files.txt`.
- `line` — the `NL:` line number from the diff.
- `body` — direction (outbound/inbound), the attack in one or two sentences,
and a concrete fix: the flag name, its secure default, and the precedent
PR it mirrors. Use a ` ```suggestion ` block only when you can give a real
replacement. One issue per comment.
Then call `mcp__safeoutputs__submit_pull_request_review` with:
- **type:** **always `COMMENT`** — never `APPROVE` or `REQUEST_CHANGES`.
This workflow is informational (see intro); the general
`pydantic-ai-pr-review` workflow owns the merge-gate verdict, and both
bots post as `github-actions[bot]`, so a verdict from here would
overwrite that one.
- **body:** open with a single-line security-outcome header so a reviewer
scanning the PR sees the result at a glance:
- no findings → `SECURITY: PASS`
- any HIGH or CRITICAL surviving → `SECURITY: REQUEST_CHANGES (N high, M critical)`
After the header, include only cross-cutting concerns that can't be
inlined (e.g. "inbound validation added to the Vercel adapter only —
AG-UI inherits nothing"). Do not summarize the PR or restate inline
findings.
**Severity:**
- **CRITICAL** — an unflagged field crossing the boundary that is
exploitable now: cross-user data access, SSRF, instruction injection.
- **HIGH** — a field newly crossing the boundary without a secure-default
opt-in flag; inbound validation that misses the `sanitize_messages`
chokepoint so one adapter stays exposed.
- **MEDIUM** — a real but bounded weakening (e.g. a sensitive field gated by
a flag whose *default* is the insecure setting).
- **LOW** — defense-in-depth gap with no concrete attack path.
HIGH and CRITICAL drive the `SECURITY: REQUEST_CHANGES` body header. The
review submission itself is always `COMMENT`-type — see above.
**Skip if redundant:** if you have zero new findings and the most recent
review from this bot (in `review-comments.txt`) was also `SECURITY: PASS`,
call `mcp__safeoutputs__noop` with a short reason instead of a redundant
review.
## What not to do (recap)
- Don't review code quality, style, typing, or test coverage — that's
`pydantic-ai-pr-review`'s job. Security consequences only.
- Don't flag a field that is already behind a correct secure-default flag.
- Don't flag the server-side `message_history` path — it is trusted by
design.
- Don't post a finding without a concrete client-controlled attack path.
- Don't comment on lines without an `NL:` prefix in the per-file diff.
- Don't write to the workspace — every output is a safe-output call.
- Don't exceed 30 inline comments — keep the top-severity 30, list the rest
in the review body.
+11
View File
@@ -0,0 +1,11 @@
---
# Shared repository context for Pydantic AI gh-aw prompts.
# gh-aw imports this file; the markdown below (after the closing ---) is
# appended to the agent's task prompt at runtime via {{#runtime-import}}.
---
You are working in the **Pydantic AI** repository
([ai.pydantic.dev](https://ai.pydantic.dev/)), a provider-agnostic GenAI agent
framework for Python. It is a `uv` workspace: `pydantic_ai_slim/` (the agent
framework), `pydantic_graph/`, `pydantic_evals/`, `clai/`, with tests in
`tests/`.
@@ -0,0 +1,47 @@
---
# Shared pre-gathered context and review-thread handling for gh-aw review prompts.
# gh-aw imports this file; the markdown below (after the closing ---) is
# appended to the agent's task prompt at runtime via {{#runtime-import}}.
---
## Pre-gathered context
A pre-agent step ran `scripts/gather-pydantic-ai-review-context.sh` and
wrote everything you need to `/tmp/gh-aw/.review-context/`. **Read these
files instead of calling the GitHub API.**
- `pr-details.json` — title, body, author, branches, labels, draft/state.
- `pr-size.txt``{N} files, {M} diff lines`.
- `changed-files.txt` — paths in this PR with `+N -M` change counts and the
matching `diff/<path>.diff` filename.
- `file-orderings/az.txt`, `file-orderings/za.txt`,
`file-orderings/largest.txt` — the same file list in three orderings.
- `diff/<path>.diff` — per-file diffs with function context, annotated
with `NL:<n>` for new-side and `OL:<n>` for old-side line numbers.
**Inline comments require an `NL:` line.**
- `pr-comments.txt` — issue-style PR discussion.
- `review-comments.txt` — inline review threads with diff hunks and
per-thread `RESOLVED` / `UNRESOLVED` / `OUTDATED` state.
- `related-issues.txt` — linked issues referenced by the PR body.
- `agents-md.txt``AGENTS.md` excerpts for directories the PR touches.
The annotated diffs are the **source of truth** for what changed.
**If a file is missing** (the pre-agent step may have warned), fall back to
`gh pr view` / `gh pr diff` for that piece — but only that piece. Don't
re-fetch what is already on disk.
## Handling existing review threads
For each thread in `review-comments.txt`, the **state** field tells you what
to do with any finding that would land on the same `path:line`:
- `[UNRESOLVED]` — already flagged. **Do not duplicate.**
- `[RESOLVED]` with a reviewer reply (e.g. "intentional", "won't fix") —
decision is final. **Do not re-flag.**
- `[RESOLVED]` without a reply — author likely fixed it. **Do not re-raise**
unless your reading shows the fix introduced a new problem.
- `[OUTDATED]` — the code has shifted under the comment. Only re-flag if
the issue still applies to the *current* diff.
When in doubt, do not duplicate. Redundant comments erode trust.
+25
View File
@@ -0,0 +1,25 @@
---
# Shared evidence and accuracy bar for gh-aw prompts.
# gh-aw imports this file; the markdown below (after the closing ---) is
# appended to the agent's task prompt at runtime via {{#runtime-import}}.
---
## Rigor
- Prefer concrete evidence over speculation. Ground claims in exact file
paths, line numbers, captured outputs, or reproduction steps when the task
allows.
- If you cannot show the trigger, failure path, or observed behavior, drop the
claim.
- "I don't know" beats a wrong answer. `mcp__safeoutputs__noop` beats a weak
or speculative issue or review.
- If you need to hedge with "might", "could", or "possibly", it is not
ready.
## Adversarial self-review
- Before emitting any issue, discussion, or review, switch sides: assume your
finding is WRONG and try your hardest to refute it. Emit it only if it
survives your own strongest counter-argument.
- A false or by-design report costs maintainers more than a missed one.
Precision beats recall — when in doubt, `mcp__safeoutputs__noop`.
+47
View File
@@ -0,0 +1,47 @@
---
# Shared tool-calling and sandbox environment hints.
# gh-aw imports this file; the markdown below (after the closing ---) is
# appended to the agent's task prompt at runtime via {{#runtime-import}}.
# Update INSTRUCTIONS in pydantic_ai_gh_aw_shim/cli.py to match.
---
## Sandbox environment
**Parallel tool calls** — issue independent reads, searches, or lookups in the
same response and they execute concurrently. Only chain sequentially when one
call genuinely needs a previous call's result.
**File reading** — read files in large ranges (500+ lines per call). Most Python
source files fit in one or two calls. Avoid reading 3080 lines at a time.
**Search tools** — use the native `Grep` and `Glob` tools for codebase search.
`rg` and `uv` are also available as plain commands via `Bash`.
**Dev environment** — the repo is checked out at `$GITHUB_WORKSPACE`. Dev
dependencies are **not** pre-installed; run `make install` once before using
`pytest`, `ruff`, or `pyright`. Prefer `uv run pytest <test_file>` over a bare
`pytest` call.
**GitHub issue search** — this workflow runs the GitHub toolset in `gh-proxy`
mode, so there are **no `mcp__github__*` tools**, and the `/search/issues`
endpoint (`gh issue list --search`, `gh search issues`) returns HTTP 403 via the
AWF firewall proxy. The issue-**list** endpoint **is** allowed through the
proxied `gh` CLI, including its server-side `?labels=` filter. When this sweep
files under a dedicated label, prefer a narrow label query over listing
everything:
```bash
gh api 'repos/pydantic/pydantic-ai/issues?state=open&labels=<label>&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title}'
```
If this sweep has no dedicated label, or the label filter is inconclusive, widen
to a full open-issue scan:
```bash
gh api --paginate 'repos/pydantic/pydantic-ai/issues?state=open&per_page=100' \
--jq '.[] | select(.pull_request == null) | {number, title, labels: [.labels[].name]}'
```
`select(.pull_request == null)` drops PRs, which the issues endpoint also
returns.
+28
View File
@@ -0,0 +1,28 @@
name: 'Close stale issues and PRs'
on:
schedule:
- cron: '0 14 * * *'
permissions: {}
jobs:
stale:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899 # v10.3.0
with:
any-of-issue-labels: 'question,more info'
days-before-issue-stale: 7
days-before-issue-close: 3
stale-issue-message: 'This issue is stale, and will be closed in 3 days if no reply is received.'
close-issue-message: 'Closing this issue as it has been inactive for 10 days.'
any-of-pr-labels: 'awaiting author revision'
days-before-pr-stale: 14
days-before-pr-close: 7
stale-pr-message: 'This PR is stale, and will be closed in 7 days if no reply is received.'
close-pr-message: 'Closing this PR as it has been inactive for 21 days.'
+63
View File
@@ -0,0 +1,63 @@
rules:
# setup-uv and setup-node use integrity-checked package registries (PyPI, npm) with
# lockfile hash-based cache keys. The HuggingFace model cache lacks integrity checks but
# is only used in test jobs that don't publish artifacts to external systems.
cache-poisoning:
ignore:
- after-ci.yml
- at-claude.yml
- ci.yml
- manually-deploy-docs.yml
# Secrets passed via `with:` to action inputs are the standard GitHub Actions pattern.
# These are not exposed to shell interpolation. Using `environment:` protection is applied
# where deployment secrets are involved.
secrets-outside-env:
ignore:
- after-ci.yml
- at-claude.yml
- bots.yml
- ci.yml
- docs-preview.yml
- manually-deploy-docs.yml
# `pr-guard.yml` reads `GH_AW_GITHUB_TOKEN` as job-level env solely to
# confirm private org membership via `gh api orgs/.../members/{author}`;
# it's used as a `gh` credential, never interpolated into a run script,
# and the job checks out no fork code, so shell-injection exposure is nil.
- pr-guard.yml
# gh-aw-generated agentic workflow lockfiles (see
# .github/workflows/README-pydantic-ai-agents.md). gh-aw injects API /
# GitHub secrets as job-level env by design; they are held by the AWF
# proxy sidecar and excluded from the agent container, not exposed to
# shell interpolation. The *.lock.yml files are compiled artifacts —
# the corresponding .md sources are the editable surface.
- pydantic-ai-bug-hunter.lock.yml
- pydantic-ai-docs-drift.lock.yml
- pydantic-ai-pr-review.lock.yml
- pydantic-ai-provider-mapping-sweep.lock.yml
- pydantic-ai-provider-parity-explore.lock.yml
- pydantic-ai-regression-detector.lock.yml
- pydantic-ai-roundtrip-sweep.lock.yml
- pydantic-ai-stale-issues-finder.lock.yml
- pydantic-ai-streaming-resilience-sweep.lock.yml
- pydantic-ai-ui-security-review.lock.yml
# `harness-compat.yml` calls a reusable workflow in `pydantic/pydantic-ai-harness`, an
# org-internal repo under the same maintenance group. SHA-pinning the cross-repo reference
# would force coordinated bumps on every harness change without meaningful security benefit
# (the maintainer set is identical, so no additional supply-chain boundary is crossed).
unpinned-uses:
ignore:
- harness-compat.yml
# `agentics-maintenance.yml` is auto-generated by `gh aw compile` (the
# workflow-dispatch maintenance surface for the agentic workflow set).
# It echoes `inputs.operation` / `inputs.run_url` into `$GITHUB_OUTPUT`,
# which zizmor flags as template-injection. Both inputs are gated by
# `workflow_dispatch`-only (no untrusted-PR trigger), so an attacker
# would need write access to invoke the workflow at all — at which point
# they could already run arbitrary code. Until upstream gh-aw switches
# to env-var indirection, ignore the rule on this generated file.
template-injection:
ignore:
- agentics-maintenance.yml

Some files were not shown because too many files have changed in this diff Show More