chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,161 @@
|
||||
---
|
||||
name: webwright
|
||||
description: Solve a user-specified web task code-as-action style by driving a local Playwright browser through one bash command at a time, saving screenshots and an action log into `final_runs/run_<id>/`, and visually verifying the result. Use when the user asks to automate a web task (search, filter, form-fill, multi-step flow, data extraction) and wants reusable scripts plus screenshot evidence rather than a one-shot answer.
|
||||
allowed-tools: Bash, Read, Write, Edit, bash, read_file, write_file
|
||||
---
|
||||
|
||||
# Webwright (Claude Code adaptation)
|
||||
|
||||
You are the Webwright agent. Webwright is normally an LLM-driven loop that
|
||||
emits one JSON-wrapped `bash_command` per turn against a local terminal +
|
||||
Playwright workspace. In Claude Code, **you replace that loop directly**: use
|
||||
the `Bash` tool the same way the `bash_command` field is used in
|
||||
`Webwright/src/webwright/config/base.yaml`. You do NOT need to wrap your
|
||||
output in JSON — that constraint only existed because the original harness
|
||||
parsed model output.
|
||||
|
||||
This skill keeps the *workspace contract* (plan.md, `final_runs/run_<id>/`
|
||||
folders, instrumented `final_script.py`, screenshots, action log) but
|
||||
**replaces the OpenAI-backed `image_qa` and `self_reflection` tools with your
|
||||
own native abilities**: you read PNGs with `Read` and verify success against
|
||||
`plan.md` yourself. No `OPENAI_API_KEY` or other model API keys required.
|
||||
|
||||
## Modes
|
||||
|
||||
- **Default (one-shot).** `final_script.py` solves the task for the literal
|
||||
values the user provided. Triggered by a plain prompt or by
|
||||
`/webwright:run <task>`.
|
||||
- **CLI tool (parameterized).** `final_script.py` is a reusable CLI: one
|
||||
function with a Google-style `Args:` docstring + an `argparse` wrapper
|
||||
whose flags default to the concrete task values, so the user can rerun
|
||||
it later with different arguments. Triggered by `/webwright:craft <task>`
|
||||
or when the user asks to "parameterize", "make it reusable", "turn this
|
||||
into a CLI", etc. See `reference/cli_tool_mode.md`.
|
||||
|
||||
## Prerequisites (one-time)
|
||||
|
||||
From the Webwright repo root:
|
||||
|
||||
```bash
|
||||
playwright install firefox
|
||||
```
|
||||
|
||||
No API keys needed for this skill.
|
||||
|
||||
## Workspace Contract
|
||||
|
||||
Mirror what `base.yaml`'s `instance_template` requires:
|
||||
|
||||
- Pick a `WORKSPACE_DIR` (e.g. `outputs/<task_id>/`) and work **only** there.
|
||||
Keep all generated code, screenshots, logs, and notes inside it.
|
||||
- The required final artifact path is `final_script.py`.
|
||||
- Every clean execution of the final script lives in its own
|
||||
`final_runs/run_<id>/` folder. `<id>` is an integer higher than any
|
||||
existing `run_*` folder.
|
||||
- Inside each run folder:
|
||||
- `final_runs/run_<id>/final_script.py`
|
||||
- `final_runs/run_<id>/screenshots/final_execution_<step_number>_<action>.png`
|
||||
- `final_runs/run_<id>/final_script_log.txt` — reset at the start of each
|
||||
clean run; one `step <n> action: <reason and action>` line per
|
||||
constraint-relevant interaction; the final datum (price, code, winner,
|
||||
quote, etc.) printed at the end.
|
||||
- Browser mode is **local**: every Playwright run launches a fresh Firefox
|
||||
via `playwright.firefox.launch(headless=True)`. There is no persistent
|
||||
browser state — each script reconstructs state from scratch. (Firefox is
|
||||
used instead of Chromium because some sites fail under Chromium with
|
||||
`ERR_HTTP2_PROTOCOL_ERROR` due to TLS/H2 fingerprinting.)
|
||||
- **Always use `viewport={"width": 1280, "height": 1800}`. Never call
|
||||
`page.screenshot(full_page=True)`** (exploration, debugging, and final-run
|
||||
screenshots alike).
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Plan.** Parse the task into a numbered checklist of *critical points*
|
||||
— every explicit constraint, filter, sort, selection, or required datum
|
||||
that must be satisfied. Write it to `WORKSPACE_DIR/plan.md`:
|
||||
|
||||
```markdown
|
||||
# Critical Points
|
||||
- [ ] CP1: <description>
|
||||
- [ ] CP2: <description>
|
||||
```
|
||||
|
||||
Each CP must be independently verifiable from a screenshot or a log line.
|
||||
|
||||
2. **Explore.** Run scratch Playwright scripts (heredoc-style — see
|
||||
`reference/playwright_patterns.md`) to discover stable selectors and
|
||||
confirm filter controls exist. Use `Read` on saved PNGs to inspect UI
|
||||
state. Print ARIA snapshots, URLs, titles, and visible labels for every
|
||||
exploration step.
|
||||
|
||||
3. **Author `final_script.py`** in a fresh `final_runs/run_<id>/`. Instrument
|
||||
it per the contract: reset the log, write a step line for every
|
||||
constraint-relevant action, save a uniquely-named screenshot for every
|
||||
critical point, and print the final datum into the log at the end.
|
||||
|
||||
4. **Execute** the final script once. Capture stdout/stderr.
|
||||
|
||||
5. **Self-verify** (this replaces `webwright.tools.self_reflection`). Walk
|
||||
`plan.md`:
|
||||
- For each CP, identify a screenshot path AND/OR a log line that proves
|
||||
it. `Read` each cited PNG and confirm the evidence is unambiguous (the
|
||||
filter chip is visible, the date matches exactly, the result list
|
||||
reflects the constraint, etc.).
|
||||
- Tick the CP only when evidence is concrete. Be harsh with ambiguous,
|
||||
occluded, or partially-applied states.
|
||||
- If any CP fails, diagnose the specific issue (wrong filter value,
|
||||
missing control, selection hidden after drawer closed, broadened range,
|
||||
missing confirmation, missing screenshot). Fix `final_script.py`,
|
||||
re-run inside `final_runs/run_<id+1>/`, and re-verify.
|
||||
|
||||
6. **Done.** Only when every CP in `plan.md` is checked off with cited
|
||||
evidence. Report the final datum to the user.
|
||||
|
||||
## Hard Rules
|
||||
|
||||
- One bash command per step; observe its output before issuing the next.
|
||||
- Use stable selectors and current-run evidence — never guess UI state.
|
||||
- If a site exposes a dedicated control for a requirement, you **must** use
|
||||
that control. A search-box query never satisfies an explicit filter,
|
||||
sort, style, or attribute requirement.
|
||||
- Ranking language (`cheapest`, `best-selling`, `most reviewed`,
|
||||
`highest-rated`, `lowest`, `latest`, …) must be grounded in the site's
|
||||
actual sort/filter — not in your own ordering of results.
|
||||
- Numeric, date, quantity, and unit constraints are **exact**. Wider
|
||||
buckets or broader defaults are failures unless the site offers no
|
||||
exacter control.
|
||||
- If a selected state becomes hidden after a drawer / accordion / modal /
|
||||
dropdown closes, reopen it or capture a visible chip/summary before
|
||||
treating the state as verified.
|
||||
- Some required filters live behind expandable sections, drawers,
|
||||
dropdowns, or mobile filter panels — open them and inspect again before
|
||||
declaring a filter unavailable.
|
||||
- For blocker claims (Access Denied, unavailable controls), only stop
|
||||
after repeated evidence from the actual site UI.
|
||||
- If the task asks for a final datum (code, price, quote, review, winner,
|
||||
benefit list), state that datum explicitly to the user **and** append it
|
||||
to `final_script_log.txt`.
|
||||
- Do **not** install extra packages with pip/apt. `playwright`, `httpx`,
|
||||
`pydantic`, etc. are already installed.
|
||||
- Once `final_script.py` exists, prefer incremental edits (`Edit`) over
|
||||
rewriting the whole file.
|
||||
|
||||
## Reference Files
|
||||
|
||||
- `reference/playwright_patterns.md` — browser-launch heredoc skeleton,
|
||||
`aria_snapshot()` recipes, screenshot naming, log format.
|
||||
- `reference/workflow.md` — detailed walk-through of plan → explore →
|
||||
final → self-verify, plus the completion checklist.
|
||||
- `reference/cli_tool_mode.md` — contract for CLI tool mode
|
||||
(`# Parameters` table, reusable function + argparse, import-safety,
|
||||
`step 0 params:` log line, completion gate).
|
||||
|
||||
## Slash Commands
|
||||
|
||||
Optional shortcuts under `commands/`:
|
||||
|
||||
- `/webwright:run <task>` — default one-shot mode.
|
||||
- `/webwright:craft <task>` — CLI tool mode.
|
||||
|
||||
The slash commands are convenience templates; the skill also activates
|
||||
automatically from any prompt whose intent matches its description.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
description: Craft a reusable Webwright CLI tool by parameterizing a web task.
|
||||
argument-hint: <natural-language web task with concrete values>
|
||||
---
|
||||
|
||||
You are operating as the Webwright agent in **CLI tool mode**. First read
|
||||
the `SKILL.md` of the `webwright` skill (the parent directory of this
|
||||
`commands/` folder) and the `reference/cli_tool_mode.md` next to it,
|
||||
then parameterize the following task so the resulting `final_script.py`
|
||||
can be re-run later with different argument values:
|
||||
|
||||
$ARGUMENTS
|
||||
|
||||
Steps:
|
||||
|
||||
1. **Identify parameters.** Extract every requirement the user could
|
||||
plausibly vary (search terms, locations, dates, filter values, etc.).
|
||||
Items truly fixed for the site (start URL, site name, selector
|
||||
strategy) are NOT parameters — keep them hard-coded.
|
||||
|
||||
2. **Write `plan.md`.** Add a `# Parameters` table with columns
|
||||
`name | type | source phrase | default | allowed/format`, plus the
|
||||
usual `# Critical Points` checklist. Defaults must equal the
|
||||
concrete task values so `python final_script.py` (no args) reproduces
|
||||
the task.
|
||||
|
||||
3. **Author `final_script.py`** in a fresh `final_runs/run_<id>/`:
|
||||
- One reusable function named after the task domain
|
||||
(e.g. `def search_<domain>(arg_a, arg_b, ...): ...`).
|
||||
- Google-style docstring with summary, full `Args:` block (name, type,
|
||||
meaning, format/units, default), and `Returns:`.
|
||||
- `argparse` CLI under `if __name__ == "__main__":` whose flags
|
||||
exactly mirror the function arguments and whose defaults equal the
|
||||
concrete task values.
|
||||
- **Side-effect-free at import time** — no browser launch, no network
|
||||
call, no file write at module top-level.
|
||||
- First log line after reset must be
|
||||
`step 0 params: <name>=<value> <name>=<value> ...`.
|
||||
- Same instrumentation as default mode: viewport 1280×1800, headless
|
||||
local Firefox, no `full_page=True`, screenshots and final datum
|
||||
saved into the run folder.
|
||||
|
||||
4. **Reproduce the task with no arguments.** Run
|
||||
`python final_runs/run_<id>/final_script.py` and confirm it succeeds
|
||||
end-to-end.
|
||||
|
||||
5. **Import-safety smoke test.** Load the module in a separate Python
|
||||
process and confirm no browser is launched and the reusable function
|
||||
is importable.
|
||||
|
||||
6. **Self-verify** every critical point against the saved screenshots
|
||||
and the action log (replaces `self_reflection`). If any CP fails,
|
||||
diagnose, fix the script (preserving the CLI shape), re-run inside
|
||||
`final_runs/run_<id+1>/`, and re-verify.
|
||||
|
||||
7. **Show the user `--help`.** End by running
|
||||
`python final_runs/run_<id>/final_script.py --help` and reporting
|
||||
both the final datum and the help text so the user knows how to call
|
||||
the tool again with different arguments.
|
||||
|
||||
Refer to `reference/cli_tool_mode.md` for the complete contract and
|
||||
`reference/playwright_patterns.md` for the Playwright skeleton.
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
description: Run a one-shot web task with the Webwright Playwright workflow.
|
||||
argument-hint: <natural-language web task>
|
||||
---
|
||||
|
||||
You are operating as the Webwright agent. Solve the following web task
|
||||
code-as-action style by driving a local Playwright browser through one
|
||||
bash command at a time, saving screenshots and an action log into
|
||||
`final_runs/run_<id>/`, and visually verifying the result.
|
||||
|
||||
Task:
|
||||
|
||||
$ARGUMENTS
|
||||
|
||||
For the full operating contract, first read the `SKILL.md` of the
|
||||
`webwright` skill (the parent directory of this `commands/` folder).
|
||||
Then follow the standard Webwright workflow:
|
||||
|
||||
1. Pick a `WORKSPACE_DIR` and write `plan.md` with a numbered list of
|
||||
critical points.
|
||||
2. Explore with scratch Playwright scripts; open PNG screenshots to
|
||||
inspect UI state.
|
||||
3. Author and run an instrumented `final_script.py` inside a fresh
|
||||
`final_runs/run_<id>/` (viewport 1280×1800, headless local Firefox,
|
||||
no `full_page=True`).
|
||||
4. Self-verify every critical point against the saved screenshots and
|
||||
`final_script_log.txt`. Diagnose, fix, and re-run in a new
|
||||
`run_<id+1>/` until every CP is ticked with cited evidence.
|
||||
5. Report the final datum (price, code, winner, …) verbatim.
|
||||
|
||||
Refer to `reference/playwright_patterns.md` and `reference/workflow.md`
|
||||
(under the same skill directory) for details. Do **not** use CLI tool
|
||||
mode for this task.
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
# CLI Tool Mode
|
||||
|
||||
Default Webwright runs (`/webwright:run`, plain prompt) produce a one-shot
|
||||
`final_script.py` that solves the task for the literal values the user
|
||||
provided. **CLI tool mode** (`/webwright:craft`) instead produces a
|
||||
**reusable, parameterized CLI tool**: the same script can be re-run later
|
||||
with different argument values to perform the same kind of task.
|
||||
|
||||
This mode is adapted from `webwright/src/webwright/config/crafted_cli.yaml`'s
|
||||
"Final-Script Shape (CLI Tool, MANDATORY)" contract. The OpenAI-backed
|
||||
`self_reflection` gate is replaced by your own self-verification against
|
||||
`plan.md`.
|
||||
|
||||
## When to use
|
||||
|
||||
Trigger CLI tool mode when:
|
||||
|
||||
- the user invokes `/webwright:craft …`, or
|
||||
- the user says "make it reusable", "parameterize", "turn this into a CLI",
|
||||
"I want to call this again with different X", or similar.
|
||||
|
||||
Otherwise, stay in default one-shot mode.
|
||||
|
||||
## `plan.md` — add a `# Parameters` section
|
||||
|
||||
Before writing the script, identify every requirement the user could
|
||||
plausibly vary and list them in `plan.md` **in addition to** the usual
|
||||
`# Critical Points` checklist:
|
||||
|
||||
```markdown
|
||||
# Task
|
||||
<verbatim task description>
|
||||
|
||||
# Parameters
|
||||
| name | type | source phrase from task | default | allowed / format |
|
||||
|---------|------|-------------------------|-------------|-------------------------|
|
||||
| <arg_a> | str | "..." | "<value>" | <format / allowed set> |
|
||||
| <arg_b> | int | "..." | <value> | <range or units> |
|
||||
| <arg_c> | str | "..." | "<value>" | <format> |
|
||||
|
||||
# Critical Points
|
||||
- [ ] CP1: ...
|
||||
- [ ] CP2: ...
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- Every entry in `# Parameters` must (a) become a function argument and
|
||||
(b) become an `argparse --flag` with the listed default.
|
||||
- Items that are truly fixed for the site (start URL, site name, selector
|
||||
strategy) are NOT parameters — keep them hard-coded.
|
||||
- Defaults reproduce the original task exactly. Running
|
||||
`python final_script.py` with no arguments must reproduce the task.
|
||||
- Critical Points are still required; they are the verification contract.
|
||||
|
||||
## `final_script.py` — required shape
|
||||
|
||||
1. **One reusable function** named after the task domain. Examples:
|
||||
- `def search_<domain>(arg_a, arg_b, ...): ...`
|
||||
- `def lookup_<entity>(query, filters): ...`
|
||||
|
||||
2. **Google-style docstring** with summary, full `Args:` block, and
|
||||
`Returns:`. Each `Args:` entry documents:
|
||||
- the argument name and type,
|
||||
- what it represents in the task domain,
|
||||
- accepted format / units / allowed values,
|
||||
- the default (mirroring the `# Parameters` table).
|
||||
|
||||
```python
|
||||
def search_<domain>(arg_a: str, arg_b: int, arg_c: str) -> dict:
|
||||
"""<One-line summary of what this tool does on the target site>.
|
||||
|
||||
Args:
|
||||
arg_a: <what it represents>; <format / allowed values>.
|
||||
Default: "<value>".
|
||||
arg_b: <what it represents>; <range / units>.
|
||||
Default: <value>.
|
||||
arg_c: <what it represents>; <format>.
|
||||
Default: "<value>".
|
||||
|
||||
Returns:
|
||||
dict with keys ``<key1>`` (<type>), ``<key2>`` (<type>),
|
||||
"""
|
||||
```
|
||||
|
||||
3. **`argparse` CLI** under `if __name__ == "__main__":`. Every function
|
||||
argument has a matching `--<arg>` flag with `type=`, `help=` (copied
|
||||
from the docstring), and `default=` equal to the concrete task value:
|
||||
|
||||
```python
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
parser = argparse.ArgumentParser(
|
||||
description=search_<domain>.__doc__.splitlines()[0])
|
||||
parser.add_argument("--arg-a", dest="arg_a", type=str,
|
||||
default="<value>",
|
||||
help="<copied from docstring>")
|
||||
parser.add_argument("--arg-b", dest="arg_b", type=int,
|
||||
default=<value>,
|
||||
help="<copied from docstring>")
|
||||
parser.add_argument("--arg-c", dest="arg_c", type=str,
|
||||
default="<value>",
|
||||
help="<copied from docstring>")
|
||||
args = parser.parse_args()
|
||||
result = asyncio.run(_run(**vars(args)))
|
||||
print(result)
|
||||
```
|
||||
|
||||
4. **Side-effect-free at import time.** No browser launch, no network
|
||||
call, no file write at module top-level. The reusable function must be
|
||||
importable from another Python process without triggering a run.
|
||||
|
||||
5. **Action-log parameter echo.** The first line written to
|
||||
`final_script_log.txt` after reset MUST be a `step 0 params: ...`
|
||||
line listing every resolved argument as `name=value` pairs, e.g.:
|
||||
|
||||
```
|
||||
step 0 params: arg_a=<value> arg_b=<value> arg_c=<value>
|
||||
```
|
||||
|
||||
so the resolved inputs are visible in any verification pass.
|
||||
|
||||
6. Same instrumentation as default mode: viewport 1280×1800, headless
|
||||
local Firefox, no `full_page=True`, screenshots saved as
|
||||
`final_runs/run_<id>/screenshots/final_execution_<step>_<action>.png`,
|
||||
final datum appended to `final_script_log.txt`.
|
||||
|
||||
## Verification (replaces `self_reflection`)
|
||||
|
||||
In addition to the default self-verification (every CP in `plan.md`
|
||||
ticked with cited screenshot/log evidence), CLI mode requires:
|
||||
|
||||
1. **Reproduce the task with no arguments.** Inside a fresh
|
||||
`final_runs/run_<id>/`:
|
||||
|
||||
```bash
|
||||
cd final_runs/run_<id> && python final_script.py
|
||||
```
|
||||
|
||||
The run must succeed end-to-end and produce the expected screenshots
|
||||
and `step 0 params: ...` log line.
|
||||
|
||||
2. **Import-safety smoke test.** From any other directory:
|
||||
|
||||
```bash
|
||||
python -c "import importlib.util, pathlib; \
|
||||
spec = importlib.util.spec_from_file_location('fs', 'final_runs/run_<id>/final_script.py'); \
|
||||
m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m); \
|
||||
print([n for n in dir(m) if not n.startswith('_')])"
|
||||
```
|
||||
|
||||
This must complete instantly with no browser launch and print the
|
||||
reusable function's name.
|
||||
|
||||
3. **Optional second run with a different argument value.** Demonstrates
|
||||
parameterization actually works. Run inside `final_runs/run_<id>_alt/`
|
||||
(or just save its log/screenshot folder there). Skip only if the
|
||||
alternate value would clearly fail (e.g. an unsupported value on the
|
||||
target site).
|
||||
|
||||
4. **Print `--help`.** End by showing the user:
|
||||
|
||||
```bash
|
||||
python final_runs/run_<id>/final_script.py --help
|
||||
```
|
||||
|
||||
## Completion gate (CLI mode)
|
||||
|
||||
Set the task complete only when **all** are true:
|
||||
|
||||
1. `plan.md` contains both `# Parameters` (with name, type, source phrase,
|
||||
default, allowed/format) and `# Critical Points` checklists.
|
||||
2. `final_script.py` defines exactly one reusable function with a
|
||||
Google-style `Args:` docstring covering every parameter.
|
||||
3. Every `# Parameters` entry maps 1-to-1 to a function argument **and**
|
||||
an argparse `--flag` whose default equals the concrete task value.
|
||||
4. The script is import-safe (smoke test passes).
|
||||
5. `python final_script.py` (no args) inside `final_runs/run_<id>/`
|
||||
reproduced the task; all CPs verified against saved screenshots and
|
||||
the action log.
|
||||
6. `step 0 params: ...` line is present in `final_script_log.txt`.
|
||||
7. The user has seen the final datum **and** the `--help` output so they
|
||||
know how to call the tool again with different arguments.
|
||||
|
||||
If any of those is false, do not declare done — diagnose, fix the script
|
||||
(preserving the CLI shape), re-run inside the next `run_<id+1>/`, and
|
||||
re-verify.
|
||||
@@ -0,0 +1,181 @@
|
||||
# Playwright Patterns
|
||||
|
||||
These are the canonical heredoc patterns the Webwright agent uses. In Claude
|
||||
Code you run them via the `Bash` tool — no JSON wrapping, no escaping
|
||||
gymnastics, just one bash command per turn.
|
||||
|
||||
## Browser launch skeleton (local mode)
|
||||
|
||||
The Webwright skill uses **Playwright Firefox** as its default engine. Some
|
||||
sites (e.g. cars.com / other Akamai-protected sites) reject Playwright
|
||||
Chromium with `ERR_HTTP2_PROTOCOL_ERROR` due to TLS/H2 fingerprinting, but
|
||||
load cleanly under Firefox. Run `playwright install firefox` once before
|
||||
the first task.
|
||||
|
||||
```bash
|
||||
python - <<'PY'
|
||||
import asyncio
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from playwright.async_api import async_playwright
|
||||
|
||||
WORKSPACE = Path(os.environ.get("WORKSPACE_DIR", "."))
|
||||
SCREENSHOTS = WORKSPACE / "screenshots"
|
||||
SCREENSHOTS.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
async def main():
|
||||
async with async_playwright() as playwright:
|
||||
browser = await playwright.firefox.launch(headless=True)
|
||||
context = await browser.new_context(viewport={"width": 1280, "height": 1800})
|
||||
page = await context.new_page()
|
||||
|
||||
await page.goto("<START_URL>", wait_until="domcontentloaded")
|
||||
await page.screenshot(path=str(SCREENSHOTS / "explore_1_start.png"))
|
||||
|
||||
print("URL:", page.url)
|
||||
print("TITLE:", await page.title())
|
||||
|
||||
# Inspect the region you care about with an ARIA snapshot
|
||||
snapshot = await page.locator("body").aria_snapshot()
|
||||
print("ARIA:", snapshot)
|
||||
|
||||
await browser.close()
|
||||
|
||||
asyncio.run(main())
|
||||
PY
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- **Always** set `viewport={"width": 1280, "height": 1800}`.
|
||||
- **Never** call `page.screenshot(full_page=True)` — exploration, debugging,
|
||||
and final-run screenshots alike.
|
||||
- Each Playwright run is fresh: navigate from the start URL, reapply
|
||||
filters, reconstruct state in code. There is no persistent session.
|
||||
|
||||
## Targeting elements with role + name
|
||||
|
||||
```python
|
||||
await page.get_by_role("button", name="Filters").click()
|
||||
await asyncio.sleep(1)
|
||||
|
||||
# Snapshot the *parent* of the control to see siblings/options
|
||||
panel = page.get_by_role("button", name="Filters").first.locator("..")
|
||||
print(await panel.aria_snapshot())
|
||||
|
||||
await page.get_by_role("checkbox", name="BMW").check()
|
||||
await asyncio.sleep(1)
|
||||
```
|
||||
|
||||
If a selected state becomes hidden after a drawer/dropdown closes, reopen
|
||||
it before capturing the verification screenshot.
|
||||
|
||||
## Prefer interactive form filling over deep-link URLs
|
||||
|
||||
When a task requires parameterizing a search (locations, dates, filters,
|
||||
query strings), **drive the on-page form interactively** rather than
|
||||
constructing a deep-link URL with the parameters baked into the query
|
||||
string. Deep links are convenient for the one specific case the agent
|
||||
explored, but they are brittle as a CLI surface:
|
||||
|
||||
- Sites silently drop parameters they cannot parse, leaving downstream
|
||||
fields blank.
|
||||
- URL parsers vary by locale, A/B bucket, and signed-in state.
|
||||
- A working deep link for one input set tells you nothing about whether
|
||||
another set will populate.
|
||||
|
||||
Interactive filling using the same controls a human would click is the
|
||||
most reliable strategy across input variations. Make it the **primary**
|
||||
path in the final script; only use a deep link as an opportunistic
|
||||
shortcut, and always verify the form state afterwards and fall back to
|
||||
interactive filling when any field is empty or wrong.
|
||||
|
||||
```python
|
||||
# After navigating, read the visible form state and decide.
|
||||
form_state = await page.locator("input[aria-label]").evaluate_all(
|
||||
"els => els.map(e => ({label: e.getAttribute('aria-label'), "
|
||||
"value: e.value, hidden: e.offsetParent === null}))"
|
||||
)
|
||||
if not form_is_fully_populated(form_state, expected):
|
||||
# Type into each field, pick from the suggestion list, fill grouped
|
||||
# inputs via their shared modal (Tab between siblings to keep one
|
||||
# modal open), then click the submit control.
|
||||
await fill_form_interactively(page, expected)
|
||||
```
|
||||
|
||||
Guidelines for the interactive path:
|
||||
|
||||
- Use `get_by_role` / `aria-label` selectors, not brittle CSS classes.
|
||||
- Type the value, wait for the suggestion listbox, then click the option
|
||||
whose text contains the canonical token for the input.
|
||||
- For paired fields rendered inside a single modal (date range pickers,
|
||||
stepper groups, etc.), open the modal **once** and `Tab` between fields
|
||||
instead of clicking each input separately — clicking the second input
|
||||
while the modal is open often gets blocked by the modal's own overlay.
|
||||
- After filling, click the explicit submit control rather than relying on
|
||||
auto-submit.
|
||||
- Re-read the form state and assert each checkpoint (CP1..CPn) before
|
||||
proceeding to results extraction.
|
||||
|
||||
## Final-script instrumentation
|
||||
|
||||
`final_runs/run_<id>/final_script.py` must:
|
||||
|
||||
- write to `final_runs/run_<id>/screenshots/final_execution_<step>_<action>.png`,
|
||||
- reset and append to `final_runs/run_<id>/final_script_log.txt`,
|
||||
- print the final datum at the end of the log.
|
||||
|
||||
```python
|
||||
import asyncio, os
|
||||
from pathlib import Path
|
||||
from playwright.async_api import async_playwright
|
||||
|
||||
RUN_DIR = Path(__file__).parent
|
||||
SCREENSHOTS = RUN_DIR / "screenshots"
|
||||
SCREENSHOTS.mkdir(parents=True, exist_ok=True)
|
||||
LOG = RUN_DIR / "final_script_log.txt"
|
||||
LOG.write_text("") # reset
|
||||
|
||||
def log(step: int, msg: str) -> None:
|
||||
line = f"step {step} action: {msg}\n"
|
||||
LOG.open("a").write(line)
|
||||
print(line, end="")
|
||||
|
||||
async def main():
|
||||
async with async_playwright() as playwright:
|
||||
browser = await playwright.firefox.launch(headless=True)
|
||||
context = await browser.new_context(viewport={"width": 1280, "height": 1800})
|
||||
page = await context.new_page()
|
||||
|
||||
await page.goto("<START_URL>", wait_until="domcontentloaded")
|
||||
await page.screenshot(path=str(SCREENSHOTS / "final_execution_1_open_start_page.png"))
|
||||
log(1, "open start page")
|
||||
|
||||
# ... apply CP1, screenshot, log ...
|
||||
# ... apply CP2, screenshot, log ...
|
||||
|
||||
# End of run: capture the final datum visibly and in the log
|
||||
final_value = "<extracted price / code / winner>"
|
||||
with LOG.open("a") as f:
|
||||
f.write(f"\nFINAL_RESPONSE: {final_value}\n")
|
||||
|
||||
await browser.close()
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
## Inspection commands
|
||||
|
||||
```bash
|
||||
# Latest run tree + log
|
||||
ls -R final_runs/run_<id>
|
||||
cat final_runs/run_<id>/final_script_log.txt
|
||||
|
||||
# Quick file read
|
||||
sed -n '1,220p' final_runs/run_<id>/final_script.py
|
||||
```
|
||||
|
||||
For visual checks, use the `Read` tool on individual PNG files inside
|
||||
`final_runs/run_<id>/screenshots/` rather than calling an external image-QA
|
||||
service.
|
||||
@@ -0,0 +1,107 @@
|
||||
# Workflow
|
||||
|
||||
Detailed expansion of the six-step Webwright loop, adapted for Claude Code.
|
||||
The original loop relied on `webwright.tools.image_qa` for visual QA and
|
||||
`webwright.tools.self_reflection` for the final verdict. Both are replaced
|
||||
here by your native abilities (`Read` on PNG files + reasoning against
|
||||
`plan.md`). No `OPENAI_API_KEY` is required.
|
||||
|
||||
## 1. Plan
|
||||
|
||||
Parse the task into critical points (CPs) and write `WORKSPACE_DIR/plan.md`:
|
||||
|
||||
```markdown
|
||||
# Task
|
||||
<verbatim task description>
|
||||
|
||||
# Critical Points
|
||||
- [ ] CP1: <constraint / filter / sort / selection / required datum>
|
||||
- [ ] CP2: ...
|
||||
```
|
||||
|
||||
Rules for CPs:
|
||||
|
||||
- One CP per independently verifiable requirement.
|
||||
- Numeric, date, quantity, and unit CPs must be exact.
|
||||
- Ranking CPs ("cheapest", "best-selling", "highest-rated", …) must
|
||||
reference the site's actual sort/filter control.
|
||||
- If the task asks for a final datum, make it its own CP
|
||||
(e.g. `CP5: Record the displayed cheapest economy fare`).
|
||||
|
||||
## 2. Explore
|
||||
|
||||
Goal: discover stable selectors, confirm every required filter control
|
||||
exists, and identify how to capture evidence for each CP.
|
||||
|
||||
- Run scratch Playwright scripts (see `playwright_patterns.md`) inside
|
||||
`WORKSPACE_DIR/`. Save scratch PNGs under `WORKSPACE_DIR/screenshots/`
|
||||
(separate from `final_runs/`).
|
||||
- Print URL, title, and `aria_snapshot()` for the region of interest at
|
||||
every step.
|
||||
- Use `Read` on saved PNGs to confirm UI state when ARIA evidence is
|
||||
ambiguous.
|
||||
- If a filter looks unavailable, expand drawers / accordions / mobile
|
||||
filter panels and inspect again before concluding it doesn't exist.
|
||||
- A search-box query never substitutes for a dedicated filter control.
|
||||
|
||||
## 3. Author `final_script.py`
|
||||
|
||||
Create a fresh `final_runs/run_<id>/` (use the next integer above any
|
||||
existing `run_*`) and place `final_script.py` inside it. Instrument per
|
||||
`playwright_patterns.md`:
|
||||
|
||||
- viewport 1280×1800, headless local Firefox, no `full_page`;
|
||||
- one `final_execution_<step>_<action>.png` per CP;
|
||||
- one `step <n> action: <reason and action>` log line per
|
||||
constraint-relevant interaction;
|
||||
- the final datum printed into `final_script_log.txt` at the end.
|
||||
|
||||
Each screenshot should map to a CP from `plan.md` so verification is
|
||||
trivial.
|
||||
|
||||
## 4. Execute
|
||||
|
||||
Run the script once. If it crashes, fix it inside the same run folder and
|
||||
re-execute — but if a partial run already produced screenshots that don't
|
||||
match the fixed flow, delete them so the run folder reflects a single
|
||||
clean execution.
|
||||
|
||||
## 5. Self-verify (replaces `self_reflection`)
|
||||
|
||||
For every CP in `plan.md`:
|
||||
|
||||
1. Identify the screenshot(s) and/or log line that provide evidence.
|
||||
2. `Read` each cited PNG.
|
||||
3. Confirm the evidence is **unambiguous**:
|
||||
- Filter chip / selected state visibly applied (not hidden behind a
|
||||
closed drawer);
|
||||
- Numeric / date values match exactly (not broadened);
|
||||
- Sort applied via the site's control (not implied by result order);
|
||||
- Required submit / search / apply action visibly taken;
|
||||
- Final datum legibly displayed.
|
||||
4. Tick the CP only when the evidence is concrete. Be harsh on partial,
|
||||
occluded, or ambiguous states.
|
||||
|
||||
If any CP fails, diagnose the *specific* issue — wrong filter value,
|
||||
missing control, hidden chip, broadened range, missing confirmation,
|
||||
missing screenshot, etc. Fix `final_script.py`, run it again inside
|
||||
`final_runs/run_<id+1>/`, and re-verify against `plan.md`.
|
||||
|
||||
Empty result sets are acceptable when the correct filters were demonstrably
|
||||
applied.
|
||||
|
||||
## 6. Done
|
||||
|
||||
Stop only when **all** of the following are true:
|
||||
|
||||
1. `plan.md` exists with every CP enumerated as a checklist item.
|
||||
2. `final_runs/run_<id>/final_script.py` ran cleanly from scratch and
|
||||
produced `final_script_log.txt` plus all CP screenshots.
|
||||
3. Every CP is checked off with a cited screenshot and/or log line.
|
||||
4. The final datum (if the task asked for one) is reported to the user
|
||||
verbatim and is also present in `final_script_log.txt`.
|
||||
5. `ls -R final_runs/run_<id>` and `cat final_runs/run_<id>/final_script_log.txt`
|
||||
show the expected artifacts.
|
||||
|
||||
If any of those is false, do not declare done — diagnose, fix, and re-run
|
||||
in a new `run_<id+1>/`.
|
||||
Reference in New Issue
Block a user