wehub-resource-sync
534bb94eea
Test Migrations / Migrations (SQLite) (push) Has been cancelled
Build Dev Image / build-dev-image (push) Has been cancelled
Check i18n Keys / Check i18n Key Consistency (push) Has been cancelled
Lint / Ruff Lint & Format (push) Has been cancelled
Lint / Frontend Lint (push) Has been cancelled
Test Migrations / Migrations (PostgreSQL) (push) Has been cancelled
2.0 KiB
2.0 KiB
Agent Browser QA Principles
This document fixes the direction of LangBot agent testing so the project does not drift into a backend API smoke-test framework.
Primary Goal
langbot-skills should help an agent behave like a QA engineer using the product, not like a backend curl script.
The primary path is:
developer intent -> lbs test plan -> agent controls browser -> UI result + console + logs -> report/assets
Rules
- Browser/UI interaction is the source of truth for product QA cases.
- A backend API or curl response is never enough to mark a UI case passed.
- API/curl/log checks are allowed as diagnostics after a UI path is attempted or when debugging environment readiness.
- A case passes only when the user-visible UI result is correct.
- The agent should inspect browser console/network output when available.
- If screenshot or vision capability is available, the agent should check for blank pages, overlap, hidden actions, broken layout, and error toasts.
- If no visual model is available, use DOM/accessibility snapshots and console output instead.
- New stable UI paths should be added as
cases/*.yaml. - New recurring failure modes should be added as
troubleshooting/*.yaml. - Secrets, tokens, API keys, and localStorage token values must never be printed.
Command Semantics
lbs manages assets and produces plans. It does not replace the agent's browser-control ability.
bin/lbs test plan pipeline-debug-chat
This command outputs:
- environment variables to use
- required skills
- browser steps
- UI/console/visual/log checks
- diagnostic options
- related troubleshooting patterns
- report template
The active agent then executes the plan with Computer Use, Playwright MCP, or another available browser-control tool.
Diagnostics
Diagnostics can include:
bin/lbs env doctor- browser console/network inspection
- backend logs
- targeted API/curl checks
Diagnostics answer "where did it fail?" They do not replace "did the user-visible UI work?"