wehub-resource-sync
1 line
2.0 KiB
JSON
1 line
2.0 KiB
JSON
{"content": "---\nname: quality-streak-loop\ndescription: Runs realistic scenarios and fixes failures until the suite passes N times in a row with no regressions.\ncategory: evaluation\ninterval: 20m\nstop-condition: The scenario suite passes N consecutive times (e.g. 5) with zero failures.\ncomponents: [agent:performance-testing/test-automator, hook:testing/test-runner, command:testing/test-quality-analyzer]\ntags: [evaluation, reliability, testing, loop]\n---\n\n# Quality Streak Loop\n\n> **Loop Engineering** — don't trust a single green run. Require a *streak* of clean passes to prove stability before stopping.\n\n## 🎯 Goal\nExercise the product with realistic scenarios, fix every failure, and keep going until the suite passes N consecutive times — catching flakiness and intermittent bugs.\n\n## ⏱️ Schedule\nSuggested interval: `20m`.\n\n## ▶️ Run it\n```\n/loop 20m \"Run the realistic scenario suite. Fix any failure you find, including flaky ones. Reset the streak counter on any failure. Continue until the suite passes 5 times in a row.\"\n```\n\n## 🔁 Iteration steps\n1. **Perceive** — run the scenario suite via the `test-runner` hook.\n2. **Reason** — analyze failures (real bug vs flake) with `/test-quality-analyzer`.\n3. **Plan** — fix the failure and harden the test.\n4. **Act** — implement the fix with `test-automator`; reset the streak on any failure.\n5. **Observe** — repeat; only stop after N consecutive clean runs.\n\n## 🛑 Stopping condition\nN consecutive passes (default 5) with zero failures or flakes.\n\n## 🧩 Referenced components\n- `agent:performance-testing/test-automator` — fixes and hardens tests.\n- `hook:testing/test-runner` — auto-runs the suite.\n- `command:testing/test-quality-analyzer` — distinguishes real bugs from flakes.\n\n## 💡 Example\nThe suite passes 4 times then a timing-dependent test flakes; the loop stabilizes it with a deterministic clock and restarts the streak until it hits 5 clean.\n"} |