wehub-resource-sync
d48cda4081
CI / Test (ubuntu-latest, Node 18.x, bun) (push) Failing after 15m1s
CI / Test (ubuntu-latest, Node 18.x, npm) (push) Failing after 15m1s
CI / Test (ubuntu-latest, Node 18.x, pnpm) (push) Failing after 15m1s
CI / Test (ubuntu-latest, Node 18.x, yarn) (push) Failing after 15m1s
CI / Test (ubuntu-latest, Node 20.x, bun) (push) Failing after 17m13s
CI / Test (ubuntu-latest, Node 20.x, npm) (push) Failing after 18m42s
CI / Test (ubuntu-latest, Node 20.x, pnpm) (push) Failing after 15m0s
CI / Test (ubuntu-latest, Node 20.x, yarn) (push) Failing after 49m44s
CI / Test (ubuntu-latest, Node 22.x, bun) (push) Failing after 51m55s
CI / Test (ubuntu-latest, Node 22.x, pnpm) (push) Failing after 21m57s
CI / Test (ubuntu-latest, Node 22.x, npm) (push) Failing after 37m39s
CI / Test (ubuntu-latest, Node 22.x, yarn) (push) Failing after 34m7s
CI / Validate Components (push) Failing after 37m15s
CI / Python Tests (push) Failing after 10m1s
CI / Security Scan (push) Failing after 10m1s
CI / Lint (push) Failing after 17m12s
CI / Coverage (push) Failing after 20m19s
CI / Test (macos-latest, Node 18.x, bun) (push) Has been cancelled
CI / Test (macos-latest, Node 18.x, npm) (push) Has been cancelled
CI / Test (macos-latest, Node 18.x, pnpm) (push) Has been cancelled
CI / Test (macos-latest, Node 18.x, yarn) (push) Has been cancelled
CI / Test (windows-latest, Node 18.x, npm) (push) Has been cancelled
CI / Test (windows-latest, Node 18.x, pnpm) (push) Has been cancelled
CI / Test (windows-latest, Node 18.x, yarn) (push) Has been cancelled
CI / Test (macos-latest, Node 20.x, bun) (push) Has been cancelled
CI / Test (macos-latest, Node 20.x, npm) (push) Has been cancelled
CI / Test (macos-latest, Node 20.x, pnpm) (push) Has been cancelled
CI / Test (macos-latest, Node 20.x, yarn) (push) Has been cancelled
CI / Test (windows-latest, Node 20.x, npm) (push) Has been cancelled
CI / Test (windows-latest, Node 20.x, pnpm) (push) Has been cancelled
CI / Test (windows-latest, Node 20.x, yarn) (push) Has been cancelled
CI / Test (macos-latest, Node 22.x, bun) (push) Has been cancelled
CI / Test (macos-latest, Node 22.x, npm) (push) Has been cancelled
CI / Test (macos-latest, Node 22.x, pnpm) (push) Has been cancelled
CI / Test (macos-latest, Node 22.x, yarn) (push) Has been cancelled
CI / Test (windows-latest, Node 22.x, npm) (push) Has been cancelled
CI / Test (windows-latest, Node 22.x, pnpm) (push) Has been cancelled
CI / Test (windows-latest, Node 22.x, yarn) (push) Has been cancelled
121 lines
2.3 KiB
Markdown
121 lines
2.3 KiB
Markdown
# Comando Eval
|
|
|
|
Gerencie o fluxo de desenvolvimento orientado por evals.
|
|
|
|
## Uso
|
|
|
|
`/eval [define|check|report|list] [feature-name]`
|
|
|
|
## Definir Evals
|
|
|
|
`/eval define feature-name`
|
|
|
|
Crie uma nova definição de eval:
|
|
|
|
1. Crie `.claude/evals/feature-name.md` com o template:
|
|
|
|
```markdown
|
|
## EVAL: feature-name
|
|
Created: $(date)
|
|
|
|
### Evals de Capacidade
|
|
- [ ] [Descrição da capacidade 1]
|
|
- [ ] [Descrição da capacidade 2]
|
|
|
|
### Evals de Regressão
|
|
- [ ] [Comportamento existente 1 ainda funciona]
|
|
- [ ] [Comportamento existente 2 ainda funciona]
|
|
|
|
### Critérios de Sucesso
|
|
- pass@3 > 90% para evals de capacidade
|
|
- pass^3 = 100% para evals de regressão
|
|
```
|
|
|
|
2. Peça ao usuário para preencher os critérios específicos
|
|
|
|
## Verificar Evals
|
|
|
|
`/eval check feature-name`
|
|
|
|
Rode evals para uma feature:
|
|
|
|
1. Leia a definição de eval em `.claude/evals/feature-name.md`
|
|
2. Para cada eval de capability:
|
|
- Tente verificar o critério
|
|
- Registre PASS/FAIL
|
|
- Salve tentativa em `.claude/evals/feature-name.log`
|
|
3. Para cada eval de regressão:
|
|
- Rode os testes relevantes
|
|
- Compare com baseline
|
|
- Registre PASS/FAIL
|
|
4. Reporte status atual:
|
|
|
|
```
|
|
EVAL CHECK: feature-name
|
|
========================
|
|
Capability: X/Y passing
|
|
Regression: X/Y passing
|
|
Status: IN PROGRESS / READY
|
|
```
|
|
|
|
## Relatório de Evals
|
|
|
|
`/eval report feature-name`
|
|
|
|
Gere relatório completo de eval:
|
|
|
|
```
|
|
EVAL REPORT: feature-name
|
|
=========================
|
|
Generated: $(date)
|
|
|
|
CAPABILITY EVALS
|
|
----------------
|
|
[eval-1]: PASS (pass@1)
|
|
[eval-2]: PASS (pass@2) - required retry
|
|
[eval-3]: FAIL - see notes
|
|
|
|
REGRESSION EVALS
|
|
----------------
|
|
[test-1]: PASS
|
|
[test-2]: PASS
|
|
[test-3]: PASS
|
|
|
|
METRICS
|
|
-------
|
|
Capability pass@1: 67%
|
|
Capability pass@3: 100%
|
|
Regression pass^3: 100%
|
|
|
|
NOTES
|
|
-----
|
|
[Any issues, edge cases, or observations]
|
|
|
|
RECOMMENDATION
|
|
--------------
|
|
[SHIP / NEEDS WORK / BLOCKED]
|
|
```
|
|
|
|
## Listar Evals
|
|
|
|
`/eval list`
|
|
|
|
Mostre todas as definições de eval:
|
|
|
|
```
|
|
EVAL DEFINITIONS
|
|
================
|
|
feature-auth [3/5 passing] IN PROGRESS
|
|
feature-search [5/5 passing] READY
|
|
feature-export [0/4 passing] NOT STARTED
|
|
```
|
|
|
|
## Argumentos
|
|
|
|
$ARGUMENTS:
|
|
- `define <name>` - Criar nova definição de eval
|
|
- `check <name>` - Rodar e verificar evals
|
|
- `report <name>` - Gerar relatório completo
|
|
- `list` - Mostrar todos os evals
|
|
- `clean` - Remover logs antigos de eval (mantém as últimas 10 execuções)
|