wehub-resource-sync
2114ccd278
CI / Lint & Test (Python 3.13) (push) Failing after 2s
CI / Lint & Test (Python 3.14) (push) Failing after 1s
CI / Lint & Test (Python 3.12) (push) Failing after 2s
CI / DCO Check (push) Has been skipped
Scorecard supply-chain security / Scorecard analysis (push) Failing after 2s
22 lines
699 B
Markdown
22 lines
699 B
Markdown
# Eval Dataset Scanning
|
|
|
|
SkillSpector treats authored eval datasets as test-case data, not installed
|
|
skill logic.
|
|
|
|
The static pattern analyzers skip these dataset files:
|
|
|
|
- `evals/evals.json`
|
|
- `evals/evals.jsonl`
|
|
- `evals/evals.yaml`
|
|
- `evals/evals.yml`
|
|
- `eval/dataset.json`
|
|
- `eval/dataset.jsonl`
|
|
- `eval/dataset.yaml`
|
|
- `eval/dataset.yml`
|
|
|
|
This applies to both the agentskills.io format and the legacy flat ACES format.
|
|
Security analysis still covers executable skill code, instructions, scripts,
|
|
dependencies, MCP metadata, and other install-time surfaces. Eval prompts,
|
|
expected outputs, assertions, and ground-truth strings are not treated as code
|
|
accessing credentials or exfiltrating data.
|