nvidia--skillspector/docs/SC4-osv-live-vulnerability-lookups.md

SC4: Live Vulnerability Lookups via OSV.dev

Author: Nraghavan | Date: 2026-03-17 | Status: Implemented
Component: static_patterns_supply_chain.py (SC4 rule), osv_client.py

---

1. Background

The SC4 rule in skillspector's supply-chain analyzer flags dependencies with known CVEs. Previously this relied on two manually curated lists hardcoded in static_patterns_supply_chain.py:

• _KNOWN_VULNERABLE_PACKAGES — 15 Python (PyPI) entries
• _KNOWN_VULNERABLE_NPM — 9 npm entries

Problems with the static approach:

Issue	Impact
Staleness	24 entries vs. tens of thousands of published advisories. New CVEs are disclosed daily and the list was immediately out of date.
Manual maintenance	Every update required a code change, review, and release. No one owned the update cadence.
Incomplete coverage	High-profile packages only. A skill depending on a vulnerable transitive dependency not in the list would pass undetected.
Version logic was fragile	The custom _version_lt() comparator did simple numeric-tuple comparison and mishandled pre-release tags, date-based versions (e.g. certifi 2022.12.07), and epoch-prefixed versions.

SC5 (abandoned packages) and SC6 (typosquatting / popular-package lists) are not affected — those sets change infrequently and remain static.

---

2. Solution — OSV.dev API

OSV.dev is Google's open, free vulnerability database. It aggregates advisories from PyPI (via the PyPA Advisory Database), the GitHub Advisory Database, NVD, and ecosystem-specific sources.

Why OSV.dev over alternatives

Criteria	OSV.dev	PyPI JSON API	GitHub Advisory DB	pip-audit (lib)
Covers PyPI	Yes	Yes	Yes	Yes
Covers npm	Yes	No	Yes	No
Auth required	No	No	Yes (token)	No
Rate limits	None	Undocumented	Yes	N/A
Batch queries	Yes	No	Limited	No
New dependency needed	No (httpx)	No	No	Yes
Authoritative data	Yes — PyPI + GHSA + NVD	PyPI only	GHSA + NVD	Delegates to PyPI/OSV

API shape (batch endpoint)

POST https://api.osv.dev/v1/querybatch

{
  "queries": [
    {"package": {"name": "jinja2",   "ecosystem": "PyPI"}, "version": "2.4.1"},
    {"package": {"name": "requests", "ecosystem": "PyPI"}, "version": "2.25.0"},
    {"package": {"name": "lodash",   "ecosystem": "npm"},  "version": "4.17.20"}
  ]
}

Response returns, per query, a list of matching vulnerability IDs (GHSA, PYSEC, CVE aliases). A follow-up GET /v1/vulns/{id} call retrieves severity, summary, and fix versions for the finding message.

OSV handles all version-range matching server-side using ecosystem-aware semver/PEP 440 logic, eliminating the fragile _version_lt() comparator.

---

3. Implementation

3.1 Architecture

_analyze_dependencies(content, file_path)
  ├── _extract_packages_from_requirements() / _extract_packages_from_package_json()
  │         (unchanged — returns list of (name, version, line_num))
  │
  ├── SC4: _sc4_from_osv(packages, ecosystem)
  │         osv_client.query_batch()  →  map vulns back to packages  →  emit SC4 findings
  │         On empty results / failure  →  _sc4_from_fallback() using static list
  │
  ├── SC5: _ABANDONED_PACKAGES lookup                  (unchanged)
  └── SC6: _is_typosquat() against popular sets        (unchanged)

3.2 Key files

File	Purpose
src/skillspector/nodes/analyzers/osv_client.py	OSV.dev batch API client — query_batch(), VulnResult dataclass, in-memory cache, severity parsing
src/skillspector/nodes/analyzers/static_patterns_supply_chain.py	Refactored SC4 with _sc4_from_osv() and _sc4_from_fallback()
tests/unit/test_osv_client.py	13 tests covering severity parsing, batch queries, cache behavior, network failures
tests/unit/test_patterns_new.py	Updated SC4-SC6 tests with OSV mocking; 7 new SC4 test cases

3.3 Design decisions

Decision	Choice	Rationale
Sync vs. async	Synchronous httpx.Client	The analyzer pipeline is sync. A single batch call completes in <500 ms for typical dependency files.
Caching	In-memory dict with 1-hour TTL, keyed on (name, version, ecosystem)	Prevents redundant API calls when multiple skills share dependencies. Skillspector runs are short-lived CLI invocations, so memory is not a concern.
Graceful degradation	On timeout/network error, fall back to the static list (_FALLBACK_VULNERABLE_*)	Ensures the tool works in air-gapped or offline environments. A warning is logged when falling back.
Finding detail level	Batch query returns vuln IDs; detail fetched via GET /v1/vulns/{id} for up to 10 vulns per package	Keeps latency low — the 10-vulnerability cap limits the number of sequential detail fetches per package. The implementation selects the first 10 vulnerability IDs as returned by the OSV batch API (_fetch_vuln_details(vuln_ids[:10])) with no severity-based sorting or prioritisation. When more than 10 IDs are returned, the OSV client logs a warning indicating the total count and that only the first 10 will be processed, so users are alerted to the truncation. Finding messages include OSV/GHSA/CVE IDs with summaries.
Confidence mapping	Map OSV severity → confidence: CRITICAL=0.9, HIGH=0.8, MEDIUM=0.7, LOW=0.6	Replaces the per-entry hardcoded confidence values with a systematic mapping.
Timeout	10 s connect + read	Generous for a single POST. If exceeded, fallback activates gracefully.
Severity aggregation	When multiple advisories affect one package, the worst severity is used for the finding	A single SC4 finding is emitted per package with a count and summary of all advisories.

3.4 What stays static

• _ABANDONED_PACKAGES (SC5) — no API exists for "abandoned" status.
• _POPULAR_PYPI / _POPULAR_NPM (SC6) — stable lists for typosquatting heuristic.
• _FALLBACK_VULNERABLE_PYPI / _FALLBACK_VULNERABLE_NPM — renamed from original lists, kept as offline safety net.

3.5 What was removed

• Direct iteration over hardcoded CVE tuples in the SC4 hot path — replaced by _sc4_from_osv().
• The _version_lt() comparator is retained only for fallback mode; OSV handles version comparison server-side in the primary path.

---

4. Risks & Mitigations

Risk	Likelihood	Mitigation
OSV.dev API downtime	Low (Google-hosted, high uptime)	Fallback to static list + warning log
Latency increase (~200-500 ms per scan)	Certain but minor	Batch queries minimize round-trips; caching eliminates repeat calls
False positives from OSV (disputed/withdrawn advisories)	Low	OSV filters withdrawn entries; a suppression list can be added if needed
Breaking API changes	Very low (versioned API, stable since 2021)	Pinned to /v1/ endpoints
Air-gapped / firewalled environments	Medium	Static fallback ensures functionality; documented that live mode needs outbound HTTPS to api.osv.dev

---

5. Validation

• 245 tests pass across unit and analyzer test suites (0 regressions).
• 13 new OSV client tests cover severity parsing, batch queries, cache hits, network failures, and npm ecosystem support.
• 7 new SC4 integration tests verify OSV-driven findings, fallback behavior, multi-advisory aggregation, and severity mapping.
• No new dependencies — uses the existing httpx>=0.28.0 dependency.