27 KiB
ECC Pro: Hosted Multi-Repo Agent Security Posture Dashboard
Status: draft design for review. Produced 2026-06-21 by an architecture agent grounded in the existing ecc-agentshield primitives. Proposes the hosted ECC Pro surface; does not implement it. Companion to docs/ECC-PRO-SECURITY-ROADMAP.md (the "next" flagship item).
1. Title, Thesis, and Wedge
ECC Pro is a hosted, authenticated, multi-repo "Sentry for agent security" surface built on top of the existing ecc-agentshield local CLI primitives. AgentShield already does ~30K npm downloads/month with near-zero monetization. The thesis: the continuous and fleet primitives that make a hosted product valuable already exist as local CLI building blocks (evidence packs with bundleDigest integrity, operatorReadback/reviewItems promotion routing, fs.watch drift detection, NDJSON runtime allow/block logging, baseline diffing, and policy promotion gates). The fastest path to MRR is not new science; it is hosting these primitives as authenticated multi-repo and multi-org surfaces and unifying config-scan posture with runtime telemetry over time.
The wedge: Snyk and similar SCA tools are scan-only and have no concept of agent-runtime semantics (no PreToolUse deny decisions, no MCP/hook/agent injection model, no drift-over-time on agent config). Sentry has time-series and alerting but zero security semantics; it does not know what a hardcoded sk-ant- key, a Bash(*) allow rule, or an autoApprove MCP server is. CodeRabbit reviews PR diffs but is point-in-time and has no fleet posture rollup or runtime block-rate trend. ECC Pro is the only surface that charts score trend, drift history, blocked-command rate, and injection-attempt rate across a fleet of repos, anchored on a security-specific rule engine (102 rules across secrets/permissions/hooks/mcp/agents) that nobody else has. AgentShield was featured at the Cerebral Valley x Anthropic Claude Code Hackathon (Feb 2026); the hosted surface is the commercial extension of that featured tooling.
2. Scope: Free Local-First vs Pro Hosted
The free local-first scanner stays the moat. We never paywall the scanner itself; we monetize hosting, history, and multi-repo aggregation. Local-first capability is also what produces the redacted, integrity-checked artifacts the hosted product ingests, so a strong free tier directly grows the funnel.
Free, zero-account, local-only (unchanged, MIT):
agentshield scanand all 102 rules,--format terminal|json|markdown|html|sarif.--fix,agentshield init,--opusdeep analysis (user supplies their ownANTHROPIC_API_KEY).--evidence-pack <dir>,evidence-pack verify|inspect|fleet(local fleet routing stays free).--baseline,--save-baseline,agentshield baseline write,--gate.agentshield runtime install|status|repair, localruntime.ndjsonlogging.agentshield policy init|export|promote, all 6 policy packs (oss,team,enterprise,regulated,high-risk-hooks-mcp,ci-enforcement).- Local
agentshield watch(fs.watch drift, terminal/webhook alerts). - GitHub Action
affaan-m/agentshield@v1(CI scanning, SARIF upload, baseline gate). - MiniClaw local server.
Pro, hosted, account-required (the recurring-revenue surface):
- Persisted history: every scan/baseline/drift/runtime event retained and charted over time (free CLI is point-in-time and stateless on the local box).
- Multi-repo and org rollup: cross-repo posture, fleet
operatorReadbackaggregation, org-level score trend. - Authenticated ingestion endpoints for CI scan results,
runtime.ndjsonstreaming, and watch/drift events. - Hosted dashboard frontend (posture, drift timeline, blocked-command rate, injection-attempt rate, secret-exposure events).
- Hosted alerting and routing: turn
reviewItemsinto assignable tickets, deliver to Slack/Linear/GitHub via the ecc-tools GitHub App. - RBAC, audit log, retention/compliance, SSO (Enterprise).
- Hosted policy promotion gate: org-level promotion approval workflow on top of
policy promotereviewItems.
The hard line: anything that runs against local files and produces a redacted artifact stays free. Anything that stores, aggregates, charts, or routes across repos/time/people is Pro. We never require an account to find a vulnerability; we require one to track a fleet of them over time.
3. Architecture
The hosted backend is a thin, stateless ingestion and query layer over the existing artifact shapes. The CLI/Action/App remain the producers; the backend never re-implements scanning. It receives already-redacted artifacts (the CLI redacts paths/usernames/emails/tokens by default in createRedactor/buildReplacements) and persists summaries plus time-series rollups.
Component diagram (ASCII):
PRODUCERS (free, local-first, already redacted)
+-----------------------+ +------------------------+ +-------------------------+
| GitHub Action | | agentshield watch | | runtime PreToolUse hook |
| (CI scan + evidence | | (fs.watch, diffBaseline,| | (evaluateToolCall -> |
| pack, SARIF, baseline)| | DriftResult, webhook) | | runtime.ndjson) |
+-----------+-----------+ +-----------+------------+ +-----------+-------------+
| | |
| POST evidence-pack | POST drift event | POST/stream ndjson batch
| summary + manifest digest | (DriftResult) | (RuntimeLogEntry[])
v v v
+-----------------------------------------------------------------------------------+
| INGESTION GATEWAY (stateless, authenticated) |
| - API token auth + org/repo identity resolution |
| - schema validation (Zod, reuse SecurityReport / DriftResult / RuntimeLogEntry) |
| - bundleDigest re-verification, idempotency on digest |
| - reject-if-not-redacted guard (manifest.redacted must be true for hosted) |
+-----------------------------------+-----------------------------------------------+
|
+--------------------+--------------------+
v v
+-----------------------------+ +-------------------------------+
| PRIMARY STORE (Postgres) | | TIME-SERIES ROLLUP STORE |
| org, repo, scan, baseline, | | score_trend, drift_history, |
| finding, runtime_event, | rollup job | blocked_cmd_rate, |
| drift_event, policy_eval, |------------->| injection_rate, secret_events |
| evidence_pack, review_item | | (Postgres time buckets or |
+--------------+--------------+ | ClickHouse for high-volume |
| | runtime ndjson) |
| +---------------+----------------+
| |
v v
+-----------------------------------------------------------------------------------+
| QUERY API (authenticated, RBAC-filtered, multi-tenant isolated by org_id) |
+-----------------------------------+-----------------------------------------------+
|
v
+-----------------------------+ +-------------------------------------------+
| DASHBOARD FRONTEND (Next.js) | | ROUTING/ALERTS (ecc-tools GitHub App, |
| posture, trends, drift, fleet| | Slack/Linear) from reviewItems + tickets |
+-----------------------------+ +-------------------------------------------+
Ingestion sources and their existing producers:
- CI scan results: GitHub Action already emits the full
SecurityReportJSON, SARIF, and an evidence pack withmanifest.json(bundleDigest, per-artifactsha256/bytes) plusci-context.json(EvidencePackGitHubContext:repository,sha,runId,workflow,ref,actor). The Action gets a new optional inputecc-pro-ingest-url+ token; on success it POSTs the inspected pack summary (EvidencePackInspectionResult) and the manifest digest. - Runtime telemetry: the PreToolUse hook (
evaluateToolCall->logEvalResult) writesRuntimeLogEntrylines to.agentshield/runtime.ndjson. A smallagentshield runtime shipcommand (Pro) tails and batch-POSTs new NDJSON lines. - Watch/drift events:
startWatcheralready computesDriftResultand callsdispatchAlert. We add awebhookalert target that points at the hosted ingest endpoint; the existingformatWebhookPayloadcarriesnewFindings,resolvedFindings,scoreDelta,isRegression,hasCritical.
Storage choice: Postgres (Supabase) for the relational entities and most rollups; ClickHouse only if runtime NDJSON volume per org makes per-row retention in Postgres uneconomical (runtime events are append-only and high-cardinality, which is the ClickHouse sweet spot). Default MVP is Postgres-only.
4. API Contract
All endpoints are authenticated with an org-scoped API token (header Authorization: Bearer eccp_...). Request/response shapes reuse the real field names from the CLI so the producers do not need a translation layer. Ingestion is idempotent keyed on bundleDigest (scans) or (repo_id, timestamp, tool, decision) hash (runtime).
4.1 Ingest a scan / evidence pack summary
POST /v1/ingest/scan
The body is the existing EvidencePackInspectionResult plus the ci-context summary. The backend never asks for raw evidence; it consumes the already-computed inspection summary so it can re-derive the same rollups the local evidence-pack inspect produces.
Request:
{
"repository": "acme/agent-platform",
"bundleDigest": "sha256:9f2c...e1",
"expectedBundleDigest": "sha256:9f2c...e1",
"generatedAt": "2026-06-21T17:42:00.000Z",
"redacted": true,
"report": {
"score": { "grade": "C", "numericScore": 66 },
"findings": { "total": 29, "critical": 1, "high": 7, "medium": 8, "low": 10, "info": 3 },
"runtimeConfidence": { "active-runtime": 11, "template-example": 14, "project-local-optional": 4 }
},
"policy": { "status": "failed", "policyPack": "enterprise", "violations": 3 },
"baseline": { "status": "regressed", "newFindings": 4, "resolvedFindings": 1, "scoreDelta": -8 },
"supplyChain": { "totalPackages": 22, "riskyPackages": 2, "criticalCount": 0, "highCount": 1 },
"ciContext": {
"provider": "github-actions",
"repository": "acme/agent-platform",
"workflow": "security.yml",
"runId": "1182334455",
"sha": "4c1d9ab"
},
"remediation": { "totalFindings": 29, "autoFixable": 2, "manualReview": 7 }
}
Response:
{
"ok": true,
"scanId": "scan_01J...",
"repoId": "repo_01H...",
"ingestedAt": "2026-06-21T17:42:03.114Z",
"deduped": false,
"rollupsUpdated": ["score_trend", "drift_history", "secret_exposure_events"]
}
Server-side guards: reject with 422 if redacted !== true (hosted tenants must never store unredacted bundles), and reject with 409 deduped echo if bundleDigest already ingested for that repo. If expectedBundleDigest is present and differs from bundleDigest, mark integrity: "mismatch" on the stored scan.
4.2 Ingest runtime telemetry batch
POST /v1/ingest/runtime
Body is an array of the existing RuntimeLogEntry shape from src/runtime/types.ts.
Request:
{
"repository": "acme/agent-platform",
"sessionId": "sess_4f8a",
"entries": [
{ "timestamp": "2026-06-21T17:50:01.002Z", "tool": "Bash", "decision": "block", "reason": "Input matches denied pattern \"rm -rf\"", "durationMs": 2 },
{ "timestamp": "2026-06-21T17:50:02.114Z", "tool": "Read", "decision": "allow", "durationMs": 1 }
]
}
Response:
{ "ok": true, "accepted": 2, "blocked": 1, "allowed": 1, "rollupsUpdated": ["blocked_command_rate"] }
Note: RuntimeLogEntry already carries no raw input payload (only tool, decision, reason, durationMs), so runtime ingestion is safe-by-construction. We keep it that way; the hosted API must not add a raw-input field.
4.3 Ingest a drift event
POST /v1/ingest/drift
Body is the existing DriftResult from src/watch/types.ts (already what formatWebhookPayload emits).
Request:
{
"repository": "acme/agent-platform",
"timestamp": "2026-06-21T18:01:10.000Z",
"newFindings": [ { "id": "secrets-hardcoded-anthropic", "severity": "critical", "category": "secrets", "title": "Hardcoded Anthropic API key", "file": "<target-path>/CLAUDE.md" } ],
"resolvedFindings": [],
"scoreDelta": -25,
"previousScore": 66,
"currentScore": 41,
"isRegression": true,
"hasCritical": true
}
Response:
{ "ok": true, "driftEventId": "drift_01J...", "alertRouted": true }
4.4 Query: org fleet rollup
GET /v1/org/{orgId}/fleet
Response reuses the EvidencePackFleetInspectionResult operatorReadback shape so the dashboard and the existing evidence-pack fleet consumers share one contract:
{
"ok": false,
"requiresAttention": true,
"summary": { "totalPacks": 12, "verifiedPacks": 11, "invalidPacks": 1, "critical": 2, "high": 9, "policyFailures": 3, "baselineRegressions": 2, "riskyPackages": 5 },
"operatorReadback": {
"status": "blocked",
"ready": false,
"requiresApproval": true,
"digest": "sha256:aa17...",
"reviewItemCount": 5,
"blockingItemCount": 2,
"ownerCount": 3,
"owners": ["acme/agent-platform security owner"],
"routesRequiringApproval": ["policy-review", "security-blocker"],
"approvalIds": ["agsr_2b1c8f0d9e7a4c11"],
"nextAction": "Route review items to listed owners and attach approval before promotion."
}
}
4.5 Query: per-repo posture and trend
GET /v1/repo/{repoId}/posture?from=...&to=...&bucket=day
Returns score_trend, latest EvidencePackInspectionResult, latest DriftResult, and runtime rollups for the window.
4.6 Query: review items (routing)
GET /v1/repo/{repoId}/review-items
Returns the existing EvidencePackFleetReviewItem[] (route, severity, priority, approvalId, owner, evidencePaths, beforeState, afterState, reversibleAction, actions, recommendation, and the Linear-friendly ticket.externalId). These map one-to-one to assignable hosted tickets; no new schema needed.
5. Data Model
Persisted relational entities (Postgres). All carry org_id for tenant isolation; all timestamps are ISO-8601 UTC.
org:id,name,github_org_login,plan(team|enterprise),created_at,sso_enabled.repo:id,org_id,full_name(e.g.acme/agent-platform),github_repo_id(fromEvidencePackGitHubContext.repositoryId),default_provider(github-actions|local),created_at.scan:id,repo_id,bundle_digest(unique per repo, idempotency key),generated_at,redacted,grade,numeric_score,score_breakdown(jsonb: secrets/permissions/hooks/mcp/agents),total_findings,critical/high/medium/low/info,provider,ci_sha,ci_run_id,ci_workflow,integrity(ok|mismatch).finding:id,scan_id,finding_key(theFinding.id, e.g.mcp-risky-filesystem),severity,category(FindingCategory),title,file(already redacted to<target-path>form),runtime_confidence(RuntimeConfidence),fingerprint(reusefingerprintFindingso the same finding across scans collapses to one timeline). Never storeevidenceraw for hosted; store only the redactedfileandtitle.baseline:id,repo_id,baseline_timestamp,numeric_score,finding_count,source_scan_id. MirrorsSerializedBaseline(version,timestamp,score,findingswithfingerprint).baseline_comparison:id,repo_id,scan_id,is_regression,new_findings_count,resolved_findings_count,unchanged_count,score_delta,new_critical_count,new_high_count(theBaselineComparisonshape).runtime_event:id,repo_id,session_id,timestamp,tool,decision(allow|block),reason,duration_ms(theRuntimeLogEntryshape; high-volume, candidate for ClickHouse).drift_event:id,repo_id,timestamp,score_delta,previous_score,current_score,is_regression,has_critical,new_findings(jsonb summary),resolved_findings(jsonb summary) (theDriftResultshape).policy_eval:id,scan_id,policy_name,policy_pack(PolicyPack),passed,violation_count,score,min_score,exception_summary(jsonb:total/active/expiringSoon/expiredfromPolicyExceptionSummary). MirrorsPolicyEvaluation.evidence_pack:id,scan_id,bundle_digest,expected_bundle_digest,artifact_count,verified_artifact_count,redacted,generated_at. MirrorsEvidencePackInspectionResult.review_item:id,repo_id,approval_id(theagsr_...id),route(EvidencePackFleetRoute),severity,priority,owner,recommendation,ticket_external_id,status(open|approved|dismissed),assignee. MirrorsEvidencePackFleetReviewItem.
Time-series rollups to chart (materialized from the entities above, bucketed by hour/day/week):
score_trend: per repo and org-aggregatenumeric_scoreandgradeover time (fromscan.numeric_score). The headline chart.drift_history: count and severity ofdrift_eventregressions over time, withscore_deltaband. Answers "is this repo's agent posture decaying?".blocked_command_rate:runtime_eventwheredecision = blockover total, per tool, over time. The "Sentry-style" live signal nobody else has.injection_attempt_rate: count of blocked runtime events whosereasonmatches injection deny patterns, plus scan findings withcategory = injection, over time.secret_exposure_events: timeline offindingrows withcategory = secretsandseverity = critical(e.g.secrets-hardcoded-*), de-duplicated byfingerprint, so a recurring committed key shows as one persistent event until resolved.cross_repo_org_rollup: org-level fold ofscore_trend, openreview_itemcount byroute,policyFailures, andbaselineRegressions(theEvidencePackFleetSummaryfields), feeding theoperatorReadback.statusbadge at org scope.
6. Auth Model
Identity and tenancy:
- Org is the top-level tenant, anchored to a GitHub org login (the ecc-tools GitHub App install scope is the natural onboarding boundary).
reporows are children of exactly oneorg;github_repo_idfromEvidencePackGitHubContext.repositoryIdis the stable external key. - Multi-tenant isolation: every row carries
org_id. On Supabase Postgres, enforce Row Level Security so every query is filtered by the caller'sorg_id; the query API never accepts a client-suppliedorg_idthat is not in the caller's token claims. No cross-org joins exist in any query path.
API tokens:
- Org-scoped ingestion tokens (
eccp_...) are minted per org and optionally per repo. Tokens are hashed at rest (store only a SHA-256 of the token, never the token), shown once on creation. CI uses a repo-scoped token in GitHub Actions secrets; runtime/watch shippers use the same. - Tokens have a
scope(ingest:scan,ingest:runtime,ingest:drift,read) so a CI token cannot read the dashboard API and a read token cannot write.
RBAC tiers (per org):
owner: billing, SSO config, token management, member management, policy promotion approval.admin: token management, review-item assignment, alert routing config.member: view all posture, assign review items to self, comment.viewer: read-only posture and trends (auditor / buyer-review persona).
Prohibited handling (hard requirements, enforced server-side):
- Never store raw secrets. The CLI already redacts paths, usernames, emails, and token-shaped strings by default via
createRedactor/buildReplacements(coverssk-,gh*_,github_pat_,glpat-,npm_,AKIA, JWTeyJ..., Slack tokens, emails, etc.). The ingestion gateway must reject any scan payload wheremanifest.redacted/redactedis nottrue. Preserve redaction end-to-end; the hosted store only ever holds the<redacted-token>/<target-path>/<home>/<user>forms. runtime_eventingestion accepts only theRuntimeLogEntryfields (tool,decision,reason,durationMs); it must not accept raw toolinput. The localToolCall.inputstays local.- Remediation plans and baselines already omit raw evidence and before/after token-shaped strings; preserve that omission in the hosted projection. Findings stored hosted carry redacted
file+title+fingerprintonly, never rawevidence. - Audit log: every token mint/revoke, review-item state change, and policy promotion approval is appended to an immutable per-org audit trail (defense-in-depth, least-privilege, secure-by-default).
7. MVP vs v2 vs v3 (Build Order)
MVP (smallest shippable Pro v1) -- "history + multi-repo posture for CI scans":
- Org/repo model, GitHub App (ecc-tools) install -> org/repo provisioning, org-scoped ingest tokens with RLS isolation.
POST /v1/ingest/scanconsumingEvidencePackInspectionResult+ci-context; persistscan,finding,evidence_pack,policy_eval,baseline_comparison; idempotent onbundleDigest; reject-if-not-redacted guard.- GitHub Action gets
ecc-pro-ingest-url+ token inputs; on scan it POSTs the inspected summary. - Dashboard v1:
score_trendchart, per-repo finding table (severity +runtimeConfidencefilter), org fleet table reusingoperatorReadback.status. - Stripe billing, Team plan ($19/seat/mo per the existing ecc-tools Pro listing), per-org token quota.
This is shippable because it only stitches existing artifacts to storage + a chart. No new scanning logic.
v2 -- "runtime telemetry + drift over time + routing":
6. POST /v1/ingest/runtime + agentshield runtime ship shipper; runtime_event store; blocked_command_rate and injection_attempt_rate charts.
7. POST /v1/ingest/drift + watch webhook target -> hosted; drift_history chart; secret_exposure_events timeline.
8. review_item ingestion + assignable tickets, alert routing to Slack/Linear/GitHub via ecc-tools App, reusing approvalId and ticket.externalId for dedupe.
v3 -- "Enterprise governance":
9. Hosted policy promotion gate: org approval workflow on top of policy promote reviewItems; required approvals before operatorReadback.ready.
10. SSO/SAML, custom retention, audit-log export, per-org data residency; ClickHouse migration for runtime events if volume warrants.
8. Pricing and Packaging Hooks
- Team ($19/seat/mo, matches the current ecc-tools Pro listing): per-seat billing; included repo cap (e.g. 25 repos); 90-day history retention; scan + drift ingestion; Slack/GitHub routing; standard RBAC (owner/admin/member/viewer).
- Enterprise (per-repo or platform-fee, sales-assisted): metered by
repocount rather than seats because security platform value scales with fleet size, not headcount; unlimited seats; SSO/SAML; unlimited retention + audit-log export; hosted policy promotion approval gate;regulated/enterprisepolicy packs with required-approval enforcement; data residency.
Gating levers (what flips Team -> Enterprise): runtime telemetry retention window, number of repos under management, SSO requirement, policy-promotion approval workflow, audit-log export, and routesRequiringApproval enforcement (Enterprise can require that operatorReadback.requiresApproval blocks promotion; Team only surfaces it). Per-seat captures small teams; per-repo captures the platform-team buyer whose value is fleet breadth.
9. Risks and Open Questions
Risks:
- Cannibalization: a too-generous hosted free tier could erode the local moat, or a too-aggressive paywall could stall the 30K/mo funnel. Mitigation: never paywall detection; only paywall persistence/aggregation/routing.
- Redaction trust boundary: the hosted product's entire safety story depends on the CLI redactor being complete. A new token format the regex set misses would be ingested unredacted. Mitigation: reject-if-not-redacted is necessary but not sufficient; add a server-side secondary redaction pass over inbound
title/file/reasonstrings as defense-in-depth, and keepbuildReplacementspatterns under test. - Runtime volume economics:
runtime.ndjsoncan be high-cardinality per active agent; Postgres retention could get expensive. Mitigation: pre-aggregate toblocked_command_raterollups on ingest and retain rawruntime_eventonly for the plan's window (ClickHouse for Enterprise). - Idempotency edge:
bundleDigestexcludesmanifest.jsonandREADME.md(BUNDLE_DIGEST_EXCLUDED_FILES), so two scans with identical findings but differentgeneratedAtproduce the same digest. That is correct for dedupe but means we must key the time-series ongeneratedAt/ci_run_id, not on digest alone.
Open questions:
- Should drift/runtime ingestion from purely local
watch/runtime (no CI,provider: "local") be allowed for Pro, given there is no GitHub-verifiable repo identity? Proposal: allow it but tagprovider: localand require a repo-scoped token bound at mint time to afull_name. - Do we attribute runtime events to a GitHub identity (
ci-context.actor) for per-developer block-rate, or keep them repo-anonymous for privacy? Leaning repo-anonymous by default with opt-in actor attribution. - Is org identity strictly GitHub-org-bound, or do we need a GitHub-independent org for GitLab/local-only users in v2? MVP is GitHub-org-bound via the ecc-tools App.
- For the
injection_attempt_ratechart, do we trust runtimereasonstring matching, or do we need a structuredmatchedRulefield shipped fromEvalResult(which hasmatchedRule) instead of onlyRuntimeLogEntry(which drops it)? Proposal: extend the runtime shipper to includematchedRuleso injection attribution is structured, not string-parsed.
Relevant grounding files (all absolute):
agentshield/src/evidence-pack/index.ts(EvidencePackInspectionResult,EvidencePackFleetOperatorReadback,EvidencePackFleetReviewItem,bundleDigest,BUNDLE_DIGEST_EXCLUDED_FILES,createRedactor,buildReplacements)agentshield/src/runtime/types.ts(RuntimeLogEntry,EvalResult,RuntimePolicy) andagentshield/src/runtime/evaluator.ts(evaluateToolCall,logEvalResult)agentshield/src/watch/types.ts(DriftResult,WatchConfig) andagentshield/src/watch/index.ts(formatWebhookPayload,dispatchAlert)agentshield/src/baseline/types.ts(SerializedBaseline,SerializedFinding,BaselineComparison) andagentshield/src/baseline/index.ts(fingerprintFinding)agentshield/src/policy/types.ts(PolicyEvaluation,PolicyPack,PolicyExceptionSummary)agentshield/src/types.ts(Finding,RuntimeConfidence,FindingCategory,SecurityReport,SecurityScore)agentshield/README.md(GitHub Action inputs/outputs, ecc-tools GitHub App,ecc-agentshieldnpm, ECC Tools Pro $19/seat/mo)