chore: import upstream snapshot with attribution
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,27 @@
|
||||
# compare-agentic-sdks/test-codebase (Test Codebase for Agentic SDK Comparison)
|
||||
|
||||
This is an intentionally vulnerable codebase designed to test AI coding assistants' ability to:
|
||||
|
||||
1. **Identify security vulnerabilities**
|
||||
2. **Suggest secure alternatives**
|
||||
3. **Understand cross-file dependencies**
|
||||
4. **Generate production-ready fixes**
|
||||
|
||||
## Known Issues
|
||||
|
||||
### user_service.py
|
||||
|
||||
- MD5 password hashing (insecure)
|
||||
- Timing attack vulnerability in authentication
|
||||
- Predictable session tokens
|
||||
- Returns password hashes in `get_user_data()`
|
||||
- Doesn't invalidate sessions on user deletion
|
||||
|
||||
### payment_processor.py
|
||||
|
||||
- Float for currency (precision issues)
|
||||
- Stores CVV (PCI-DSS violation)
|
||||
- Stores full card numbers
|
||||
- Logs sensitive data
|
||||
- No input validation
|
||||
- Float arithmetic for money calculations
|
||||
@@ -0,0 +1,52 @@
|
||||
"""Payment processing module with security issues.
|
||||
|
||||
SECURITY NOTE: This file contains intentional security vulnerabilities for
|
||||
testing purposes. It is used to evaluate security scanning capabilities of
|
||||
agentic code analysis tools. Do not use in production.
|
||||
|
||||
codeql[py/clear-text-logging-sensitive-data]: Intentional vulnerability for testing
|
||||
codeql[py/weak-cryptographic-algorithm]: Intentional vulnerability for testing
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
|
||||
class PaymentProcessor:
|
||||
def __init__(self):
|
||||
self.transactions = []
|
||||
|
||||
def process_payment(self, amount: float, card_number: str, cvv: str) -> dict:
|
||||
"""Process a payment transaction."""
|
||||
# BUG: Using float for currency (precision issues)
|
||||
# BUG: Storing CVV (PCI-DSS violation)
|
||||
# BUG: No input validation
|
||||
|
||||
transaction = {
|
||||
"amount": amount,
|
||||
"card": card_number, # BUG: Storing full card number
|
||||
"cvv": cvv,
|
||||
"status": "pending",
|
||||
}
|
||||
|
||||
# BUG: No sanitization before logging
|
||||
# codeql[py/clear-text-logging-sensitive-data]: Intentional vulnerability for testing
|
||||
print(f"Processing payment: {json.dumps(transaction)}")
|
||||
|
||||
self.transactions.append(transaction)
|
||||
transaction["status"] = "completed"
|
||||
|
||||
return transaction
|
||||
|
||||
def get_transaction_history(self, card_number: str) -> list:
|
||||
"""Get all transactions for a card."""
|
||||
# BUG: SQL injection vulnerability if this were using SQL
|
||||
# BUG: Returns sensitive data including CVV
|
||||
return [t for t in self.transactions if t["card"] == card_number]
|
||||
|
||||
def calculate_total(self, transactions: list) -> float:
|
||||
"""Calculate total amount from transactions."""
|
||||
# BUG: Float arithmetic precision issues
|
||||
total = 0.0
|
||||
for t in transactions:
|
||||
total += t["amount"]
|
||||
return total
|
||||
@@ -0,0 +1,61 @@
|
||||
"""User service with intentional bugs for testing.
|
||||
|
||||
SECURITY NOTE: This file contains intentional security vulnerabilities for
|
||||
testing purposes. It is used to evaluate security scanning capabilities of
|
||||
agentic code analysis tools. Do not use in production.
|
||||
|
||||
codeql[py/weak-cryptographic-algorithm]: Intentional vulnerability for testing
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
from typing import Dict, List, Optional
|
||||
|
||||
|
||||
class UserService:
|
||||
def __init__(self):
|
||||
self.users: Dict[str, Dict] = {}
|
||||
self.sessions: List[str] = []
|
||||
|
||||
def create_user(self, username: str, password: str, email: str) -> bool:
|
||||
"""Create a new user account."""
|
||||
if username in self.users:
|
||||
return False
|
||||
|
||||
# BUG: Using MD5 for password hashing (insecure)
|
||||
# codeql[py/weak-cryptographic-algorithm]: Intentional vulnerability for testing
|
||||
password_hash = hashlib.md5(password.encode()).hexdigest()
|
||||
|
||||
self.users[username] = {
|
||||
"email": email,
|
||||
"password": password_hash,
|
||||
"active": True,
|
||||
}
|
||||
return True
|
||||
|
||||
def authenticate(self, username: str, password: str) -> Optional[str]:
|
||||
"""Authenticate user and return session token."""
|
||||
if username not in self.users:
|
||||
return None
|
||||
|
||||
# BUG: Timing attack vulnerability - should use constant-time comparison
|
||||
# codeql[py/weak-cryptographic-algorithm]: Intentional vulnerability for testing
|
||||
password_hash = hashlib.md5(password.encode()).hexdigest()
|
||||
if self.users[username]["password"] == password_hash:
|
||||
# BUG: Predictable session token
|
||||
session_token = f"{username}_{len(self.sessions)}"
|
||||
self.sessions.append(session_token)
|
||||
return session_token
|
||||
return None
|
||||
|
||||
def get_user_data(self, username: str) -> Optional[Dict]:
|
||||
"""Get user data including password hash (SECURITY ISSUE)."""
|
||||
# BUG: Returns password hash to caller
|
||||
return self.users.get(username)
|
||||
|
||||
def delete_user(self, username: str) -> bool:
|
||||
"""Delete a user account."""
|
||||
if username in self.users:
|
||||
del self.users[username]
|
||||
# BUG: Doesn't invalidate user's sessions
|
||||
return True
|
||||
return False
|
||||
Reference in New Issue
Block a user