promptfoo--promptfoo/site/docs/model-audit/ci-cd.md

sidebar_label, description, keywords

sidebar_label	description	keywords
CI/CD	Integrate ModelAudit into GitHub Actions, GitLab CI, Jenkins, and CircleCI pipelines for automated ML model security scanning with SARIF output and deployment gates	modelaudit ci cd github actions model scanning gitlab ci model security jenkins model scanning circleci model security automated model scanning sarif integration model security automation production deployment gates scheduled security scans strict mode validation exit codes security pipeline integration github code scanning model scan workflow continuous security ml security automation automated vulnerability scanning

CI/CD Integration

ModelAudit integrates into CI/CD pipelines to automatically scan ML model files for security vulnerabilities before deployment.

Exit Codes

ModelAudit uses specific exit codes for CI/CD automation:

• 0: No security issues found ✅
• 1: Security issues detected (warnings or critical findings) 🟡
• 2: Operational errors or inconclusive scans (file access, installation, timeouts, or no scanned files) 🔴

In CI/CD pipelines, exit code 1 indicates findings that should be reviewed. Only exit code 2 represents actual scan failures.

GitHub Actions

Scan Changed Model Files

Scan model files modified in pull requests:

name: Model Security Scan

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  scan-models:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '24'

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install Promptfoo and ModelAudit
        run: |
          npm install -g promptfoo
          pip install modelaudit

      - name: Get changed model files
        id: changed-files
        run: |
          MODEL_EXTENSIONS='pkl|pickle|dill|pth|pt|ckpt|checkpoint|orbax-checkpoint|h5|hdf5|keras|onnx|pb|tflite|safetensors|gguf|ggml|ggmf|ggjt|ggla|ggsa|bin|engine|plan|msgpack|flax|orbax|jax|joblib|skops|npy|npz|pmml|zip|tar|7z|json|yaml|yml|xml|toml|config|jinja|j2|template|bst|model|ubj'
          CHANGED=$(git diff --name-only --diff-filter=ACM \
            ${{ github.event.pull_request.base.sha }} ${{ github.sha }} | \
            grep -Ei "\.(${MODEL_EXTENSIONS})$" || true)

          if [ -z "$CHANGED" ]; then
            echo "has_changes=false" >> $GITHUB_OUTPUT
          else
            echo "$CHANGED"
            echo "has_changes=true" >> $GITHUB_OUTPUT
            echo "$CHANGED" > changed_files.txt
          fi

      - name: Scan changed models
        if: steps.changed-files.outputs.has_changes == 'true'
        run: |
          mkdir -p scan_results
          while IFS= read -r file; do
            if [ -f "$file" ]; then
              echo "Scanning: $file"
              report_id=$(printf '%s' "$file" | sha256sum | cut -d ' ' -f1)
              promptfoo scan-model "$file" \
                --format json \
                --output "scan_results/${report_id}.json" || true
            fi
          done < changed_files.txt

      - name: Upload scan results
        if: steps.changed-files.outputs.has_changes == 'true'
        uses: actions/upload-artifact@v4
        with:
          name: model-scan-results
          path: scan_results/

      - name: Check for critical issues
        if: steps.changed-files.outputs.has_changes == 'true'
        run: |
          jq -es '
            all(.[];
              type == "object" and
              (.issues | type == "array") and
              (.checks | type == "array") and
              (.has_errors | type == "boolean") and
              (.failed_checks | type == "number") and
              (.has_errors == false) and
              (.failed_checks == 0) and
              ([.checks[]? | select(.status == "failed")] | length == 0) and
              ([.issues[]? | select(.severity == "critical" or .severity == "error")] | length == 0)
            )
          ' scan_results/*.json >/dev/null

          WARNING_COUNT=$(jq -es '[.[] | .issues[]? | select(.severity == "warning")] | length' scan_results/*.json)

          if [ "$WARNING_COUNT" -gt 0 ]; then
            echo "⚠️  Found warnings in $WARNING_COUNT file(s)"
            exit 0
          else
            echo "✅ No security issues detected"
          fi

Upload to GitHub Advanced Security

Use SARIF format to integrate with GitHub Code Scanning:

name: Model Security Scan with SARIF

on:
  push:
    branches: [main]
    paths:
      - '**/*.pkl'
      - '**/*.pth'
      - '**/*.h5'
      - '**/*.onnx'
  pull_request:
    branches: [main]

jobs:
  scan-models:
    runs-on: ubuntu-latest
    permissions:
      security-events: write
      contents: read

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '24'

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install tools
        run: |
          npm install -g promptfoo
          pip install modelaudit

      - name: Scan models directory
        id: scan
        continue-on-error: true
        run: |
          promptfoo scan-model models/ \
            --no-write \
            --format sarif \
            --output modelaudit.sarif

      - name: Upload SARIF to GitHub
        if: always()
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: modelaudit.sarif
          category: model-security

      - name: Fail if scan found issues or errors
        if: steps.scan.outcome == 'failure'
        run: exit 1

Scheduled Security Scans

Run periodic scans on all models:

name: Weekly Model Security Audit

on:
  schedule:
    - cron: '0 2 * * 0' # Sundays at 2 AM UTC
  workflow_dispatch:

jobs:
  comprehensive-scan:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '24'

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install tools
        run: |
          npm install -g promptfoo
          pip install modelaudit[all]

      - name: Comprehensive model scan
        continue-on-error: true
        run: |
          promptfoo scan-model models/ \
            --format json \
            --output scan_results.json \
            --strict \
            --sbom sbom.json

      - name: Generate SBOM
        run: |
          echo "Software Bill of Materials generated: sbom.json"

      - name: Check for critical issues
        id: check-issues
        run: |
          if ! jq -e '
            type == "object" and
            (.issues | type == "array") and
            (.checks | type == "array") and
            (.has_errors | type == "boolean") and
            (.failed_checks | type == "number") and
            (.has_errors == false) and
            (.failed_checks == 0) and
            ([.checks[]? | select(.status == "failed")] | length == 0) and
            ([.issues[]? | select(.severity == "critical" or .severity == "error")] | length == 0)
          ' scan_results.json >/dev/null; then
            echo "critical=true" >> $GITHUB_OUTPUT
            exit 1
          fi

      - name: Create issue if scan fails closed
        if: failure() && steps.check-issues.outputs.critical == 'true'
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const results = JSON.parse(fs.readFileSync('scan_results.json', 'utf8'));

            const issues = Array.isArray(results.issues) ? results.issues : [];
            const criticalIssues = issues
                .filter(i => i.severity === 'critical' || i.severity === 'error')
                .map(i => `- ${i.message}`)
                .join('\n');
            const failedCheckCount = results.failed_checks || 0;
            const findingsSummary = criticalIssues || `- Failed checks: ${failedCheckCount}`;

            await github.rest.issues.create({
              owner: context.repo.owner,
              repo: context.repo.repo,
              title: '🚨 Critical Model Security Issues Detected',
              body: `Weekly security scan found non-clean results:\n\n${findingsSummary}`,
              labels: ['security', 'critical', 'model-audit']
            });

      - name: Upload artifacts
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: weekly-audit-results
          path: |
            scan_results.json
            sbom.json

Strict Mode for Production

Block deployments on any security findings:

name: Production Model Validation

on:
  push:
    branches: [main]
    paths:
      - 'models/production/**'

jobs:
  strict-validation:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '24'

      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install tools
        run: |
          npm install -g promptfoo
          pip install modelaudit[all]

      - name: Strict security scan
        run: |
          promptfoo scan-model models/production/ \
            --strict \
            --format json \
            --output results.json

The scan step fails if strict mode finds warnings or critical issues.

GitLab CI

Example .gitlab-ci.yml:

model-security-scan:
  stage: test
  image: node:24

  before_script:
    - apt-get update && apt-get install -y python3 python3-pip
    - npm install -g promptfoo
    - pip3 install modelaudit

  script:
    - |
      MODEL_EXTENSIONS='pkl|pickle|dill|pth|pt|ckpt|checkpoint|orbax-checkpoint|h5|hdf5|keras|onnx|pb|tflite|safetensors|gguf|ggml|ggmf|ggjt|ggla|ggsa|bin|engine|plan|msgpack|flax|orbax|jax|joblib|skops|npy|npz|pmml|zip|tar|7z|json|yaml|yml|xml|toml|config|jinja|j2|template|bst|model|ubj'
      CHANGED=$(git diff --name-only --diff-filter=ACM $CI_COMMIT_BEFORE_SHA $CI_COMMIT_SHA | \
        grep -Ei "\.(${MODEL_EXTENSIONS})$" || true)

      if [ -n "$CHANGED" ]; then
        mkdir -p scan_results
        printf '%s\n' "$CHANGED" > changed_files.txt
        scan_status=0
        while IFS= read -r file; do
          if [ -f "$file" ]; then
            report_id=$(printf '%s' "$file" | sha256sum | cut -d ' ' -f1)
            file_status=0
            promptfoo scan-model "$file" \
              --format json \
              --output "scan_results/${report_id}.json" || file_status=$?
            if [ "$file_status" -gt "$scan_status" ]; then scan_status=$file_status; fi
          fi
        done < changed_files.txt
        exit "$scan_status"
      else
        echo "No model files changed"
      fi

  artifacts:
    paths:
      - scan_results/
    when: always
    expire_in: 30 days

  only:
    - merge_requests
    - main

Jenkins

Example Jenkinsfile:

pipeline {
    agent any

    stages {
        stage('Setup') {
            steps {
                sh '''
                    npm install -g promptfoo
                    pip install modelaudit
                '''
            }
        }

        stage('Scan Models') {
            steps {
                script {
                    def changed = sh(
                        script: '''
                            MODEL_EXTENSIONS='pkl|pickle|dill|pth|pt|ckpt|checkpoint|orbax-checkpoint|h5|hdf5|keras|onnx|pb|tflite|safetensors|gguf|ggml|ggmf|ggjt|ggla|ggsa|bin|engine|plan|msgpack|flax|orbax|jax|joblib|skops|npy|npz|pmml|zip|tar|7z|json|yaml|yml|xml|toml|config|jinja|j2|template|bst|model|ubj'
                            git diff --name-only HEAD~1 HEAD | \
                            grep -Ei "\\.(${MODEL_EXTENSIONS})$" || true
                        ''',
                        returnStdout: true
                    ).trim()

                    if (changed) {
                        changed.split('\n').each { file ->
                            def reportFile = "scan_${file.bytes.encodeHex().toString()}.json"
                            withEnv(["MODEL_FILE=${file}", "REPORT_FILE=${reportFile}"]) {
                                sh '''
                                    promptfoo scan-model "$MODEL_FILE" \
                                      --format json \
                                      --output "$REPORT_FILE"
                                '''
                            }
                        }

                        // Check for critical issues
                        def critical = sh(
                            script: '''
                                jq -es '
                                  all(.[];
                                    type == "object" and
                                    (.issues | type == "array") and
                                    (.checks | type == "array") and
                                    (.has_errors | type == "boolean") and
                                    (.failed_checks | type == "number") and
                                    (.has_errors == false) and
                                    (.failed_checks == 0) and
                                    ([.checks[]? | select(.status == "failed")] | length == 0) and
                                    ([.issues[]? | select(.severity == "critical" or .severity == "error")] | length == 0)
                                  )
                                ' scan_*.json >/dev/null
                            ''',
                            returnStatus: true
                        )

                        if (critical != 0) {
                            error("Found model scan failures or critical security issues")
                        }
                    }
                }
            }
        }
    }

    post {
        always {
            archiveArtifacts artifacts: 'scan_*.json', allowEmptyArchive: true
        }
    }
}

CircleCI

Example .circleci/config.yml:

version: 2.1

jobs:
  model-scan:
    docker:
      - image: cimg/node:lts

    steps:
      - checkout

      - run:
          name: Install tools
          command: |
            npm install -g promptfoo
            pip install modelaudit

      - run:
          name: Scan changed models
          command: |
            MODEL_EXTENSIONS='pkl|pickle|dill|pth|pt|ckpt|checkpoint|orbax-checkpoint|h5|hdf5|keras|onnx|pb|tflite|safetensors|gguf|ggml|ggmf|ggjt|ggla|ggsa|bin|engine|plan|msgpack|flax|orbax|jax|joblib|skops|npy|npz|pmml|zip|tar|7z|json|yaml|yml|xml|toml|config|jinja|j2|template|bst|model|ubj'
            CHANGED=$(git diff --name-only origin/main...HEAD | \
              grep -Ei "\.(${MODEL_EXTENSIONS})$" || true)

            if [ -n "$CHANGED" ]; then
              echo "$CHANGED" | while read -r file; do
                if [ -f "$file" ]; then
                  promptfoo scan-model "$file" --format json >> scan_results.json
                fi
              done
            fi

      - store_artifacts:
          path: scan_results.json
          destination: model-scan-results

workflows:
  security:
    jobs:
      - model-scan:
          filters:
            branches:
              only:
                - main
                - develop

Best Practices

Scan Strategy

• Pull Requests: Scan only changed files for fast feedback
• Main Branch: Run comprehensive scans on merge
• Scheduled Scans: Weekly or daily full audits
• Production Gate: Use --strict mode to block deployments

Performance

# Cache dependencies
- name: Cache npm and pip
  uses: actions/cache@v3
  with:
    path: |
      ~/.npm
      ~/.cache/pip
    key: ${{ runner.os }}-deps-${{ hashFiles('**/package-lock.json', '**/requirements.txt') }}

# Parallel scanning
- name: Parallel scan
  run: |
    find models/ -name "*.pkl" -print0 | \
      xargs -0 -P 4 -I {} promptfoo scan-model {} --format json --output {}.json

Security

• Store credentials as encrypted secrets
• Use read-only tokens when possible
• Rotate credentials regularly
• Audit access to scan results

Timeout Configuration

For large models (8GB+):

- name: Scan large model
  run: |
    promptfoo scan-model large_model.bin \
      --timeout 1800 \
      --verbose \
      --format json \
      --output results.json

Retry Logic

- name: Scan with retry
  uses: nick-fields/retry@v2
  with:
    timeout_minutes: 10
    max_attempts: 3
    command: promptfoo scan-model models/ --format json --output results.json

Notifications

- name: Notify on critical issues
  if: failure()
  uses: slackapi/slack-github-action@v1
  with:
    payload: |
      {
        "text": "🚨 Critical model security issues detected!",
        "blocks": [
          {
            "type": "section",
            "text": {
              "type": "mrkdwn",
              "text": "*Model Security Scan Failed*\nRepository: ${{ github.repository }}\n<${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Details>"
            }
          }
        ]
      }
  env:
    SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

Troubleshooting

Verbose Output

promptfoo scan-model models/ --verbose

File Size Limits

# Set maximum file size
promptfoo scan-model models/ --max-size 1GB

Dry Run

# Preview scan without processing
promptfoo scan-model models/ --dry-run

Next Steps

• Scanner Reference - Security checks performed
• Advanced Usage - Cloud storage, authentication, remote sources
• Overview - Getting started with ModelAudit