chore: import upstream snapshot with attribution
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:24:08 +08:00
commit 0d3cb498a3
5438 changed files with 1316560 additions and 0 deletions
@@ -0,0 +1,43 @@
# config-websockets/basic (Websockets Provider Example)
You can run this example with:
```bash
npx promptfoo@latest init --example config-websockets/basic
cd config-websockets/basic
```
This example shows how to connect promptfoo to a WebSocket-based LLM service.
## Prerequisites
- Node.js ^20.20.0 or >=22.22.0 (Node.js 20 support ends July 30, 2026; Node.js 24 LTS recommended)
- API keys for LLM providers set as environment variables:
- `OPENAI_API_KEY` - Get from [OpenAI API keys page](https://platform.openai.com/api-keys)
- `ANTHROPIC_API_KEY` - Get from [Anthropic Console](https://console.anthropic.com/) (optional)
## Quick Start
1. Start the test server (simulates a WebSocket LLM service):
```bash
cd test-server
npm install
node server.js
```
2. In a new terminal, run the evaluation:
```bash
promptfoo eval
```
## Expected Results
This example will:
- Start a mock WebSocket server that simulates an LLM service
- Connect promptfoo to the WebSocket server using the custom provider
- Run evaluations through the WebSocket connection
- Demonstrate how to integrate promptfoo with custom WebSocket-based APIs
- Save results that can be viewed with `promptfoo view`
@@ -0,0 +1,26 @@
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: 'Websocket test'
prompts:
- 'Rephrase this in {{language}}: {{body}}'
- 'Translate this to conversational {{language}}: {{body}}'
providers:
- id: 'ws://localhost:4000'
config:
messageTemplate: '{"prompt": {{prompt | dump}}, "model": {{model | dump}}}'
transformResponse: 'data.output'
tests:
- vars:
language: French
body: Hello world
- vars:
language: French
body: I'm hungry
- vars:
language: Pirate
body: Hello world
- vars:
language: Pirate
body: I'm hungry
@@ -0,0 +1,15 @@
{
"name": "websockets",
"version": "1.0.0",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "MIT",
"private": true,
"description": "",
"dependencies": {
"ws": "^8.19.0"
}
}
@@ -0,0 +1,40 @@
const WebSocket = require('ws');
const http = require('http');
const server = http.createServer();
const wss = new WebSocket.Server({ server });
wss.on('connection', (ws) => {
console.log('Client connected');
ws.on('message', (message) => {
console.log('Received:', message.toString());
try {
const data = JSON.parse(message);
// Simulate some processing time
setTimeout(() => {
const response = {
output: `Processed: ${data.prompt} at ${new Date().toISOString()}`,
model: data.model || 'default',
timestamp: new Date().toISOString(),
};
ws.send(JSON.stringify(response));
}, 1000);
} catch (error) {
console.error('Error processing message:', error);
ws.send(JSON.stringify({ error: 'Invalid JSON' }));
}
});
ws.on('close', () => {
console.log('Client disconnected');
});
});
const PORT = process.env.PORT || 4000;
server.listen(PORT, () => {
console.log(`WebSocket server is running on ws://localhost:${PORT}`);
});