docs: preserve upstream English README
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Shell Format Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Shell Format Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
This commit is contained in:
+101
@@ -0,0 +1,101 @@
|
||||
# Promptfoo: LLM evals & red teaming
|
||||
|
||||
<p align="center">
|
||||
<a href="https://npmjs.com/package/promptfoo"><img src="https://img.shields.io/npm/v/promptfoo" alt="npm"></a>
|
||||
<a href="https://npmjs.com/package/promptfoo"><img src="https://img.shields.io/npm/dm/promptfoo" alt="npm"></a>
|
||||
<a href="https://github.com/promptfoo/promptfoo/actions/workflows/main.yml"><img src="https://img.shields.io/github/actions/workflow/status/promptfoo/promptfoo/main.yml" alt="GitHub Workflow Status"></a>
|
||||
<a href="https://github.com/promptfoo/promptfoo/blob/main/LICENSE"><img src="https://img.shields.io/github/license/promptfoo/promptfoo" alt="MIT license"></a>
|
||||
<a href="https://discord.gg/promptfoo"><img src="https://img.shields.io/discord/1146610656779440188?logo=discord&label=promptfoo" alt="Discord"></a>
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<code>promptfoo</code> is a CLI and library for evaluating and red-teaming LLM apps. Stop the trial-and-error approach - start shipping secure, reliable AI apps.
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://www.promptfoo.dev">Website</a> ·
|
||||
<a href="https://www.promptfoo.dev/docs/getting-started/">Getting Started</a> ·
|
||||
<a href="https://www.promptfoo.dev/docs/red-team/">Red Teaming</a> ·
|
||||
<a href="https://www.promptfoo.dev/docs/">Documentation</a> ·
|
||||
<a href="https://discord.gg/promptfoo">Discord</a>
|
||||
</p>
|
||||
|
||||
> Promptfoo is now part of OpenAI. Promptfoo remains open source and MIT licensed. Read the [company update](https://www.promptfoo.dev/blog/promptfoo-joining-openai/).
|
||||
|
||||
## Quick Start
|
||||
|
||||
Requires [Node.js](https://nodejs.org/en/download) `^20.20.0` or `>=22.22.0` for npm and npx usage.
|
||||
[Node.js 20 support ends July 30, 2026 at 00:00 UTC](https://www.promptfoo.dev/docs/installation/#nodejs-runtime-support); upgrade to Node.js 24 LTS before updating promptfoo at or after the cutoff.
|
||||
|
||||
```sh
|
||||
npm install -g promptfoo
|
||||
promptfoo init --example getting-started
|
||||
```
|
||||
|
||||
Also available via `brew install promptfoo` and `pip install promptfoo`. You can also use `npx promptfoo@latest` to run any command without installing.
|
||||
|
||||
Most LLM providers require an API key. Set yours as an environment variable:
|
||||
|
||||
```sh
|
||||
export OPENAI_API_KEY=sk-abc123
|
||||
```
|
||||
|
||||
Once you're in the example directory, run an eval and view results:
|
||||
|
||||
```sh
|
||||
cd getting-started
|
||||
promptfoo eval
|
||||
promptfoo view
|
||||
```
|
||||
|
||||
See [Getting Started](https://www.promptfoo.dev/docs/getting-started/) (evals) or [Red Teaming](https://www.promptfoo.dev/docs/red-team/) (vulnerability scanning) for more.
|
||||
|
||||
## What can you do with Promptfoo?
|
||||
|
||||
- **Test your prompts and models** with [automated evaluations](https://www.promptfoo.dev/docs/getting-started/)
|
||||
- **Secure your LLM apps** with [red teaming](https://www.promptfoo.dev/docs/red-team/) and vulnerability scanning
|
||||
- **Compare models** side-by-side (OpenAI, Anthropic, Azure, Bedrock, Ollama, and [more](https://www.promptfoo.dev/docs/providers/))
|
||||
- **Automate checks** in [CI/CD](https://www.promptfoo.dev/docs/integrations/ci-cd/)
|
||||
- **Review pull requests** for LLM-related security and compliance issues with [code scanning](https://www.promptfoo.dev/docs/code-scanning/)
|
||||
- **Share results** with your team
|
||||
|
||||
Here's what it looks like in action:
|
||||
|
||||
<img src="site/static/img/claude-vs-gpt-example@2x.png" alt="prompt evaluation matrix - web viewer" width="700">
|
||||
|
||||
It works on the command line too:
|
||||
|
||||
<img src="https://www.promptfoo.dev/img/docs/self-grading.gif" alt="promptfoo command line" width="700">
|
||||
|
||||
It also can generate [security vulnerability reports](https://www.promptfoo.dev/docs/red-team/):
|
||||
|
||||
<img src="https://www.promptfoo.dev/img/redteam-dashboard@2x.jpg" alt="gen ai red team" width="700">
|
||||
|
||||
## Why Promptfoo?
|
||||
|
||||
- **Developer-first**: Fast, with features like live reload and caching
|
||||
- **Private**: LLM evals run 100% locally - your prompts never leave your machine
|
||||
- **Flexible**: Works with any LLM API or programming language
|
||||
- **Battle-tested**: Powers LLM apps serving 10M+ users in production
|
||||
- **Data-driven**: Make decisions based on metrics, not gut feel
|
||||
- **Open source**: MIT licensed, with an active community
|
||||
|
||||
## Learn More
|
||||
|
||||
- [Getting Started](https://www.promptfoo.dev/docs/getting-started/)
|
||||
- [Full Documentation](https://www.promptfoo.dev/docs/intro/)
|
||||
- [Red Teaming Guide](https://www.promptfoo.dev/docs/red-team/)
|
||||
- [CLI Usage](https://www.promptfoo.dev/docs/usage/command-line/)
|
||||
- [Node.js Package](https://www.promptfoo.dev/docs/usage/node-package/)
|
||||
- [Supported Models](https://www.promptfoo.dev/docs/providers/)
|
||||
- [Code Scanning Guide](https://www.promptfoo.dev/docs/code-scanning/)
|
||||
|
||||
## Contributing
|
||||
|
||||
We welcome contributions! Check out our [contributing guide](https://www.promptfoo.dev/docs/contributing/) to get started.
|
||||
|
||||
Join our [Discord community](https://discord.gg/promptfoo) for help and discussion.
|
||||
|
||||
<a href="https://github.com/promptfoo/promptfoo/graphs/contributors">
|
||||
<img src="https://contrib.rocks/image?repo=promptfoo/promptfoo" />
|
||||
</a>
|
||||
Reference in New Issue
Block a user