chore: import upstream snapshot with attribution

2026-07-13 13:24:08 +08:00
commit 0d3cb498a3
5438 changed files with 1316560 additions and 0 deletions
@@ -0,0 +1,228 @@
+---
+sidebar_label: Replicate
+description: "Deploy and run open-source AI models in the cloud using Replicate's scalable API for image, text, and audio generation"
+---
+
+# Replicate
+
+Replicate is an API for machine learning models. It currently hosts models like [Llama v2](https://replicate.com/replicate/llama70b-v2-chat), [Gemma](https://replicate.com/google-deepmind/gemma-7b-it), and [Mistral/Mixtral](https://replicate.com/mistralai/mixtral-8x7b-instruct-v0.1).
+
+:::info
+The Replicate provider in promptfoo uses direct HTTP requests to the Replicate API, so no additional SDK installation is required.
+:::
+
+To run a model, specify the Replicate model name and optionally the version:
+
+```
+# With specific version (recommended for consistency)
+replicate:replicate/llama70b-v2-chat:e951f18578850b652510200860fc4ea62b3b16fac280f83ff32282f87bbd2e48
+
+# Without version (uses latest)
+replicate:meta/meta-llama-3-8b-instruct
+```
+
+:::tip
+For production use, always specify the version to ensure consistent results. You can find version IDs on the model's page on Replicate.
+:::
+
+## Examples
+
+Here's an example of using Llama on Replicate. In the case of Llama, the version hash and everything under `config` is optional:
+
+```yaml
+providers:
+  - id: replicate:meta/llama-2-7b-chat
+    config:
+      temperature: 0.01
+      max_length: 1024
+      prompt:
+        prefix: '[INST] '
+        suffix: ' [/INST]'
+```
+
+Here's an example of using Gemma on Replicate. Note that unlike Llama, it does not have a default version, so we specify the model version:
+
+```yaml
+providers:
+  - id: replicate:google-deepmind/gemma-7b-it:2790a695e5dcae15506138cc4718d1106d0d475e6dca4b1d43f42414647993d5
+    config:
+      temperature: 0.01
+      max_new_tokens: 1024
+      prompt:
+        prefix: "<start_of_turn>user\n"
+        suffix: "<end_of_turn>\n<start_of_turn>model"
+```
+
+## Configuration
+
+The Replicate provider supports several [configuration options](https://github.com/promptfoo/promptfoo/blob/main/src/providers/replicate.ts#L24) that can be used to customize the behavior of the models, like so:
+
+| Parameter            | Description                                                   |
+| -------------------- | ------------------------------------------------------------- |
+| `temperature`        | Controls randomness in the generation process.                |
+| `max_length`         | Specifies the maximum length of the generated text.           |
+| `max_new_tokens`     | Limits the number of new tokens to generate.                  |
+| `top_p`              | Nucleus sampling: a float between 0 and 1.                    |
+| `top_k`              | Top-k sampling: number of highest probability tokens to keep. |
+| `repetition_penalty` | Penalizes repetition of words in the generated text.          |
+| `system_prompt`      | Sets a system-level prompt for all requests.                  |
+| `stop_sequences`     | Specifies stopping sequences that halt the generation.        |
+| `seed`               | Sets a seed for reproducible results.                         |
+
+:::warning
+Not every model supports every completion parameter. Be sure to review the API provided by the model beforehand.
+:::
+
+These parameters are supported for all models:
+
+| Parameter       | Description                                                              |
+| --------------- | ------------------------------------------------------------------------ |
+| `apiKey`        | The API key for authentication with Replicate.                           |
+| `prompt.prefix` | String added before each prompt. Useful for instruction/chat formatting. |
+| `prompt.suffix` | String added after each prompt. Useful for instruction/chat formatting.  |
+
+Supported environment variables:
+
+- `REPLICATE_API_TOKEN` - Your Replicate API key.
+- `REPLICATE_API_KEY` - An alternative to `REPLICATE_API_TOKEN` for your API key.
+- `REPLICATE_MAX_LENGTH` - Specifies the maximum length of the generated text.
+- `REPLICATE_TEMPERATURE` - Controls randomness in the generation process.
+- `REPLICATE_REPETITION_PENALTY` - Penalizes repetition of words in the generated text.
+- `REPLICATE_TOP_P` - Controls the nucleus sampling: a float between 0 and 1.
+- `REPLICATE_TOP_K` - Controls the top-k sampling: the number of highest probability vocabulary tokens to keep for top-k-filtering.
+- `REPLICATE_SEED` - Sets a seed for reproducible results.
+- `REPLICATE_STOP_SEQUENCES` - Specifies stopping sequences that halt the generation.
+- `REPLICATE_SYSTEM_PROMPT` - Sets a system-level prompt for all requests.
+
+## Images
+
+Image generators such as SDXL can be used like so:
+
+```yaml
+prompts:
+  - 'Generate an image: {{subject}}'
+
+providers:
+  - id: replicate:image:stability-ai/sdxl:7762fd07cf82c948538e41f63f77d685e02b063e37e496e96eefd46c929f9bdc
+    config:
+      width: 768
+      height: 768
+      num_inference_steps: 50
+
+tests:
+  - vars:
+      subject: fruit loops
+```
+
+## Supported Parameters for Images
+
+These parameters are supported for image generation models:
+
+| Parameter             | Description                                                   |
+| --------------------- | ------------------------------------------------------------- |
+| `width`               | The width of the generated image.                             |
+| `height`              | The height of the generated image.                            |
+| `refine`              | Which refine style to use                                     |
+| `apply_watermark`     | Apply a watermark to the generated image.                     |
+| `num_inference_steps` | The number of inference steps to use during image generation. |
+
+:::warning
+Not every model supports every image parameter. Be sure to review the API provided by the model beforehand.
+:::
+
+Supported environment variables for images:
+
+- `REPLICATE_API_TOKEN` - Your Replicate API key.
+- `REPLICATE_API_KEY` - An alternative to `REPLICATE_API_TOKEN` for your API key.
+
+:::warning
+**Important:** Replicate image URLs are temporary and typically expire after 24 hours. If you need to preserve generated images, download them immediately or use the automated download hook described below.
+:::
+
+## Downloading Generated Images
+
+Since Replicate image URLs expire, you may want to automatically download and save images during evaluation. You can use an `afterEach` hook for this purpose:
+
+Create a file `save-images.js`:
+
+```javascript
+const fs = require('fs');
+const path = require('path');
+
+// For Node >= 20, fetch is available globally
+const { fetch } = globalThis;
+
+/**
+ * Downloads and saves Replicate generated images after each test
+ */
+module.exports = {
+  async hook(hookName, context) {
+    // Only run for afterEach hook and when we have an output
+    if (hookName !== 'afterEach') {
+      return;
+    }
+
+    // Extract URL from markdown image format
+    const output = context.result?.response?.output;
+    if (!output || typeof output !== 'string') {
+      return;
+    }
+
+    const match = output.match(/!\[.*?\]\((.*?)\)/);
+    const imageUrl = match?.[1];
+    if (!imageUrl || !imageUrl.includes('replicate.delivery')) {
+      return;
+    }
+
+    try {
+      // Create images directory if it doesn't exist
+      const imagesDir = path.join(__dirname, 'images');
+      await fs.promises.mkdir(imagesDir, { recursive: true });
+
+      // Generate filename from test description and timestamp
+      const testDesc = context.test.description || 'unnamed';
+      const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
+      const sanitizedName = testDesc
+        .replace(/[^a-z0-9\s-]/gi, '')
+        .trim()
+        .replace(/\s+/g, '-')
+        .toLowerCase();
+      const filename = `${sanitizedName}-${timestamp}.png`;
+      const filepath = path.join(imagesDir, filename);
+
+      // Download and save the image
+      const response = await fetch(imageUrl);
+      if (!response.ok) {
+        throw new Error(`HTTP error: ${response.status}`);
+      }
+
+      const buffer = await response.arrayBuffer();
+      await fs.promises.writeFile(filepath, Buffer.from(buffer));
+
+      console.log(`✓ Saved image: ${filename}`);
+    } catch (error) {
+      console.error(`❌ Failed to save image: ${error.message}`);
+    }
+  },
+};
+```
+
+Then reference it in your promptfoo configuration:
+
+```yaml
+# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
+extensions:
+  - file://save-images.js:hook
+
+prompts:
+  - 'Generate an image: {{subject}}'
+
+providers:
+  - replicate:image:black-forest-labs/flux-dev
+
+tests:
+  - vars:
+      subject: a beautiful sunset over mountains
+```
+
+This hook will automatically download all generated images to an `images/` directory with descriptive filenames based on the test description and timestamp.