chore: import upstream snapshot with attribution
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
CI / Shell Format Check (push) Has been cancelled
CI / Check Ruby (3.4) (push) Has been cancelled
CI / CI Config (push) Has been cancelled
CI / Test on Node ${{ matrix.node }} and ${{ matrix.os }}${{ matrix.shard && format(' (shard {0}/3)', matrix.shard) || '' }} (push) Has been cancelled
CI / Build on Node ${{ matrix.node }} (push) Has been cancelled
CI / Style Check (push) Has been cancelled
CI / Generate Assets (push) Has been cancelled
CI / Check Python (3.14) (push) Has been cancelled
CI / Check Python (3.9) (push) Has been cancelled
CI / Build Docs (push) Has been cancelled
CI / Code Scan Action (push) Has been cancelled
CI / Site tests (push) Has been cancelled
CI / webui tests (push) Has been cancelled
CI / Run Integration Tests (push) Has been cancelled
CI / Run Smoke Tests (push) Has been cancelled
CI / Go Tests (push) Has been cancelled
CI / Share Test (push) Has been cancelled
CI / Redteam (Production API) (push) Has been cancelled
CI / Redteam (Staging API) (push) Has been cancelled
CI / GitHub Actions Lint (push) Has been cancelled
CI / Check Ruby (3.0) (push) Has been cancelled
release-please / release-please (push) Has been cancelled
release-please / build (push) Has been cancelled
release-please / publish-npm (push) Has been cancelled
release-please / publish-npm-backfill (push) Has been cancelled
release-please / docker (push) Has been cancelled
release-please / publish-code-scan-action (push) Has been cancelled
release-please / attest-code-scan-action (push) Has been cancelled
Deploy local.promptfoo.app / Deploy to Cloudflare Pages (push) Has been cancelled
Test and Publish Multi-arch Docker Image / test (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-amd64 platform:linux/amd64 runner:ubuntu-latest]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / build-docker-and-push-digests (map[digest-suffix:linux-arm64 platform:linux/arm64 runner:ubuntu-24.04-arm]) (push) Has been cancelled
Test and Publish Multi-arch Docker Image / merge-docker-digests (push) Has been cancelled
Test and Publish Multi-arch Docker Image / Attest Multi-arch Image (push) Has been cancelled
Validate Renovate Config / Validate Renovate Configuration (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,58 @@
|
||||
# provider-lm-studio (LM Studio Example with Promptfoo)
|
||||
|
||||
You can run this example with:
|
||||
|
||||
```bash
|
||||
npx promptfoo@latest init --example provider-lm-studio
|
||||
cd provider-lm-studio
|
||||
```
|
||||
|
||||
This example demonstrates how to use Promptfoo with LM Studio for prompt evaluation. It showcases configuration for interacting with the LM Studio API using a locally hosted language model.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **LM Studio**: Install LM Studio from [lmstudio.ai](https://lmstudio.ai/).
|
||||
2. **Model**: Download the `bartowski/gemma-2-9b-it-GGUF` model in LM Studio.
|
||||
|
||||
## Setup
|
||||
|
||||
1. **Start LM Studio Server**:
|
||||
- Open LM Studio and load the `bartowski/gemma-2-9b-it-GGUF` model.
|
||||
- Start a local server to host the model (usually at `http://localhost:1234`).
|
||||
|
||||
2. **Configure Promptfoo**:
|
||||
|
||||
Create a `promptfooconfig.yaml` file with the following content:
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
- id: 'http://localhost:1234/v1/chat/completions'
|
||||
config:
|
||||
method: 'POST'
|
||||
headers:
|
||||
'Content-Type': 'application/json'
|
||||
body:
|
||||
messages: '{{ prompt }}'
|
||||
model: 'bartowski/gemma-2-9b-it-GGUF'
|
||||
temperature: 0.7
|
||||
max_tokens: -1
|
||||
transformResponse: 'json.choices[0].message.content'
|
||||
```
|
||||
|
||||
Note that you can view the specific configuration for each model within LM Studio's examples in the server tab.
|
||||
|
||||
## Usage
|
||||
|
||||
1. **Run Evaluation**:
|
||||
|
||||
```bash
|
||||
npx promptfoo eval
|
||||
```
|
||||
|
||||
2. **View Results**:
|
||||
|
||||
```sh
|
||||
npx promptfoo view
|
||||
```
|
||||
|
||||
For more information, see the [Promptfoo documentation](https://www.promptfoo.dev/docs/providers/http) on how to set up a custom HTTP provider.
|
||||
@@ -0,0 +1,3 @@
|
||||
module.exports = (json, text) => {
|
||||
return json.choices[0].message.content;
|
||||
};
|
||||
@@ -0,0 +1,19 @@
|
||||
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
|
||||
description: 'LM Studio Eval Example'
|
||||
prompts:
|
||||
- file://prompts.json
|
||||
providers:
|
||||
- id: 'http://localhost:1234/v1/chat/completions'
|
||||
config:
|
||||
method: 'POST'
|
||||
headers:
|
||||
'Content-Type': 'application/json'
|
||||
body:
|
||||
messages: '{{ prompt }}'
|
||||
model: 'bartowski/gemma-2-9b-it-GGUF'
|
||||
temperature: 0.7
|
||||
max_tokens: 100
|
||||
transformResponse: 'file://parser.js'
|
||||
tests:
|
||||
- vars:
|
||||
topic: bananas
|
||||
@@ -0,0 +1,4 @@
|
||||
[
|
||||
{ "role": "system", "content": "You are a helpful assistant" },
|
||||
{ "role": "user", "content": "Tell me about {{topic}}" }
|
||||
]
|
||||
Reference in New Issue
Block a user