chore: import upstream snapshot with attribution

This commit is contained in:
wehub-resource-sync
2026-07-13 13:32:05 +08:00
commit 3db5260065
2344 changed files with 440315 additions and 0 deletions
+129
View File
@@ -0,0 +1,129 @@
---
id: tutorial-setup
title: Set Up DeepEval
sidebar_label: Set Up DeepEval
---
import { ASSETS } from "@site/src/assets";
## Installing DeepEval
**DeepEval** is a powerful LLM evaluation framework. Here's how you can easily get started by installing and running your first evaluation using DeepEval.
Start by installing DeepEval using pip:
```bash
pip install -U deepeval
```
### Write your first test
Let's evaluate the correctness of an LLM output using [`GEval`](https://deepeval.com/docs/metrics-llm-evals), a powerful metric based on LLM-as-a-judge evaluation.
:::note
When you pass a file explicitly (e.g. `deepeval test run evals/my_eval.py`), DeepEval runs it regardless of its name. The `test_` prefix (like `test_app.py`) is only needed for pytest's automatic discovery when you point it at a directory.
:::
```python title="test_app.py"
from deepeval import evaluate
from deepeval.test_case import LLMTestCase, SingleTurnParams
from deepeval.metrics import GEval
correctness_metric = GEval(
name="Correctness",
criteria="Determine if the 'actual output' is correct based on the 'expected output'.",
evaluation_params=[SingleTurnParams.ACTUAL_OUTPUT, SingleTurnParams.EXPECTED_OUTPUT],
threshold=0.5
)
test_case = LLMTestCase(
input="I have a persistent cough and fever. Should I be worried?",
# Replace this with the actual output from your LLM application
actual_output="A persistent cough and fever could signal various illnesses, from minor infections to more serious conditions like pneumonia or COVID-19. It's advisable to seek medical attention if symptoms worsen, persist beyond a few days, or if you experience difficulty breathing, chest pain, or other concerning signs.",
expected_output="A persistent cough and fever could indicate a range of illnesses, from a mild viral infection to more serious conditions like pneumonia or COVID-19. You should seek medical attention if your symptoms worsen, persist for more than a few days, or are accompanied by difficulty breathing, chest pain, or other concerning signs."
)
evaluate([test_case], [correctness_metric])
```
To run your first evaluation, enter the following command in your terminal:
```bash
deepeval test run test_app.py
```
:::note
DeepEval's powerful **LLM-as-a-judge** metrics (like `GEval` used in this example) rely on an underlying LLM called the _Evaluation Model_ to perform evaluations. By default, DeepEval uses OpenAI's models for this purpose.
So you'll have to set your `OPENAI_API_KEY` as an environment variable as shown below.
```bash
export OPENAI_API_KEY="your_api_key"
```
To use ANY custom LLM of your choice, [Check out our docs on custom evaluation models](https://deepeval.com/guides/guides-using-custom-llms).
:::
Congratulations! You've successfully run your first LLM evaluation with DeepEval.
## Setting Up Confident AI
While DeepEval works great standalone, you can connect it to [Confident AI](https://www.confident-ai.com) — an AI quality platform with observability, evals, and monitoring that DeepEval integrates with natively for dashboards, logging, collaboration, and more. **Its free to get started.**
You can [sign up here](https://www.confident-ai.com), or run:
```bash
deepeval login
```
Navigate to your Settings page and copy your Confident AI API Key from the Project API Key box. If you used the `deepeval login` command to log in, you'll be prompted to paste your Confident AI API Key after creating an account.
<div
style={{
display: "flex",
alignItems: "center",
justifyContent: "center",
}}
>
<ImageDisplayer src={ASSETS.tutorialSetup01} />
</div>
Alternatively, if you already have an account, you can log in directly using Python:
```python title="main.py"
deepeval.login("your-confident-api-key")
```
Or through the CLI:
```bash
deepeval login --confident-api-key "your-confident-api-key"
```
:::note[Login persistence]
`deepeval login` persists your key to a dotenv file by default (.env.local).
To change the target, use `--save`, e.g.:
```bash
# custom path
deepeval login --confident-api-key "ck_..." --save dotenv:.env.custom
```
For compatibility, the key is saved under `api_key` and `CONFIDENT_API_KEY`.
Secrets are never written to the JSON keystore.
:::
:::tip[Logging out / rotating keys]
Use deepeval logout to clear the JSON keystore and remove saved keys from your dotenv file:
```bash
# default removes from .env.local
deepeval logout
# or specify a custom target
deepeval logout --save dotenv:.myconf.env
```
:::
You're all set! You can now evaluate LLMs locally and monitor them in Confident AI.