comet-ml--opik/apps/opik-documentation/documentation/fern/docs-v2/integrations/ragas.mdx

---
description: Use Ragas evaluation metrics to assess your LLM application quality and
  automatically track results in Opik for comprehensive performance monitoring.
headline: Ragas | Opik Documentation
og:description: Learn to integrate Ragas with Opik for efficient evaluation of RAG
  systems and scoring of traces or spans.
og:site_name: Opik Documentation
og:title: Evaluate RAG Systems with Opik
title: Evaluate LLM Applications with Ragas Metrics in Opik
---

<Note>
  In Opik 2.0, datasets and experiments are project-scoped. Make sure to specify a `project_name` when creating datasets and running experiments so they are associated with the correct project.
</Note>

The Opik SDK provides a simple way to integrate with Ragas, a framework for evaluating RAG systems.

There are two main ways to use Ragas with Opik:

1. Using Ragas to score traces or spans.
2. Using Ragas to evaluate a RAG pipeline.

## Account Setup

[Comet](https://www.comet.com/site?from=llm&utm_source=opik&utm_medium=colab&utm_content=ragas&utm_campaign=opik) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm&utm_source=opik&utm_medium=colab&utm_content=ragas&utm_campaign=opik) and grab your API Key.

> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/overview/?from=llm&utm_source=opik&utm_medium=colab&utm_content=ragas&utm_campaign=opik) for more information.

## Getting Started

### Installation

You will first need to install the `opik` and `ragas` packages:

```bash
pip install opik ragas
```

### Configuring Opik

Configure the Opik Python SDK for your deployment type. See the [Python SDK Configuration guide](/tracing/advanced/sdk_configuration) for detailed instructions on:

- **CLI configuration**: `opik configure`
- **Code configuration**: `opik.configure()`
- **Self-hosted vs Cloud vs Enterprise** setup
- **Configuration files** and environment variables

### Configuring Ragas

In order to use Ragas, you will need to configure your LLM provider API keys. For this example, we'll use OpenAI. You can [find or create your API keys in these pages](https://platform.openai.com/settings/organization/api-keys):

You can set them as environment variables:

```bash
export OPENAI_API_KEY="YOUR_API_KEY"
```

Or set them programmatically:

```python
import os
import getpass

if "OPENAI_API_KEY" not in os.environ:
    os.environ["OPENAI_API_KEY"] = getpass.getpass("Enter your OpenAI API key: ")
```

## Using Ragas to score traces or spans

Ragas provides a set of metrics that can be used to evaluate the quality of a RAG pipeline, a full list of the supported metrics can be found in the [Ragas documentation](https://docs.ragas.io/en/latest/concepts/metrics/available_metrics/).

You can use the `RagasMetricWrapper` to easily integrate Ragas metrics with Opik tracking:

```python
# Import the required dependencies
from ragas.metrics import AnswerRelevancy
from langchain_openai.chat_models import ChatOpenAI
from langchain_openai.embeddings import OpenAIEmbeddings
from ragas.llms import LangchainLLMWrapper
from ragas.embeddings import LangchainEmbeddingsWrapper
from opik.evaluation.metrics import RagasMetricWrapper

# Initialize the Ragas metric
llm = LangchainLLMWrapper(ChatOpenAI())
emb = LangchainEmbeddingsWrapper(OpenAIEmbeddings())
ragas_answer_relevancy = AnswerRelevancy(llm=llm, embeddings=emb)

# Wrap the Ragas metric with RagasMetricWrapper for Opik integration
answer_relevancy_metric = RagasMetricWrapper(
    ragas_answer_relevancy,
    track=True  # This enables automatic tracing in Opik
)
```

Once the metric wrapper is set up, you can use it to score traces or spans:

```python
from opik import track
from opik.opik_context import update_current_trace

@track
def retrieve_contexts(question):
    # Define the retrieval function, in this case we will hard code the contexts
    return ["Paris is the capital of France.", "Paris is in France."]

@track
def answer_question(question, contexts):
    # Define the answer function, in this case we will hard code the answer
    return "Paris"

@track
def rag_pipeline(question):
    # Define the pipeline
    contexts = retrieve_contexts(question)
    answer = answer_question(question, contexts)

    # Score the pipeline using the RagasMetricWrapper
    score_result = answer_relevancy_metric.score(
        user_input=question,
        response=answer,
        retrieved_contexts=contexts
    )

    # Add the score to the current trace
    update_current_trace(
        feedback_scores=[{"name": score_result.name, "value": score_result.value}]
    )

    return answer

print(rag_pipeline("What is the capital of France?"))
```

In the Opik UI, you will be able to see the full trace including the score calculation:

<Frame>
  <img src="/img/tracing/ragas_opik_trace.png" />
</Frame>

## Comprehensive Example: Dataset Evaluation

For more advanced use cases, you can evaluate entire datasets using Ragas metrics with the Opik evaluation platform:

### 1. Create a Dataset

```python
from datasets import load_dataset
import opik

opik_client = opik.Opik()

# Create a small dataset
fiqa_eval = load_dataset("explodinggradients/fiqa", "ragas_eval")

# Reformat the dataset to match the schema expected by the Ragas evaluate function
hf_dataset = fiqa_eval["baseline"].select(range(3))
dataset_items = hf_dataset.map(
    lambda x: {
        "user_input": x["question"],
        "reference": x["ground_truths"][0],
        "retrieved_contexts": x["contexts"],
    }
)
dataset = opik_client.get_or_create_dataset("ragas-demo-dataset", project_name="my-project")
dataset.insert(dataset_items)
```

### 2. Define Evaluation Task

```python
# Create an evaluation task
def evaluation_task(x):
    return {
        "user_input": x["question"],
        "response": x["answer"],
        "retrieved_contexts": x["contexts"],
    }
```

### 3. Run Evaluation

```python
# Use the RagasMetricWrapper directly with Opik's evaluate function
opik.evaluation.evaluate(
    dataset,
    evaluation_task,
    scoring_metrics=[answer_relevancy_metric],
    task_threads=1,
)
```

### 4. Alternative: Using Ragas Native Evaluation

You can also use Ragas' native evaluation function with Opik tracing:

```python
from datasets import load_dataset
from opik.integrations.langchain import OpikTracer
from ragas.metrics import context_precision, answer_relevancy, faithfulness
from ragas import evaluate

fiqa_eval = load_dataset("explodinggradients/fiqa", "ragas_eval")

# Reformat the dataset to match the schema expected by the Ragas evaluate function
dataset = fiqa_eval["baseline"].select(range(3))

dataset = dataset.map(
    lambda x: {
        "user_input": x["question"],
        "reference": x["ground_truths"][0],
        "retrieved_contexts": x["contexts"],
    }
)

opik_tracer_eval = OpikTracer(tags=["ragas_eval"], metadata={"evaluation_run": True})

result = evaluate(
    dataset,
    metrics=[context_precision, faithfulness, answer_relevancy],
    callbacks=[opik_tracer_eval],
)

print(result)
```

## Using Ragas metrics to evaluate a RAG pipeline

The `RagasMetricWrapper` can also be used directly within the Opik evaluation platform. This approach is much simpler than creating custom wrappers:

### 1. Define the Ragas metric

We will start by defining the Ragas metric, in this example we will use `AnswerRelevancy`:

```python
from ragas.metrics import AnswerRelevancy
from langchain_openai.chat_models import ChatOpenAI
from langchain_openai.embeddings import OpenAIEmbeddings
from ragas.llms import LangchainLLMWrapper
from ragas.embeddings import LangchainEmbeddingsWrapper
from opik.evaluation.metrics import RagasMetricWrapper

# Initialize the Ragas metric
llm = LangchainLLMWrapper(ChatOpenAI())
emb = LangchainEmbeddingsWrapper(OpenAIEmbeddings())

ragas_answer_relevancy = AnswerRelevancy(llm=llm, embeddings=emb)
```

### 2. Create the metric wrapper

Simply wrap the Ragas metric with `RagasMetricWrapper`:

```python
# Create the answer relevancy scoring metric
answer_relevancy = RagasMetricWrapper(
    ragas_answer_relevancy,
    track=True  # Enable tracing for the metric computation
)
```

<Tip>

If you are running within a Jupyter notebook, you will need to add the following line to the top of your notebook:

```python
import nest_asyncio
nest_asyncio.apply()
```

</Tip>

### 3. Use the metric wrapper within the Opik evaluation platform

You can now use the metric wrapper directly within the Opik evaluation platform:

```python
from opik.evaluation import evaluate

evaluation_task = evaluate(
    dataset=dataset,
    task=evaluation_task,
    scoring_metrics=[answer_relevancy],
    nb_samples=10,
    project_name="my-project",
)
```

The `RagasMetricWrapper` automatically handles:

- Field mapping between Opik and Ragas (e.g., `input` → `user_input`, `output` → `response`)
- Async execution of Ragas metrics
- Integration with Opik's tracing system when `track=True`
- Proper error handling for missing required fields