comet-ml--opik/apps/opik-documentation/documentation/fern/docs/tracing/annotate_traces.mdx

---
headline: Log user feedback | Opik Documentation
og:description: Capture user feedback and track performance metrics to enhance your
  agent's effectiveness with Opik's logging capabilities.
og:site_name: Opik Documentation
og:title: Log User Feedback Effectively - Opik
title: Log user feedback
canonical-url: https://www.comet.com/docs/opik/tracing/advanced/annotate_traces
---

Logging user feedback and scoring traces is a crucial aspect of evaluating and improving your agent.
By systematically recording qualitative or quantitative feedback on specific interactions or entire
conversation flows, you can:

1. Track performance over time
2. Identify areas for improvement
3. Compare different model versions or prompts
4. Gather data for fine-tuning or retraining
5. Provide stakeholders with concrete metrics on system effectiveness

## Logging user feedback using the SDK

You can use the SDKs to log user feedback and score traces:

<CodeBlocks>
```typescript title="Typescript SDK" language="typescript"
import { Opik } from "opik";

const client = new Opik();

// Create a new trace with a span
const trace = client.trace({
  name: "my_trace",
  input: { input: "Hi!" },
  output: { output: "Hello!" },
});

const span = trace.span({
  name: "processing",
  input: { step: 1 },
});
span.update({ output: { result: "processed" } });
span.end();
trace.end();

// Log feedback scores to existing traces
client.logTracesFeedbackScores([
  { id: trace.data.id, name: "overall_quality", value: 0.9, reason: "Good answer" },
  { id: trace.data.id, name: "coherence", value: 0.8 }
]);

// Log feedback scores to existing spans
client.logSpansFeedbackScores([
  { id: span.data.id, name: "accuracy", value: 0.95 }
]);

// Flush to ensure all data is sent
await client.flush();
```

```python title="Python Function Decorator" language="python"
import opik
from opik import opik_context

@opik.track
def my_function():
  opik_context.update_current_trace(
    feedback_scores=[
      {
        "name": "user_feedback",
        "value": 1,
        "reason": "Good answer" # Optional
      }
    ]
  )

  return "Hello, world!"
```

```python title="Python SDK" language="python"
import opik

client = opik.Opik()

# Log feedback scores to an existing trace
client.log_traces_feedback_scores(
  scores=[
    {"id": "trace_id", "name": "user_feedback", "value": 1, "project_name": "my-project"},
    {"id": "trace_id", "name": "accuracy", "value": 1, "reason": "Good answer", "project_name": "my-project"} # Optional reason score
  ]
)

# Log feedback score to a new trace
client.trace(
  name="my_trace",
  input={"input": "Hi!"},
  output={"output": "Hello!"},
  feedback_scores=[
    {"name": "user_feedback", "value": 1, "reason": "Good answer"}
  ]
)
```

</CodeBlocks>

## Annotating Traces through the UI

To annotate traces through the UI, you can navigate to the trace you want to annotate in the traces
page and click on the `Annotate` button. This will open a sidebar where you can add annotations to
the trace.

You can annotate both traces and spans through the UI, make sure you have selected the correct span
in the sidebar.

<Frame>
  <img src="/img/tracing/multivalue_feedback_scores.png" />
</Frame>

Once a feedback scores has been provided, you can also add a reason to explain why this particular
score was provided. This is useful to add additional context to the score.

If multiple team members are annotating the same trace, you can see the annotations of each team
member in the UI in the `Feedback scores` section. The average score will be displayed at a trace
and trace level.

<Tip>
  If you want a more dedicated annotation interface, you can use the [Annotation Queues](/v1/evaluation/annotation_queues)
  feature.
</Tip>

## Online evaluation

You don't need to manually annotate each trace to measure the performance of your agents! By using
Opik's [online evaluation feature](/v1/production/rules), you can define LLM as a Judge metrics that
will automatically score all, or a subset, of your production traces.

![Online evaluation](/img/production/online_evaluation.gif)

## Manual evaluation

While online evaluation automatically scores traces based on sampling rates and enabled rules, manual evaluation gives you complete control over which traces or threads get evaluated and when. This is particularly useful when you want to:

- Evaluate specific traces or threads that failed or require closer inspection
- Apply evaluation rules to historical data that wasn't captured by sampling
- Test new evaluation rules on selected examples before enabling them for automatic scoring
- Re-evaluate traces with updated or modified rules

### How manual evaluation works

Manual evaluation allows you to apply any existing evaluation rule to selected traces or threads directly from the UI, bypassing sampling rates and rule enablement status. You can trigger manual evaluation from:

1. **Traces page**: Select one or more traces and click "Evaluate" to apply trace-level rules
2. **Threads page**: Select one or more threads and click "Evaluate" to apply thread-level rules

<Note>
  **Important**: Trace-level rules can only be applied to traces, and thread-level rules can only be applied to threads. Make sure you're using the appropriate rule type for your selected entities.
</Note>

When you trigger manual evaluation:
- All selected traces/threads will be queued for evaluation, regardless of sampling rate
- You can apply multiple rules at once
- Rules will execute even if they are currently disabled
- Evaluation results will appear as feedback scores on the evaluated traces/threads
- The evaluation is processed asynchronously, so you may need to wait a few seconds or refresh the page to see the results

This gives you the flexibility to evaluate exactly what you need, when you need it, without waiting for the next sampled trace or modifying your online evaluation configuration.

## Next steps

You can go one step further and:

1. [Create an offline evaluation](/v1/evaluation/evaluate_prompt) to evaluate your agent before it is
   deployed to production
2. [Score your agent in production](/v1/production/rules) to track and catch specific issues with your
   agent
3. [Use annotation queues](/v1/evaluation/annotation_queues) to organize your traces for review and
   labeling by your team of experts
4. [Checkout our LLM as a Judge metrics](/v1/evaluation/metrics/overview)