comet-ml--opik/apps/opik-documentation/documentation/fern/docs/tracing/cost_tracking.mdx

---
headline: Cost tracking | Opik Documentation
og:description: Monitor and analyze cost patterns in your LLM apps using Opik's dashboard
  to identify anomalies and optimize spending.
og:site_name: Opik Documentation
og:title: Cost Tracking with Opik for LLM Applications
title: Cost tracking
canonical-url: https://www.comet.com/docs/opik/tracing/advanced/cost_tracking
---

Opik has been designed to track and monitor costs for your LLM applications by measuring token usage across all traces. Using the Opik dashboard, you can analyze spending patterns and quickly identify cost anomalies. All costs across Opik are estimated and displayed in USD.

## Monitoring Costs in the Dashboard

You can use the Opik dashboard to review costs at three levels: spans, traces, and projects. Each level provides different insights into your application's cost structure.

### Span-Level Costs

Individual spans show the computed costs (in USD) for each LLM spans of your traces:

<Frame>
  <img src="/img/tracing/cost_tracking_span.png" />
</Frame>

### Trace-Level Costs

If you are using one of Opik's integrations, we automatically aggregate costs from all spans within a trace to compute total trace costs:

<Frame>
  <img src="/img/tracing/cost_tracking_trace_view.png" />
</Frame>

### Project-Level Analytics

Track your overall project costs in:

1. The main project view, through the Estimated Cost column:

   <Frame>
     <img src="/img/tracing/cost_tracking_project.png" />
   </Frame>

2. The project Metrics tab, which shows cost trends over time:
   <Frame>
     <img src="/img/tracing/cost_tracking_project_metrics.png" />
   </Frame>

## Retrieving Costs Programmatically

You can retrieve the estimated cost programmatically for both spans and traces. Note that the cost will be `None` if the span or trace used an unsupported model. See [Exporting Traces and Spans](/v1/tracing/export_data) for more ways of exporting traces and spans.

### Retrieving Span Costs

```python
import opik

client = opik.Opik()

span = client.get_span_content("<SPAN_ID>")
# Returns estimated cost in USD, or None for unsupported models
print(span.total_estimated_cost)
```

### Retrieving Trace Costs

```python
import opik

client = opik.Opik()

trace = client.get_trace_content("<TRACE_ID>")
# Returns estimated cost in USD, or None for unsupported models
print(trace.total_estimated_cost)
```

## Manually setting the provider and model name

If you are not using one of Opik's integrations, Opik can still compute the cost. To do this, you will need to ensure the
span type is `llm` and you will need to pass:

1. `provider`: The name of the provider, typically `openai`, `anthropic` or `google_ai` for example (the most recent providers list can be found in `opik.LLMProvider` enum object)
2. `model`: The name of the model
3. `usage`: The input, output and total tokens for this LLM call.

You can then update your code to log traces and spans:

<Tabs>
<Tab title="Function decorator" value="Function decorator">
If you are using function decorators, you will need to use the `update_current_span` method:

```python
from opik import track, opik_context

@track(type="llm") # Note - Specifying the type is this is important
def llm_call(input):
  opik_context.update_current_span(
    provider="openai",
    model="gpt-3.5-turbo",
    usage={
      "prompt_tokens": 4,
      "completion_tokens": 6,
      "total_tokens": 10
    }
  )
  return "Hello, world!"

llm_call("Hello world!")
```

</Tab>
<Tab title="Low level Python SDK" value="Low level Python SDK">

When using the low level Python SDK, you will need to update the `client.span` or `trace.span` methods:

```python
import opik

client = opik.Opik()

trace = client.trace(
  name="custom_trace",
  input={"text": "Hello world!"},
)

# Logging the LLM call
span = trace.span(
  name="llm_call",
  type="llm",
  input={"text": "Hello world!"},
  output={"response": "Hello world!"},
  provider="openai",
  model="gpt-3.5-turbo",
  usage={
    "prompt_tokens": 4,
    "completion_tokens": 6,
    "total_tokens": 10
  }
)
```

</Tab>
</Tabs>

## Manually Setting Span Costs

When you need to set a custom cost or use an unsupported model, you can manually set the cost of a span. There are two approaches depending on your use case:

### Setting Costs During Span Creation

If you're manually creating spans, you can set the cost directly when creating the span:

```python
from opik import track, opik_context

@track
def llm_call(input):
  opik_context.update_current_span(
    total_cost=0.05,
  )
  return "Hello, world!"

llm_call("Hello world!")
```

### Updating Costs After Span Completion

With Opik integrations, spans are automatically created and closed, preventing updates while they're open. However, you can update the cost afterward using the `update_span` method. This works well for implementing periodic cost estimation jobs:

```python
from opik import Opik
from opik.rest_api.types.span_public import SpanPublic

# Define your own cost mapping for different models
TOKEN_COST = {
    ("openai.chat", "gpt-4o-2024-08-06"): {
        "input_tokens": 2.5e-06,
        "output_tokens": 1e-05,
    }
}

# This part would be custom for your use-case and is only here for example
def compute_cost_for_span(span: SpanPublic):
    provider = span.provider or span.input.get("ai.model.provider")
    model = span.model or span.output.get("gen_ai.response.model")
    usage = span.usage

    if (provider, model) in TOKEN_COST:
        model_cost = TOKEN_COST[(provider, model)]
        cost = (
            usage["input_tokens"] * model_cost["input_tokens"]
            + usage["output_tokens"] * model_cost["output_tokens"]
        )
        return cost
    return None

def update_span_costs(project_name, trace_id=None):
    opik_client = Opik()

    # Find LLM spans that don't have estimated costs
    spans = opik_client.search_spans(
        project_name=project_name,
        trace_id=trace_id,
        filter_string='type="llm" and total_estimated_cost=0',
    )

    for span in spans:
        cost = compute_cost_for_span(span)

        if cost:
            print(f"Updating span {span.id} of trace {span.trace_id} with cost: {cost}")
            opik_client.update_span(
                trace_id=span.trace_id,
                parent_span_id=span.parent_span_id,
                project_name=project_name,
                id=span.id,
                total_cost=cost,
            )

# Example usage in a CRON job
if __name__ == "__main__":
    update_span_costs("your-project-name")
```

This approach is particularly useful when:

- Using models or providers not yet supported by automatic cost tracking
- You have a custom pricing agreement with your provider
- You want to track additional costs beyond model usage
- You need to implement cost estimation as a background process
- Working with integrations where spans are automatically managed

<Tip>
  You can run the cost update function as a CRON job to automatically update costs for spans created without cost
  information. This is especially valuable in production environments where accurate cost data for all spans is
  essential.
</Tip>

## Supported Models, Providers, and Integrations

Opik currently calculates costs automatically for all LLM calls in the following Python SDK integrations:

- [Google ADK Integration](/v1/integrations/adk)
- [AWS Bedrock Integration](/v1/integrations/bedrock)
- [LangChain Integration](/v1/integrations/langchain)
- [OpenAI Integration](/v1/integrations/openai)
- [LiteLLM Integration](https://docs.litellm.ai/docs/observability/opik_integration)
- [Anthropic Integration](/v1/integrations/anthropic)
- [CrewAI Integration](/v1/integrations/crewai)
- [Google AI Integration](/v1/integrations/gemini)
- [Haystack Integration](/v1/integrations/haystack)
- [LlamaIndex Integration](/v1/integrations/llama_index)

### Supported Providers

Cost tracking is supported for the following LLM providers (as defined in `opik.LLMProvider` enum):

- **OpenAI** (`openai`) - Models hosted by OpenAI (https://platform.openai.com)
- **Anthropic** (`anthropic`) - Models hosted by Anthropic (https://www.anthropic.com)
- **Anthropic on Vertex AI** (`anthropic_vertexai`) - Anthropic models hosted by Google Vertex AI
- **Google AI** (`google_ai`) - Gemini models hosted in Google AI Studio (https://ai.google.dev/aistudio)
- **Google Vertex AI** (`google_vertexai`) - Gemini models hosted in Google Vertex AI (https://cloud.google.com/vertex-ai)
- **AWS Bedrock** (`bedrock`) - Models hosted by AWS Bedrock (https://aws.amazon.com/bedrock)
- **Groq** (`groq`) - Models hosted by Groq (https://groq.com)

You can find a complete list of supported models for these providers in the
[model_prices_and_context_window.json file](https://github.com/comet-ml/opik/blob/main/apps/opik-backend/src/main/resources/model_prices_and_context_window.json).

<Tip>
  We are actively expanding our cost tracking support. Need support for additional models or providers? Please [open a
  feature request](https://github.com/comet-ml/opik/issues) to help us prioritize development.
</Tip>