comet-ml--opik/apps/opik-documentation/python-sdk-docs/source/simulation/index.rst

Simulation
==========

The Opik simulation module provides tools for creating multi-turn conversation simulations between simulated users and your applications. This is particularly useful for evaluating agent behavior over multiple conversation turns.

.. toctree::
   :maxdepth: 1
   
   SimulatedUser
   run_simulation

Overview
--------

Multi-turn simulation allows you to:

- **Simulate realistic user interactions** with your agent over multiple conversation turns
- **Generate context-aware user responses** based on conversation history
- **Evaluate agent behavior** across extended conversations
- **Test different user personas** and scenarios systematically

Key Components
---------------

**SimulatedUser**: A class that generates realistic user responses using LLMs or predefined responses.

**run_simulation**: A function that orchestrates multi-turn conversations between a simulated user and your application.

Basic Usage
-----------

Here's a simple example of how to use the simulation module:

.. code-block:: python

   from opik.simulation import SimulatedUser, run_simulation
   from opik import track

   # Create a simulated user
   user_simulator = SimulatedUser(
       persona="You are a frustrated customer who wants a refund",
       model="openai/gpt-5-nano"
   )

   # Define your agent
   @track
   def my_agent(user_message: str, *, thread_id: str, **kwargs):
       # Your agent logic here
       return {"role": "assistant", "content": "I can help you with that..."}

   # Run the simulation
   simulation = run_simulation(
       app=my_agent,
       user_simulator=user_simulator,
       max_turns=5
   )

   print(f"Thread ID: {simulation['thread_id']}")
   print(f"Conversation: {simulation['conversation_history']}")

Integration with Evaluation
---------------------------

Simulations work seamlessly with Opik's evaluation framework:

.. code-block:: python

   from opik.evaluation import evaluate_threads
   from opik.evaluation.metrics import ConversationThreadMetric

   # Run multiple simulations
   simulations = []
   for persona in ["frustrated_user", "happy_customer", "confused_user"]:
       simulator = SimulatedUser(persona=f"You are a {persona}")
       simulation = run_simulation(
           app=my_agent,
           user_simulator=simulator,
           max_turns=5
       )
       simulations.append(simulation)

   # Evaluate the threads
   results = evaluate_threads(
       project_name="my_project",
       filter_string='tags contains "simulation"',
       metrics=[ConversationThreadMetric()]
   )

For more detailed examples and advanced usage patterns, see the individual component documentation.