chore: import upstream snapshot with attribution

docs/tutorials/debug.md

@@ -0,0 +1,326 @@
+# Debugging and Troubleshooting
+
+When you train your own agent with Agent-lightning, most failures surface because the agent logic is brittle or simply incorrect. Debugging becomes easier when you peel back the stack: start by driving the rollout logic on its own, dry-run the trainer loop, and only then bring the full algorithm and runner topology online. The [`examples/apo/apo_debug.py`]({{ src("examples/apo/apo_debug.py") }}) script demonstrates these techniques; this guide expands on each approach and helps you decide when to reach for them.
+
+## Debugging with Dashboard
+
+When you launch an experiment with [`Trainer.fit`][agentlightning.Trainer.fit] or start an isolated store via [`agl store`](../reference/cli.md), the terminal prints a message similar to:
+
+```text
+INFO     Agent-lightning dashboard will be available at http://192.168.0.107:4747
+```
+
+Visit that URL, and you will see the Agent-lightning dashboard:
+
+![Dashboard](../assets/dashboard-page-rollouts.png)
+
+The dashboard surfaces everything stored inside [the store](../deep-dive/store.md). Because the store mediates interactions between algorithms and runners, inspecting it often reveals which side is causing issues such as stale rollouts, unresponsive workers, or empty traces.
+
+For example, the VERL algorithm may receive no token IDs and emit `cannot reshape tensor of 0 elements into shape [1, 0, -1, 128] because the unspecified dimension size -1 can be any value and is ambiguous` ([Issue #50](https://github.com/microsoft/agent-lightning/issues/50), [Issue #76](https://github.com/microsoft/agent-lightning/issues/76)). Several scenarios can produce that error: the runner might not produce trace spans at all, it might produce spans without token IDs, or the IDs may be present but formatted incorrectly. Inspecting the dashboard traces helps you pinpoint which condition applies.
+
+![Dashboard Traces Page](../assets/dashboard-page-traces.png)
+
+By checking whether the trace span is empty and whether token IDs appear in the span attributes, you can narrow the issue to either the runner (agent) side or the algorithm side. Then apply the techniques below to debug the faulty component.
+
+## Debug-level Logging
+
+Starting from v0.3, detailed signals such as store server access logs, runner lifecycle logs, and span payloads only appear when the log level is `DEBUG` so the default output stays readable. Enable debug-level logging by adding the following snippet near the top of your script:
+
+```python
+import agentlightning as agl
+
+agl.setup_logging("DEBUG")
+```
+
+Set the log level on every process if your setup involves multiple workers. For example, when [running stores in isolation][debug-with-external-store], configure the store process explicitly:
+
+```bash
+agl store --port 4747 --log-level DEBUG
+```
+
+## Using [`Runner`][agentlightning.Runner] in Isolation
+
+[`Runner`][agentlightning.Runner] is a long-lived worker that wraps your [`LitAgent`][agentlightning.LitAgent], coordinates tracing, and talks to the [`LightningStore`][agentlightning.LightningStore]. In typical training flows the trainer manages runners for you, but being able to spin one up manually is invaluable while debugging.
+
+If you define rollout logic with [`@rollout`][agentlightning.rollout] or implement a [`LitAgent`][agentlightning.LitAgent] directly, you will get a [`LitAgent`][agentlightning.LitAgent] instance and you should be able to execute it with [`LitAgentRunner`][agentlightning.LitAgentRunner], which is a subclass of [`Runner`][agentlightning.Runner]. The runner needs but does not instantiate a [`Tracer`][agentlightning.Tracer], so supply one yourself. See [Working with Traces](./traces.md) for a walkthrough of tracer options.
+
+[`Runner.run_context`][agentlightning.Runner.run_context] prepares the runner to execute a particular agent. Besides the agent and tracer you must provide a store that will collect spans and rollouts. [`InMemoryLightningStore`][agentlightning.InMemoryLightningStore] keeps everything in-process, which is perfect for debugging sessions.
+
+```python
+import agentlightning as agl
+
+tracer = agl.OtelTracer()
+runner = agl.LitAgentRunner(tracer)
+store = agl.InMemoryLightningStore()
+
+with runner.run_context(agent=apo_rollout, store=store):
+    ...
+```
+
+Inside the [`run_context`][agentlightning.Runner.run_context] block you can call [`runner.step(...)`][agentlightning.Runner.step] to execute a single rollout. The payload includes the task input and any [`NamedResources`][agentlightning.NamedResources] the agent expects. Read [introduction to Resources][introduction-to-resources] and [NamedResources][introduction-to-named-resources] for more details. For example, if your agent references a [`PromptTemplate`][agentlightning.PromptTemplate], pass it through the `resources` argument:
+
+```python
+with runner.run_context(agent=apo_rollout, store=store):
+    resource = agl.PromptTemplate(template="You are a helpful assistant. {any_question}", engine="f-string")
+    rollout = await runner.step(
+        "Explain why the sky appears blue using principles of light scattering in 100 words.",
+        resources={"main_prompt": resource},
+    )
+```
+
+You can do as many things as you want within the [`Runner.run_context`][agentlightning.Runner.run_context] block. After the rollout finishes you can query the store to inspect what happened:
+
+```python
+print(await store.query_rollouts())
+print(await store.query_spans(rollout.rollout_id))
+```
+
+Example output (with a reward span captured):
+
+```python
+[Rollout(rollout_id='ro-519769241af8', input='Explain why the sky appears blue using principles of light scattering in 100 words.', start_time=1760706315.6996238, ..., status='succeeded')]
+[Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=1, ..., name='agentlightning.annotation', attributes={'agentlightning.reward.0.value': 0.95}, ...)]
+```
+
+Swap in an [`AgentOpsTracer`][agentlightning.AgentOpsTracer] instead of [`OtelTracer`][agentlightning.OtelTracer] to see the underlying LLM spans alongside reward information:
+
+```python
+[
+    Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=1, ..., name='openai.chat.completion', attributes={..., 'gen_ai.prompt.0.role': 'user', 'gen_ai.prompt.0.content': 'You are a helpful assistant. Explain why the sky appears blue using principles of light scattering in 100 words.', ...}),
+    Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=2, ..., name='openai.chat.completion', attributes={..., 'gen_ai.prompt.0.role': 'user', 'gen_ai.prompt.0.content': 'Evaluate how well the output fulfills the task...', ...}),
+    Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=3, ..., name='agentlightning.annotation', attributes={'agentlightning.reward.0.value': 0.95}, ...)
+]
+```
+
+!!! tip
+
+    Spans too difficult to read? Try using [`Adapter`][agentlightning.Adapter] to convert them into a [more readable format](./traces.md).
+
+[`Runner.step`][agentlightning.Runner.step] executes a full rollout even though it is named "step". The companion method [`Runner.iter`][agentlightning.Runner.iter] executes multiple "steps" by continuously pulling new rollout inputs from the store until a stop event is set. Use `iter` once you are confident the single-step path works and you have another worker [`enqueue_rollout`][agentlightning.LightningStore.enqueue_rollout] to the store.
+
+!!! tip
+
+    You can also call [`Runner.step`][agentlightning.Runner.step] to inject ad-hoc rollouts into a running store being used by another algorithm, so that the rollouts can be consumed by the algorithms. This is very recently known as the paradigm of ["online RL"](https://cursor.com/blog/tab-rl). At the moment, no algorithm in the [algorithm zoo](../algorithm-zoo/index.md) consumes externally generated rollouts, but the data flow is available there if you need it.
+
+## Debug with LLM Proxy
+
+If you are dealing with LLM optimization like Reinforcement Learning, we generally recommend using an online stable LLM service for your debugging purposes, like `openai/gpt-4.1-nano`. After the debugging is done, you can switch to a local training endpoint.
+
+However, if you want to use a local LLM features like [getting the token IDs](../deep-dive/serving-llm.md), you can also manually start a local vLLM server by:
+
+```bash
+vllm serve Qwen/Qwen2.5-0.5B-Instruct --port 8080
+```
+
+Then start the LLM proxy via the following script:
+
+```python
+import asyncio
+import aiohttp
+import agentlightning as agl
+
+async def serve_llm_proxy():
+    store = agl.InMemoryLightningStore()
+    store_server = agl.LightningStoreServer(store, "127.0.0.1", 8081)
+    await store_server.start()
+
+    llm_proxy = agl.LLMProxy(
+        port=8082,
+        model_list=[
+            {
+                "model_name": "Qwen/Qwen2.5-0.5B-Instruct",
+                "litellm_params": {
+                    "model": "hosted_vllm/Qwen/Qwen2.5-0.5B-Instruct",
+                    "api_base": "http://localhost:8080/v1",
+                },
+            }
+        ],
+        store=store_server,
+    )
+
+    await llm_proxy.start()
+    await asyncio.sleep(1000000)
+```
+
+Test the served LLM proxy with a client like:
+
+```python
+async def test_llm_proxy():
+    async with aiohttp.ClientSession() as session:
+        async with session.post("http://localhost:8082/v1/chat/completions", json={
+            "model": "Qwen/Qwen2.5-0.5B-Instruct",
+            "messages": [{"role": "user", "content": "Hello, world!"}],
+        }) as response:
+            print(await response.json())
+```
+
+You can now use the LLM proxy by specifying environment variables:
+
+```bash
+export OPENAI_API_BASE=http://localhost:8081/v1
+export OPENAI_API_KEY=dummy
+```
+
+You might see warnings about `Missing or invalid rollout_id, attempt_id, or sequence_id` in the LLM proxy logs. This is fine because you don't have a rollout and attempt yet when you are debugging. When you started the training, the algorithm will create the rollouts for you and the warnings will go away.
+
+## Hook into Runner's Lifecycle
+
+[`Runner.run_context`][agentlightning.Runner.run_context] accepts a `hooks` argument so you can observe or augment lifecycle events without editing your agent. Hooks subclass [`Hook`][agentlightning.Hook] and can respond to four asynchronous callbacks: [`on_trace_start`][agentlightning.Hook.on_trace_start], [`on_rollout_start`][agentlightning.Hook.on_rollout_start], [`on_rollout_end`][agentlightning.Hook.on_rollout_end], and [`on_trace_end`][agentlightning.Hook.on_trace_end]. This is useful for:
+
+- Capturing raw OpenTelemetry spans before they hit the store and before the [`LitAgentRunner`][agentlightning.LitAgentRunner] do postprocessing on the rollout
+- Inspecting the tracer instance after they are activated
+- Logging rollout inputs before they are processed by the agent
+
+The `hook` mode in [`examples/apo/apo_debug.py`]({{ src("examples/apo/apo_debug.py") }}) prints every span collected during a rollout:
+
+```python
+import agentlightning as agl
+
+# ... Same as previous example
+
+class DebugHook(agl.Hook):
+    async def on_trace_end(self, *, agent, runner, tracer, rollout):
+        trace = tracer.get_last_trace()
+        print("Trace spans collected during the rollout:")
+        for span in trace:
+            print(f"- {span.name} (status: {span.status}):\n  {span.attributes}")
+
+with runner.run_context(
+    agent=apo_rollout,
+    store=store,
+    hooks=[DebugHook()],
+):
+    await runner.step(
+        "Explain why the sky appears blue using principles of light scattering in 100 words.",
+        resources={"main_prompt": resource},
+    )
+```
+
+Because hooks run inside the runner process you can also attach debuggers or breakpoints directly in the callback implementations.
+
+!!! note
+
+    For a better understanding of where hooks are called, we show a pseudo code of Runner's working flow below:
+
+    ```python
+    resources = await store.get_latest_resources()
+    rollout = ...
+    try:
+        # <-- on_rollout_start
+        with tracer.trace_context(...):
+            # <--- on_trace_start
+            result = await agent.rollout(...)
+            # <--- on_trace_end
+        post_process_result(result)
+    except Exception:
+        # <-- on_rollout_end
+        await store.update_attempt(status=...)
+    ```
+
+## Dry-Run the Trainer Loop
+
+Once single rollouts behave, switch to the trainer’s dry-run mode. [`Trainer.dev`][agentlightning.Trainer.dev] spins up a lightweight fast algorithm — [`agentlightning.Baseline`][agentlightning.Baseline] by default — so you can exercise the same infrastructure as [`Trainer.fit`][agentlightning.Trainer.fit] without standing up complex stacks like RL or SFT.
+
+!!! warning
+    When you enable multiple runners via `n_runners`, the trainer may execute them in separate worker processes. Attaching a debugger such as `pdb` is only practical when `n_runners=1`, and even then the runner might not live in the main process.
+
+```python
+import agentlightning as agl
+
+dataset: agl.Dataset[str] = [
+    "Explain why the sky appears blue using principles of light scattering in 100 words.",
+    "What's the capital of France?",
+]
+resource = agl.PromptTemplate(template="You are a helpful assistant. {any_question}", engine="f-string")
+
+trainer = agl.Trainer(
+    n_runners=1,
+    initial_resources={"main_prompt": resource},
+)
+trainer.dev(apo_rollout, dataset)
+```
+
+Just like [`Runner.run_context`][agentlightning.Runner.run_context], [`Trainer.dev`][agentlightning.Trainer.dev] requires the [`NamedResources`][agentlightning.NamedResources] your agent expects. The key difference is that resources are attached to the trainer rather than the runner.
+
+[`Trainer.dev`][agentlightning.Trainer.dev] uses an almost switchable interface from [`Trainer.fit`][agentlightning.Trainer.fit]. It also needs a dataset to iterate over, similar to [`fit`][agentlightning.Trainer.fit]. Under the hood [`dev`][agentlightning.Trainer.dev] uses the same implementation as [`fit`][agentlightning.Trainer.fit], which means you can spin up multiple runners, observe scheduler behavior, and validate how algorithms adapt rollouts. The default [`Baseline`][agentlightning.Baseline] logs detailed traces so you can see each rollout as the algorithm perceives it:
+
+```text
+21:20:30 Initial resources set: {'main_prompt': PromptTemplate(resource_type='prompt_template', template='You are a helpful assistant. {any_question}', engine='f-string')}
+21:20:30 Proceeding epoch 1/1.
+21:20:30 Enqueued rollout ro-302fb202bd85 in train mode with sample: Explain why the sky appears blue using principles of light scattering in 100 words.
+21:20:30 Enqueued rollout ro-e65a3ffaa540 in train mode with sample: What's the capital of France?
+21:20:30 Waiting for 2 harvest tasks to complete...
+21:20:30 [Rollout ro-302fb202bd85] Status is initialized to queuing.
+21:20:30 [Rollout ro-e65a3ffaa540] Status is initialized to queuing.
+21:20:35 [Rollout ro-302fb202bd85] Finished with status succeeded in 3.80 seconds.
+21:20:35 [Rollout ro-302fb202bd85 | Attempt 1] ID: at-f84ad21c. Status: succeeded. Worker: Worker-0
+21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span 3a286a856af6bea8] #1 (openai.chat.completion) ... 1.95 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...]
+21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span e2f44b775e058dd6] #2 (openai.chat.completion) ... 1.24 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...]
+21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span 45ee3c94fa1070ec] #3 (agentlightning.annotation) ... 0.00 seconds. Attribute keys: ['agentlightning.reward.0.value']
+21:20:35 [Rollout ro-302fb202bd85] Adapted data: [Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=None, metadata={'response_id': '...', 'agent_name': ''}), Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=0.95, metadata={'response_id': '...', 'agent_name': ''})]
+21:20:35 Finished 1 rollouts.
+21:20:35 [Rollout ro-e65a3ffaa540] Status changed to preparing.
+21:20:40 [Rollout ro-e65a3ffaa540] Finished with status succeeded in 6.39 seconds.
+21:20:40 [Rollout ro-e65a3ffaa540 | Attempt 1] ID: at-eaefa5d4. Status: succeeded. Worker: Worker-0
+21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 901dd6acc0f50147] #1 (openai.chat.completion) ... 1.30 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...]
+21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 52e0aa63e02be611] #2 (openai.chat.completion) ... 1.26 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...]
+21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 6c452de193fbffd3] #3 (agentlightning.annotation) ... 0.00 seconds. Attribute keys: ['agentlightning.reward.0.value']
+21:20:40 [Rollout ro-e65a3ffaa540] Adapted data: [Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=None, metadata={'response_id': '...', 'agent_name': ''}), Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=1.0, metadata={'response_id': '...', 'agent_name': ''})]
+21:20:40 Finished 2 rollouts.
+```
+
+The only limitation is that resources remain static and components like [`LLMProxy`][agentlightning.LLMProxy] are not wired in. For richer dry runs you can subclass [`FastAlgorithm`][agentlightning.FastAlgorithm] and override the pieces you care about.
+
+## Debug the Algorithm-Runner Boundary
+
+[](){ #debug-with-external-store }
+
+Debugging algorithms in Agent-Lightning is often more challenging than debugging agents. Algorithms are typically **stateful** and depend on several moving parts — runners, stores, and trainers — which makes it difficult to isolate and inspect their behavior. Even mocking an agent to cooperate with an algorithm can be costly and error-prone. To simplify this, Agent-Lightning provides a way to run algorithms in isolation so you can attach a debugger and inspect internal state without interference from other components.
+
+By default, [`Trainer.fit`][agentlightning.Trainer.fit] runs the algorithm in the main process and thread, but its logs are interleaved with those from the store and runners, making it hard to follow what’s happening inside the algorithm itself. In [*Write Your First Algorithm*](../how-to/write-first-algorithm.md), we covered how to stand up a store, algorithm, and runner in isolation for your own implementations. This section extends that approach to cover two common questions:
+
+1. How can I run built-in or class-based algorithms (inheriting from [`Algorithm`][agentlightning.Algorithm]) in isolation?
+2. How can I still use [`Trainer`][agentlightning.Trainer] features like `n_runners`, `adapter`, or `llm_proxy` while debugging?
+
+The solution is to keep using a [`Trainer`][agentlightning.Trainer] instance but **manage the store yourself**, running the algorithm and runner roles separately. This approach mirrors the internal process orchestration of [`Trainer.fit`][agentlightning.Trainer.fit], but with more visibility and control. Below, we show a step-by-step guide to achieve this with the [`calc_agent` example]({{ src("examples/calc_x/train_calc_agent.py") }}).
+
+**1. Launch the store manually.**
+In a separate terminal, start the store:
+
+```bash
+agl store --port 4747
+```
+
+Add `--log-level DEBUG` to the command to see the detailed logs.
+
+Then, in your training script, create a [`LightningStoreClient`][agentlightning.LightningStoreClient] and pass it to the trainer:
+
+```python
+client = agl.LightningStoreClient("http://localhost:4747")
+trainer = agl.Trainer(store=client, ...)
+```
+
+Set the environment variable `AGL_MANAGED_STORE=0` so the trainer doesn't attempt to manage the store automatically.
+
+**2. Start the runner and algorithm processes separately.**
+Each process should run the same training script, but with different environment variables specifying the current role.
+This setup faithfully mirrors how [`Trainer.fit`][agentlightning.Trainer.fit] orchestrates these components behind the scenes.
+
+```bash
+# Terminal 2 – Runner process
+AGL_MANAGED_STORE=0 AGL_CURRENT_ROLE=runner \
+    python train_calc_agent.py --external-store-address http://localhost:4747 --val-file data/test_mini.parquet
+
+# Terminal 3 – Algorithm process
+AGL_MANAGED_STORE=0 AGL_CURRENT_ROLE=algorithm \
+    python train_calc_agent.py --external-store-address http://localhost:4747 --val-file data/test_mini.parquet
+```
+
+**3. Reuse your existing trainer configuration.**
+You can continue using the same datasets, adapters, and proxies as usual. Because the store is now external, you can:
+
+* Attach debuggers to either the algorithm or runner process
+* Add fine-grained logging or tracing
+* Simulate partial failures or latency in individual components
+
+This setup provides a faithful reproduction of the algorithm–runner interaction while keeping the store visible for inspection. Once you’ve resolved the issue, simply set `AGL_MANAGED_STORE=1` (or omit it) to return to the standard managed training workflow.

docs/tutorials/emitter.md

@@ -0,0 +1,219 @@
+# Using Emitters
+
+[](){ #using-emitter }
+
+While returning a single float for the final reward is sufficient for many algorithm-agent combinations, some advanced scenarios require richer feedback. For instance, an algorithm might learn more effectively if it receives intermediate rewards throughout a multi-step task, or if the agent needs to emit additional spans for debugging or analysis.
+
+Agent-lightning provides an **emitter** module for recording custom spans inside your agent logic. Just as [Tracer][agentlightning.Tracer] automatically instruments common operations (for example, LLM calls), each emitter helper sends a [Span][agentlightning.Span] that captures Agent-lightning-specific work so downstream algorithms can query it later. See [Working with Traces](./traces.md) for more details.
+
+For multi-step routines such as function calls, tools, or adapters, wrap code with [`operation`][agentlightning.operation] — either as a decorator or a context manager — to capture inputs, outputs, and metadata on a dedicated [`operation`][agentlightning.operation] span. This makes it easier to correlate downstream annotations (like rewards or messages) with the higher-level work that produced them.
+
+You can find the emitter functions in [`agentlightning.emitter`](../reference/agent.md).
+
+## Emitting Rewards, Messages, and More
+
+Here are the primary emitter functions:
+
+* [`emit_reward(value: float)`][agentlightning.emit_reward]: Records an intermediate/final reward, which is a convenient wrapper of [`emit_annotation`][agentlightning.emit_annotation].
+* [`emit_annotation(attributes: Dict[str, Any])`][agentlightning.emit_annotation]: Records arbitrary metadata as a span.
+* [`emit_message(message: str)`][agentlightning.emit_message]: Records a simple log message as a span.
+* [`emit_exception(exception: BaseException)`][agentlightning.emit_exception]: Records a Python exception, including its type, message, and stack trace.
+* [`emit_object(obj: Any)`][agentlightning.emit_object]: Records any JSON-serializable object, perfect for structured data.
+
+Let's first see an example of an agent using these emitters to provide detailed feedback.
+
+```python
+import agentlightning as agl
+
+@agl.rollout
+def multi_step_agent(task: dict, prompt_template: PromptTemplate) -> float:
+    try:
+        # Step 1: Initial planning
+        agl.emit_message("Starting planning phase.")
+        plan = generate_plan(task, prompt_template)
+        agl.emit_object({"plan_steps": len(plan), "first_step": plan[0]})
+
+        # Award a small reward for a valid plan
+        plan_reward = grade_plan(plan)
+        agl.emit_reward(plan_reward)
+
+        # Step 2: Execute the plan
+        agl.emit_message(f"Executing {len(plan)}-step plan.")
+        execution_result = execute_plan(plan)
+
+        # Step 3: Final evaluation
+        final_reward = custom_grade_final_result(execution_result, task["expected_output"])
+
+        # The return value is treated as the final reward for the rollout
+        return final_reward
+
+    except ValueError as e:
+        # Record the specific error and return a failure reward
+        agl.emit_exception(e)
+        return 0.0
+```
+
+Each helper accepts nested `attributes` (or keyword arguments for [`operation`][agentlightning.operation]) and automatically flattens/sanitizes them into dotted OpenTelemetry keys. This means you can pass ordinary dictionaries/lists without pre-processing and still get consistent attribute names such as `meta.any_attribute` across all emitter operations. Agent-lightning does not restrict the attributes you supply, but it is best to consult [OpenTelemetry's semantic conventions](https://opentelemetry.io/docs/specs/semconv/) for recommended names. Agent-lightning also defines [specific semconv](../reference/semconv.md) for its own use cases. The pattern looks like this:
+
+```python
+from opentelemetry.semconv.attributes import server_attributes
+from agentlightning import emit_object
+
+emit_object({
+    "name": "John Doe",
+    "age": 30,
+    "email": "john.doe@example.com",
+}, attributes={
+    server_attributes.SERVER_ADDRESS: "127.0.0.1",
+    server_attributes.SERVER_PORT: 8080,
+})
+```
+
+Running the above code sends the following span to the backend if you have a tracer active:
+
+```text
+Span(
+    name='agentlightning.object',
+    attributes={
+        'agentlightning.object.type': 'dict',
+        'agentlightning.object.json': '{"name": "John Doe", "age": 30, "email": "john.doe@example.com"}',
+        'server.address': '127.0.0.1',
+        'server.port': 8080
+    }
+)
+```
+
+!!! tip
+
+    If you don't have a tracer active, the above code will raise the following error:
+
+    ```text
+    RuntimeError: No active tracer found. Cannot emit object span.
+    ```
+
+    By default, emitter helpers delegate to the active tracer to create and export spans (specifically via [`Tracer.create_span`][agentlightning.Tracer.create_span]). If you want to emit spans without an active tracer, set `propagate=False` to keep the span local — a useful option for offline tests. The default `True` streams spans through the active tracer/exporters.
+
+When working with [agentlightning.semconv](../reference/semconv.md), you typically use utilities such as [`make_tag_attributes`][agentlightning.utils.otel.make_tag_attributes] and [`make_link_attributes`][agentlightning.utils.otel.make_link_attributes] to build the attributes dictionary. For example:
+
+```python
+from agentlightning.utils.otel import make_tag_attributes
+
+emit_annotation(make_tag_attributes(["tool", "calculator", "fast", "good"]))
+```
+
+The above code will send a span with the following attributes to the backend:
+
+```json
+{
+    "agentlightning.tag.0": "tool",
+    "agentlightning.tag.1": "calculator",
+    "agentlightning.tag.2": "fast",
+    "agentlightning.tag.3": "good"
+}
+```
+
+A counterpart utility function [`extract_tags_from_attributes`][agentlightning.utils.otel.extract_tags_from_attributes] is also available to extract the tags from the attributes dictionary.
+
+## Operations
+
+The [`operation`][agentlightning.operation] helper tracks logical units of work within your agent, capturing inputs, outputs, timing, and success/failure status. Unlike point-in-time emitters, operations create a span representing a time interval. Use operations for tool calls, multi-step workflows, debugging, and performance monitoring. [`operation`][agentlightning.operation] works as either a decorator or a context manager.
+
+The decorator automatically captures function arguments as inputs and the return value as output:
+
+```python
+import agentlightning as agl
+
+@agl.operation
+def search_documents(query: str, max_results: int = 10) -> list[dict]:
+    results = perform_search(query, max_results)
+    return results
+
+@agl.operation(category="tool", priority="high")
+def execute_calculation(expression: str) -> float:
+    return eval_safely(expression)
+```
+
+The example above emits a span with `{"category": "tool", "priority": "high"}` attributes. It also records the function input and output via [OPERATION_INPUT][agentlightning.semconv.LightningSpanAttributes.OPERATION_INPUT] and [OPERATION_OUTPUT][agentlightning.semconv.LightningSpanAttributes.OPERATION_OUTPUT]. It works with async functions too:
+
+```python
+@agl.operation
+async def async_api_call(endpoint: str, payload: dict) -> dict:
+    response = await http_client.post(endpoint, json=payload)
+    return response.json()
+```
+
+Override the operation name if needed:
+
+```python
+@agl.operation(name="custom-name")
+def any_weird_name_i_dont_want():
+    pass
+```
+
+For more control, [`operation`][agentlightning.operation] can also be used as a context manager to explicitly record inputs and outputs:
+
+```python
+with agl.operation(tool_name="web_search") as op:
+    op.set_input(query="latest AI research", filters={"date": "2024"})
+    results = search_web("latest AI research", {"date": "2024"})
+    op.set_output({"result_count": len(results), "top_result": results[0]})
+```
+
+The `propagate=False` flag also applies to [`operation`][agentlightning.operation] when you want to keep operations local without requiring an active tracer:
+
+```python
+@agl.operation(propagate=False)
+def local_test():
+    return "Not sent to backend"
+```
+
+## Linking to Other Spans
+
+Sometimes a span should explicitly point back to another span that produced the input it is working on (for example, linking a reward annotation to the [`agentlightning.operation`][agentlightning.operation] span that generated a response). Agent-lightning encodes these relationships through flattened link attributes. The helper [`make_link_attributes`][agentlightning.utils.otel.make_link_attributes] converts a dictionary of keys such as `trace_id`, `span_id`, or any custom attribute into the `"agentlightning.link.*"` ([LightningSpanAttributes.LINK][agentlightning.semconv.LightningSpanAttributes.LINK]) fields expected by the backend. Later, [`query_linked_spans`][agentlightning.utils.otel.query_linked_spans] can recover the original span(s) from those link descriptors.
+
+```python
+import opentelemetry.trace as trace_api
+from agentlightning import emit_annotation, operation
+from agentlightning.utils.otel import make_link_attributes, make_tag_attributes
+
+with operation(conversation_id="chat-42") as op:
+    # ... perform the work ...
+    link_attrs = make_link_attributes({
+        "conversation_id": "chat-42",
+    })
+
+    emit_annotation(
+        {
+            **link_attrs,
+            **make_tag_attributes(["reward", "good"]),
+        }
+    )
+```
+
+When analyzing in adapters, pass the extracted link models to [`query_linked_spans`][agentlightning.utils.otel.query_linked_spans] to retrieve the matching span(s):
+
+```python
+from agentlightning.utils.otel import extract_links_from_attributes, query_linked_spans
+
+annotation_span = ...  # Span from your trace store
+operation_spans = [...]  # list of spans you want to search
+
+link_models = extract_links_from_attributes(annotation_span.attributes)
+matches = query_linked_spans(operation_spans, link_models)
+assert matches  # Contains the original operation span
+```
+
+!!! tip "Correlating Rewards with LLM Requests"
+
+    [Tracer](./traces.md) instruments each request/response as its own span. You can link to the [`gen_ai.response.id`](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/) attribute, which comes from the LLM response ID.
+
+    ```python
+    from agentlightning import emit_reward
+    from agentlightning.utils.otel import make_link_attributes
+
+    result = call_llm(prompt)
+    reward_links = make_link_attributes({"gen_ai.response.id": result.id})
+    emit_reward(0.9, attributes=reward_links)
+    ```
+
+    Later, use the same `gen_ai.response.id` key inside `query_linked_spans` to find the reward(s) that reference that specific LLM request span.

docs/tutorials/installation.md

@@ -0,0 +1,190 @@
+# Installation Guide
+
+This guide explains how to install **Agent-Lightning**. You can install it from **PyPI** (the Python Package Index) for general use or directly from the **source code** if you plan to contribute or need fine-grained control over dependencies.
+
+!!! info "Platform and Hardware Requirements"
+    Agent-Lightning is officially supported on **Linux distributions** (Ubuntu 22.04 or later is recommended).
+    At the moment **macOS and Windows** (outside of WSL2) are not supported.
+
+    The Python runtime must be **Python 3.10 or newer**. We recommend using the latest patch release of Python 3.10, 3.11, or 3.12 to pick up performance and security updates.
+
+    A **GPU is optional**—you only need CUDA-capable hardware if you plan to fine-tune model weights or run GPU-accelerated workloads. CPU-only environments are fully supported for evaluation and inference.
+
+## Installing from PyPI
+
+The easiest way to get started is by installing Agent-Lightning directly from PyPI. This ensures you get the latest **stable release** of the package, tested for compatibility and reliability.
+
+### Install the Stable Release
+
+Run the following command in your terminal:
+
+```bash
+pip install --upgrade agentlightning
+```
+
+This installs or upgrades Agent-Lightning to the newest stable version.
+
+!!! tip
+
+    If you intend to use **Agent-Lightning** with [**VERL**](../algorithm-zoo/verl.md) or run any of its **example scripts**, you’ll need to install some additional dependencies.
+    See the sections on [Algorithm-specific installation](#algorithm-specific-installation) and [Example-specific installation](#example-specific-installation) for details.
+
+### Install the Nightly Build (Latest Features)
+
+Agent-Lightning also publishes **nightly builds**, which contain the latest experimental features and improvements from the main branch. These are available via **Test PyPI**.
+
+```bash
+pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --pre agentlightning
+```
+
+!!! warning
+    The nightly builds are cutting-edge but may include unstable or untested changes.
+    Use them **at your own risk**, especially in production environments.
+
+## Algorithm-specific Installation
+
+Agent-Lightning supports multiple learning algorithms. Some of them like [APO](../algorithm-zoo/apo.md) or [VERL](../algorithm-zoo/verl.md) require extra dependencies. You can install them automatically using **optional extras** or manually if you prefer finer control.
+
+### Installing APO
+
+[APO](../algorithm-zoo/apo.md) is an algorithm module that depends on libraries such as [POML](https://github.com/microsoft/POML).
+You can install Agent-Lightning with APO support by running:
+
+```bash
+pip install agentlightning[apo]
+```
+
+!!! warning
+    APO also depends on the [OpenAI Python SDK](https://github.com/openai/openai-python), version **2.0 or newer**.
+    Ensure your SDK version is up to date to avoid compatibility issues.
+
+### Installing VERL
+
+[VERL](../algorithm-zoo/verl.md) integrates with libraries like **PyTorch**, **vLLM**, and **VERL framework**.
+Although you *can* install all dependencies automatically, we recommend doing it manually to avoid version conflicts.
+
+```bash
+pip install agentlightning[verl]
+```
+
+!!! tip "Recommended Manual Setup (More Stable)"
+    Automated installation may cause issues if you don’t have a compatible **PyTorch** or **CUDA** version preinstalled.
+    For a more stable setup, install dependencies step-by-step:
+
+    ```bash
+    pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128
+    pip install flash-attn --no-build-isolation
+    pip install vllm==0.10.2
+    pip install verl==0.5.0
+    ```
+
+    This approach ensures compatibility with CUDA 12.8 and minimizes dependency conflicts.
+
+## Example-specific Installation
+
+Each example in the `examples/` directory may have its own additional dependencies.
+Please refer to the **README** file of each example for detailed setup instructions:
+
+[See Example READMEs]({{ src("examples") }}).
+
+## Installing from Source (for Developers and Contributors)
+
+If you plan to contribute to Agent-Lightning or prefer to work with the latest development code, install it directly from the **source repository**.
+
+### Why Install from Source?
+
+* You want to **modify or contribute** to the project.
+* You prefer an **isolated development environment**.
+* You want to test unreleased features or fix bugs locally.
+
+### Using `uv` for Dependency Management
+
+Starting with version **0.2**, Agent-Lightning uses [`uv`](https://docs.astral.sh/uv/) as its **default dependency manager**.
+
+`uv` is a fast and safe alternative to `pip` that:
+
+* Installs packages **in seconds** (instead of minutes),
+* Prevents **dependency conflicts**,
+* Supports **grouped dependencies** for optional features.
+
+Before proceeding, make sure `uv` is installed.
+
+### Minimal Developer Installation
+
+```bash
+git clone https://github.com/microsoft/agent-lightning
+cd agent-lightning
+uv sync --group dev
+```
+
+This command sets up a clean development environment with only the essential dependencies.
+
+### Installing All Extras (CPU or GPU)
+
+`uv sync` can also handle algorithm-specific and example-specific dependencies in one step.
+
+For a CPU-only machine:
+
+```bash
+uv sync --frozen \
+    --extra apo \
+    --extra verl \
+    --group dev \
+    --group torch-cpu \
+    --group torch-stable \
+    --group trl \
+    --group agents \
+    --no-default-groups
+```
+
+For a GPU-equipped machine that is CUDA 12.8 compatible:
+
+```bash
+uv sync --frozen \
+    --extra apo \
+    --extra verl \
+    --group dev \
+    --group torch-gpu-stable \
+    --group trl \
+    --group agents \
+    --no-default-groups
+```
+
+Read more about Agent-lightning managed dependency groups [here]({{ src("pyproject.toml") }}).
+
+### Building the Dashboard
+
+The Agent-Lightning dashboard is built using [Vite](https://vite.dev/). To build the dashboard, run the following command:
+
+```bash
+cd dashboard
+npm ci
+npm run build
+```
+
+Some HTML and JavaScript assets will be generated in the `agentlightning/dashboard` directory.
+
+### Activating Your Environment
+
+After syncing dependencies, `uv` automatically creates a virtual environment inside the `.venv/` directory.
+
+You can use it in two ways:
+
+```bash
+# Option 1: Prefix commands with uv run
+uv run python your_script.py
+
+# Option 2: Activate the virtual environment
+source .venv/bin/activate
+python your_script.py
+```
+
+!!! warning "Before Contributing"
+
+    Agent-Lightning enforces code style and linting rules via **pre-commit hooks**.
+    Installing them early prevents many avoidable formatting issues.
+
+    ```bash
+    uv run pre-commit install
+    uv run pre-commit run --all-files --show-diff-on-failure --color=always
+    ```

docs/tutorials/parallelize.md

@@ -0,0 +1,334 @@
+# Scaling out Agent-lightning
+
+Agent-lightning splits training into an **algorithm bundle** and a **runner bundle** that exchange work through the [`LightningStore`][agentlightning.LightningStore]. This tutorial shows how to increase rollout throughput, place bundles across processes or machines, and keep the algorithm side scalable with external frameworks.
+
+## Parallelizing Rollouts with [`Trainer`][agentlightning.Trainer]
+
+Before we dive into the details of the bundles and execution strategies, let's first revisit how to parallelize rollouts with [`Trainer`][agentlightning.Trainer].
+
+[`Trainer`][agentlightning.Trainer] is the quickest way to dial up parallelism. Even when `n_runners = 1`, calling [`Trainer.fit`][agentlightning.Trainer.fit] runs the algorithm and runners in parallel. The algorithm enqueues rollouts; runners dequeue them and execute your [`LitAgent`][agentlightning.LitAgent], and the algorithm collects spans via its [`Adapter`][agentlightning.Adapter] before scheduling the next batch.
+
+!!! note
+
+    One of the most important features of [`Trainer`][agentlightning.Trainer] is the ability to abort things gracefully. For example, if you press `Ctrl+C` in the terminal, the algorithm will abort and the runners will stop executing. If the algorithm crashes, the runners will also stop executing.
+
+Increase throughput by setting `n_runners` when constructing the trainer. The following example comes from [train_calc_agent.py]({{ src("examples/calc_x/train_calc_agent.py") }}). Since backend LLMs usually use techniques like [continuous batching](https://docs.vllm.ai/en/latest/) to increase throughput, you do not have to worry about overwhelming the backend with too many requests.
+
+```python
+import agentlightning as agl
+from datasets import Dataset as HFDataset
+from calc_agent import calc_agent
+
+train_dataset = HFDataset.from_parquet("data/train.parquet").to_list()
+val_dataset = HFDataset.from_parquet("data/test.parquet").to_list()
+
+algorithm = agl.VERL(verl_config)
+
+trainer = agl.Trainer(
+    algorithm=algorithm,
+    n_runners=8,  # launch eight rollout workers
+    tracer=agl.OtelTracer(),
+    adapter=agl.LlmProxyTraceToTriplet(),
+)
+
+trainer.fit(calc_agent, train_dataset=train_dataset, val_dataset=val_dataset)
+```
+
+In [`Trainer`][agentlightning.Trainer], there are multiple other initialization parameters that you can use to customize the training process. For example, you can use `max_rollouts` to keep smoke tests short. Pass a concrete [`LightningStore`][agentlightning.LightningStore] instance when you need persistence or want to share the queue across multiple scripts.
+
+!!! tip
+
+    Before scaling out, run [`Trainer.dev()`][agentlightning.Trainer.dev] with `n_runners=1` to verify the rollout logic and spans without burning GPU hours.
+
+## Bundles and Execution Strategies
+
+When [`Trainer`][agentlightning.Trainer] starts, it packages its configuration into two callable **bundles**:
+
+![Illustration of bundles and execution strategies](../assets/execution-bundles.svg)
+
+The **algorithm bundle** wraps your [`Algorithm`][agentlightning.Algorithm], adapter, and any LLM proxy into a single callable that can be aborted via a signal event.
+
+```python
+async def algorithm_bundle(store: LightningStore, event: ExecutionEvent) -> None:
+    ...
+```
+
+The **runner bundle** wraps the [`Runner`][agentlightning.Runner], tracer, hooks, and agent into a single callable that can be aborted via a signal event. Unlike the algorithm bundle, the runner bundle is expected to be replicated.
+
+```python
+async def runner_bundle(store: LightningStore, worker_id: int, event: ExecutionEvent) -> None:
+    ...
+```
+
+An **execution strategy** then decides where those bundles are placed (threads vs processes vs multiple machines), how many runner replicas to launch, and how lifecycle events such as shutdown are coordinated.
+
+By default, the trainer builds an [`InMemoryLightningStore`][agentlightning.InMemoryLightningStore] if you do not provide one. Because that store has no locking or cross-process transport, the execution strategy is the component that wraps it in thread-safe or HTTP-safe facades ([`LightningStoreThreaded`][agentlightning.LightningStoreThreaded], [`LightningStoreServer`][agentlightning.LightningStoreServer]) before handing it to bundles. For a deeper look at these facades, see [Understanding the Store](../deep-dive/store.md) and [Birds' Eye View](../deep-dive/birds-eye-view.md).
+
+Agent-lightning provides two built-in execution strategies: [`SharedMemoryExecutionStrategy`][agentlightning.SharedMemoryExecutionStrategy] and [`ClientServerExecutionStrategy`][agentlightning.ClientServerExecutionStrategy]. You can pass a string alias, a configuration dictionary, or a pre-built strategy instance:
+
+```python
+import agentlightning as agl
+
+algorithm = agl.Baseline()
+
+# Short alias for the shared-memory strategy.
+# Because the runner lives on the main thread in this mode,
+# n_runners must be 1 unless you move the algorithm to the main thread.
+trainer = agl.Trainer(algorithm=algorithm, n_runners=1, strategy="shm")
+
+# Dict with overrides; keep the algorithm on the main thread so multiple runner threads can spawn.
+# Specifying `n_runners` inside strategy is equivalent to passing `n_runners` to the trainer.
+trainer = agl.Trainer(
+    algorithm=algorithm,
+    strategy={
+        "type": "shm",
+        "n_runners": 8,
+        "main_thread": "algorithm",
+    },
+)
+
+# Pass an existing strategy instance – Trainer respects the strategy's own `n_runners`.
+strategy = agl.SharedMemoryExecutionStrategy(main_thread="algorithm", n_runners=4)
+trainer = agl.Trainer(algorithm=algorithm, strategy=strategy)
+```
+
+If you omit the strategy, the trainer defaults to `ClientServerExecutionStrategy(n_runners=trainer.n_runners)`. You can still re-specify the client-server strategy through aliases or configuration to tweak ports and other settings:
+
+```python
+trainer = agl.Trainer(
+    algorithm=algorithm,
+    n_runners=8,
+    strategy={"type": "cs", "server_port": 9999},
+)
+```
+
+Environment variables give you another layer of control. For example:
+
+```python
+import os
+
+os.environ["AGL_SERVER_PORT"] = "10000"
+os.environ["AGL_CURRENT_ROLE"] = "algorithm"
+os.environ["AGL_MANAGED_STORE"] = "0"
+
+trainer = agl.Trainer(algorithm=algorithm, n_runners=8, strategy="cs")
+```
+
+The resulting [`ClientServerExecutionStrategy`][agentlightning.ClientServerExecutionStrategy] picks up the port, role, and managed-store flag from the environment.
+
+!!! tip
+
+    The same configuration patterns apply to other trainer components. For example,
+    ```python
+    trainer = agl.Trainer(algorithm=algorithm, tracer=agl.OtelTracer())
+    ```
+    wires in a custom tracer, while
+    ```python
+    trainer = agl.Trainer(algorithm=algorithm, adapter="agentlightning.adapter.TraceToMessages")
+    ```
+    swaps in a different adapter. Passing a dict lets you tweak the init parameters of defaults without naming the class explicitly:
+
+    ```python
+    trainer = agl.Trainer(
+        algorithm=algorithm,
+        adapter={"agent_match": "plan_agent", "repair_hierarchy": False},
+    )
+    ```
+
+The next sections walk through the two built-in strategies and how they affect placement and store access.
+
+## Client-server Architecture
+
+The default [`ClientServerExecutionStrategy`][agentlightning.ClientServerExecutionStrategy] starts a [`LightningStoreServer`][agentlightning.LightningStoreServer] alongside the algorithm and spawns runner processes that talk to it through [`LightningStoreClient`][agentlightning.LightningStoreClient]. All runners share the HTTP endpoint, so the queue and spans stay consistent across processes or machines.
+
+If you simply instantiate [`Trainer`][agentlightning.Trainer] (as above), it will send the algorithm bundle and runner bundle to [`ClientServerExecutionStrategy`][agentlightning.ClientServerExecutionStrategy], which will then:
+
+1. Launch \(N+1\) processes: \(N\) runner processes and 1 algorithm process (one of them could live in the main process).
+2. The algorithm process will take the store received from [`Trainer`][agentlightning.Trainer], wrap it in a [`LightningStoreServer`][agentlightning.LightningStoreServer], and start serving it over HTTP.
+3. The runner processes discard the store and create a new store, which is a client that connects to the algorithm process through [`LightningStoreClient`][agentlightning.LightningStoreClient], and start executing the runner bundle.
+4. The strategy automatically escalates shutdown (cooperative stop → `SIGINT` → `terminate()` → `kill()`) so long-running runners do not linger.
+
+You can override server placement or ports, and whether to automatically wrap the store, through constructor arguments or environment variables:
+
+```python
+trainer = agl.Trainer(
+    algorithm=algorithm,
+    n_runners=1,
+    strategy={
+        "type": "cs",
+        "server_host": "0.0.0.0",
+        "server_port": 9999,
+        "main_process": "runner",
+    },
+)
+```
+
+Set `AGL_SERVER_HOST` and `AGL_SERVER_PORT` if you prefer environment-based configuration. You can also use `AGL_MANAGED_STORE` if you do not want the execution strategy to wrap the store for you. An example is shown in [Debugging with External Store][debug-with-external-store].
+
+Algorithms sometimes require heterogeneous computation resources, such as GPU accelerators, while runners sometimes require a specific environment to run because many agent frameworks are fragile in their dependencies. A role-based launch pattern helps you place the algorithm on a dedicated machine with more GPU memory, while runners can live on another machine with more flexible dependencies. This is possible via `AGL_CURRENT_ROLE="algorithm"` or `AGL_CURRENT_ROLE="runner"` environment variables. When running on different machines, you also need to set `AGL_SERVER_HOST` and `AGL_SERVER_PORT` to the IP address and port of the algorithm machine. You might recognize that this convention is very similar to `MASTER_ADDR` and `MASTER_PORT` in [PyTorch distributed training](https://docs.pytorch.org/docs/stable/notes/ddp.html).
+
+### Launching Algorithm and Runner Roles on Separate Machines
+
+When you want to stretch the algorithm onto a GPU-rich machine and keep rollout workers close to the data source (or on machines with a more permissive dependency stack), launch the same training script in different terminals with role-specific environment variables. The client–server strategy will route each process to the right side of the queue as long as they share the same `AGL_SERVER_HOST`/`AGL_SERVER_PORT` pair.
+
+**1. Pick an address and port for the store.** Decide which machine will host the algorithm. Choose a TCP port that can be reached by the runner machines (for example, open it in your firewall configuration). In this example we will use `10.0.0.4:4747`.
+
+**2. Start the algorithm process.** On the machine that should run the algorithm, expose the store by binding to all network interfaces and mark the role as `algorithm`.
+
+```bash
+export AGL_SERVER_HOST=0.0.0.0
+export AGL_SERVER_PORT=4747
+export AGL_CURRENT_ROLE=algorithm
+
+python train_calc_agent.py
+```
+
+Leaving `AGL_MANAGED_STORE` unset (or setting it to `1`) lets the strategy create the [`LightningStoreServer`][agentlightning.LightningStoreServer] for you. Otherwise, you can use the method in the previous section to create a store on your own.
+
+**3. Start rollout workers on remote machines.** Every runner machine should point to the algorithm host and declare itself as the `runner` role. You can start multiple processes per machine or repeat the command on additional hosts.
+
+```bash
+export AGL_SERVER_HOST=10.0.0.4
+export AGL_SERVER_PORT=4747
+export AGL_CURRENT_ROLE=runner
+python train_calc_agent.py --n-runners 4
+```
+
+The runner process automatically connects via [`LightningStoreClient`][agentlightning.LightningStoreClient]. Adjust `--n-runners` to spawn the desired number of worker processes on that machine.
+
+**4. Scale out as needed.** Repeat step 3 on as many machines as you need. When you are done, stop the algorithm process. However, since the runners are on different machines, the strategy WILL NOT send a cooperative stop signal to the connected runners. So you need to kill the runners on your own.
+
+This role-based launch mirrors what [`Trainer.fit`][agentlightning.Trainer.fit] does inside a single machine while letting you spread work across a fleet. Because every process shares the same training script, you keep a single source of truth for dataset loading, adapters, and tracers, but you can tune compute resources independently for the algorithm and rollout workers.
+
+### Shared-memory Strategy
+
+[`SharedMemoryExecutionStrategy`][agentlightning.SharedMemoryExecutionStrategy] keeps everything inside one process. The runner runs on the main thread (by default) while the algorithm lives on a Python thread guarded by [`LightningStoreThreaded`][agentlightning.LightningStoreThreaded].
+
+Use it when you want easier debugging with shared breakpoints and no serialization overhead, or minimal startup time for unit tests. It's not a good choice for many algorithms that require heavy model training because [`LightningStoreThreaded`][agentlightning.LightningStoreThreaded] does not work for multiprocessing. Using it with multiprocessing algorithms will lead to undefined behavior.
+
+Sample configuration:
+
+```python
+trainer = agl.Trainer(
+    algorithm=algorithm,
+    strategy="shm",
+)
+```
+
+You can further customize the init parameters of [`SharedMemoryExecutionStrategy`][agentlightning.SharedMemoryExecutionStrategy]. With `main_thread="runner"`, the runner occupies the main thread and `n_runners` must be `1`. The strategy respects `AGL_MANAGED_STORE`; set it to `0` to opt out of the `LightningStoreThreaded` wrapper.
+
+## Parallelizing Algorithms
+
+Runner parallelism scales rollout throughput, but the algorithm loop remains a single-process loop inside the execution strategy. We understand that many algorithms have parallelization built in, but that's outside the parallelization scope of Agent-lightning.
+
+Agent-lightning strives to make algorithms’ own parallelization work well under our execution strategies. The biggest challenge turns out to come from the store. For example, [`VERL`][agentlightning.algorithm.verl.VERL] uses [Ray](https://www.ray.io/) and launches [FSDP](https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html) and [vLLM](https://vllm.ai/) components internally. [`ClientServerExecutionStrategy`][agentlightning.ClientServerExecutionStrategy] has to make sure that the server is not simultaneously serving in multiple processes or Ray workers, and that there is only one single authoritative source of truth for all subprocesses to connect to. Subprocesses connect to the store via a small [`LightningStoreClient`][agentlightning.LightningStoreClient] bundled within [`LightningStoreServer`][agentlightning.LightningStoreServer].
+
+!!! note
+
+    The [birds' eye view][birds-eye-view-client-server-strategy] illustrates how adapters, proxies, and stores interact when the algorithm spawns additional workers. Use that diagram as a checklist when introducing new distributed components.
+
+## Parallelizing [`LightningStore`][agentlightning.LightningStore]
+
+By default, Agent-lightning persists rollouts and spans in an in-memory store. [`Trainer.fit`][agentlightning.Trainer.fit] spins it up automatically, or you can launch it yourself via the [`agl store` command](../reference/cli.md). [`InMemoryLightningStore`][agentlightning.InMemoryLightningStore] keeps all state inside the current process, which makes local iteration fast but introduces two production constraints:
+
+1. Spans are evicted once the process crosses its memory cap, so long runs risk data loss unless the host has abundant RAM.
+2. Although the store is well optimized via asynchronous programming, the store lives in a single process and remains bound by the GIL, preventing it from saturating multi-core machines.
+
+!!! note "General note for all server-client stores"
+
+    If your algorithm and runners communicate through HTTP protocol (which should be the default for 99% of the cases), you need to ensure the file limit is sufficiently large to avoid the "Too many open files" error. You can set the file limit by running the following command:
+
+    ```bash
+    ulimit -n 100000
+    ```
+
+For resilient runs, switch to a persistent backend such as [`MongoLightningStore`][agentlightning.store.mongo.MongoLightningStore], which writes data to MongoDB instead of local RAM. Agent-lightning relies on [pymongo](https://pymongo.readthedocs.io/en/stable/) to interact with MongoDB, which can be installed via:
+
+```bash
+pip install agentlightning[mongo]
+```
+
+To use the MongoDB store, you need to pass the MongoDB URI to the store constructor. The URI should be in the format of `mongodb://<host>:<port>/<database>?replicaSet=<replicaSet>`.
+
+```python
+from agentlightning.store.mongo import MongoLightningStore
+
+trainer = agl.Trainer(
+    algorithm=algorithm,
+    store=MongoLightningStore(mongo_uri="mongodb://localhost:27017/?replicaSet=rs0"),
+)
+```
+
+!!! tip "Setting up MongoDB"
+
+    MongoDB is a popular document-oriented database. Before running Agent-lightning with [`MongoLightningStore`][agentlightning.store.mongo.MongoLightningStore], make sure that you've already had a MongoDB instance running. Setting up can be conveniently done via Docker Compose via [compose.mongo.yml]({{ src("docker/compose.mongo.yml") }}). Unless targeting serious production use, we recommend creating the data folders and setting them to `777` permission to avoid permission issues.
+
+    ```bash
+    mkdir -p data/mongo-host
+    chmod 777 data/mongo-host
+    docker compose -f compose.mongo.yml up -d
+    ```
+
+    Alternatively, you can also install MongoDB manually following the [official documentation](https://www.mongodb.com/docs/manual/installation/). If you installed MongoDB manually, an important note is that you need to ensure that the MongoDB instance has enabled replica set feature, since Agent-lightning uses the transactional operations internally. The simplest approach is to use the following script (executed in the MongoDB shell) to initialize the replica set:
+
+    ```javascript
+    rs.initiate({
+      _id: "rs0",
+      members: [{ _id: 0, host: "localhost:27017" }],
+    });
+    ```
+
+To scale out further, launch the store server via [`agl store --backend mongo`](../reference/cli.md) (see [Debugging with External Store][debug-with-external-store]). The CLI accepts `--n-workers`, which starts the server under `gunicorn` with multiple worker processes so concurrent runners can push and pull at higher throughput. This option applies only to persistent backends; an in-memory store, on the other hand, cannot be sharded across workers because its state lives inside one process.
+
+!!! note
+
+    The `--n-workers` here is the number of worker processes for the store server, NOT related to the number of rollout runners.
+
+## Increasing Throughput of LLM Proxy
+
+Agent-lightning includes an optional [`LLMProxy`][agentlightning.LLMProxy] that wraps [LiteLLM](https://docs.litellm.ai/) to provide a unified OpenAI-compatible endpoint for your agents. When rollout throughput increases, the proxy can become a bottleneck. You can scale it out using the same pattern as the store server.
+
+To increase proxy throughput, pass `num_workers` when constructing the proxy:
+
+```python
+import agentlightning as agl
+
+proxy = agl.LLMProxy(
+    port=4000,
+    launch_mode="mp",  # multiprocessing mode
+    num_workers=4,  # four gunicorn workers handle concurrent requests
+)
+```
+
+You can also configure the proxy through [`Trainer`][agentlightning.Trainer]:
+
+```python
+trainer = agl.Trainer(
+    algorithm=algorithm,
+    n_runners=8,  # The runners here is the rollout runners, not related to LLM proxy replicas
+    llm_proxy={"port": 4000, "num_workers": 4},  # launch mode is actually mp by default
+)
+```
+
+When `num_workers > 1`, the launcher starts gunicorn with the specified number of worker processes. Each worker runs its own event loop, allowing the proxy to handle many concurrent LLM requests without being blocked by Python's GIL.
+
+!!! tip
+
+    When using `mp` launch mode, [`LLMProxy`][agentlightning.LLMProxy] will start the server in a separate process. To make sure the proxy is still accessing the same store as the main process, you need to set the store to be [zero-copy compatible][store-capabilities], which means, either the store is a native zero-copy store like [`MongoLightningStore`][agentlightning.store.mongo.MongoLightningStore] or the store is wrapped via [`LightningStoreServer`][agentlightning.LightningStoreServer] or [`LightningStoreClient`][agentlightning.LightningStoreClient].
+
+!!! note "Shared Server Infrastructure"
+
+    Both [`LightningStoreServer`][agentlightning.LightningStoreServer] and [`LLMProxy`][agentlightning.LLMProxy] rely on a common utility called [`PythonServerLauncherArgs`][agentlightning.utils.server_launcher.PythonServerLauncherArgs]. This dataclass captures the settings needed to launch a FastAPI application:
+
+    ```python
+    from agentlightning.utils import PythonServerLauncherArgs
+
+    args = PythonServerLauncherArgs(
+        port=8000,
+        host="0.0.0.0",
+        n_workers=4,          # spawn 4 gunicorn workers
+        launch_mode="thread", # or "mp" for multiprocessing, "asyncio" for in-loop
+    )
+    ```
+
+    Under the hood, [`PythonServerLauncher`][agentlightning.utils.server_launcher.PythonServerLauncher] reads these arguments and chooses between uvicorn (single worker) and gunicorn (multiple workers) automatically.

docs/tutorials/traces.md

@@ -0,0 +1,133 @@
+# Working with Traces
+
+Tracing is the secret capability that lets Agent-lightning train almost any agent without rewriting its core logic. The idea was born in observability tooling inside LLMOps workflows and, in Agent-lightning, evolved into a first-class primitive inside the learning loop. Beyond helping you understand what happened inside a rollout, traces provide reward spans and other learning signals that power reinforcement learning and fine-tuning algorithms.
+
+![OpenTelemetry spans](../assets/opentelemetry-trace.jpg)
+
+Agent-lightning stores every recorded operation as a [`Span`][agentlightning.Span] inside a [`LightningStore`][agentlightning.LightningStore]. The naming comes from [OpenTelemetry spans](https://opentelemetry.io/docs/concepts/signals/traces/), shown in the screenshot above. A span can represent an LLM call, a tool invocation, a graph edge, an explicit reward emission, or an arbitrary Python code block. Spans form a tree where parent spans describe higher-level steps and children record the detailed work. The sections below walk through how spans are produced and how to interpret them once they reach the store.
+
+## Writing Spans
+
+Most [`Runner`][agentlightning.Runner] implementations wire a [`Tracer`][agentlightning.Tracer] into the agent’s lifecycle. The tracer is responsible for installing instrumentation, buffering OpenTelemetry spans, and committing them to the [`LightningStore`][agentlightning.LightningStore]. When a runner executes a rollout, it allocates a store-backed tracing context:
+
+```python
+async with tracer.trace_context(
+    name="my-rollout",
+    store=store,
+    rollout_id=rollout.rollout_id,
+    attempt_id=attempt.attempt_id,
+):
+    await run_agent_logic()
+```
+
+The context manager then requests sequence numbers from the store, converts OpenTelemetry spans into [`Span`][agentlightning.Span] objects, and persists them in the middle or at the end of the attempt, depending on the tracer implementation. Agent-lightning ships two tracers out of the box; both rely on [OpenTelemetry Traces](https://opentelemetry.io/docs/concepts/signals/traces/) and ignore metrics or logs.
+
+!!! tip "What's instrumentation?"
+
+    In simple terms, *instrumentation* means adding "patches" or hooks inside your code so you can observe what it’s doing while it runs. Think of it like putting flight recorders in an airplane — instrumentation records key actions, inputs, outputs, and timings without changing how the code behaves. In Agent-lightning tracers, this instrumentation automatically creates spans (small, structured records of work) that show what each part of an agent did, how long it took, and how different steps connect together.
+
+### AgentOps Tracer
+
+[`AgentOpsTracer`][agentlightning.AgentOpsTracer] will be the default tracer when [`Trainer`][agentlightning.Trainer] is used but no tracer is explicitly specified. It bootstraps the [AgentOps SDK](https://www.agentops.ai/) locally, installs the supplied instrumentation hooks (LangChain, LangGraph, LiteLLM, FastAPI, and others) provided by the [AgentOps Python SDK](https://github.com/AgentOps-AI/agentops), and forwards everything through a local OpenTelemetry [`TracerProvider`](https://opentelemetry.io/docs/specs/otel/trace/api/). [`AgentOpsTracer`][agentlightning.AgentOpsTracer] never calls the hosted AgentOps service; instead, it attaches a `LightningSpanProcessor` implemented by the Agent-lightning team so that spans are captured and shipped straight into the store.
+
+Because it shares the AgentOps instrumentation surface, any framework supported by AgentOps automatically gains tracing in Agent-lightning. We layer additional hooks on top of AgentOps to capture features that the SDK misses today:
+
+1. Certain providers emit extra metadata — for example, [token IDs returned by vLLM](../deep-dive/serving-llm.md) — that are not recorded by the stock SDK. We augment those spans with the missing payloads.
+2. AgentOps constructs parent-child relationships on a best-effort basis, but mixed instrumentation (for example, OpenAI Agent SDK alongside direct OpenAI Chat Completion calls) can leave segments disconnected. Our implementation (actually implemented in the [`TracerTraceToTriplet`][agentlightning.TracerTraceToTriplet] adapter) repairs those relationships when the hierarchy can be inferred from rollout context.
+3. Some versions of downstream frameworks simply do not emit spans for critical events (LangGraph node entrances are a common example). The tracer installs lightweight shims so those spans appear consistently.
+
+If a vendor integration behaves unexpectedly, users are encouraged to combine the tracer with [Hooks](./debug.md) to inspect the raw spans or diagnostics, and/or implement a specialized tracer for the framework in question.
+
+### OpenTelemetry Tracer
+
+[`OtelTracer`][agentlightning.OtelTracer] is a minimal implementation that initializes a vanilla [`TracerProvider`](https://opentelemetry.io/docs/specs/otel/trace/api/) and gives you direct control over span creation through the standard `opentelemetry.trace` API. Use it when you already have explicit instrumentation in your agent, when the AgentOps SDK does not support your framework, or when you want to emit custom spans from business logic.
+
+!!! note
+
+    [Microsoft Agent Framework](https://github.com/microsoft/agent-framework) is a typical example with built-in OpenTelemetry support. Once you set `OBSERVABILITY_SETTINGS.enable_otel = True`, the framework will automatically emit OpenTelemetry spans, and [`OtelTracer`][agentlightning.OtelTracer] will be able to capture them. No extra instrumentation is needed.
+
+Inside your agent you can call `opentelemetry.trace.get_trace_provider().get_tracer("my-agent")` and use that tracer to [create spans](https://opentelemetry.io/docs/languages/python/cookbook/) exactly as you would in any OpenTelemetry application. The Lightning span processor attached by [`OtelTracer`][agentlightning.OtelTracer] guarantees that every span is sequenced, converted, and written to the store. The same applies for emitted rewards ([`emit_reward`][agentlightning.emit_reward]) and other emitter signals, which are just a special case of manually-created spans.
+
+### Weave Tracer (Experimental)
+
+[`WeaveTracer`][agentlightning.tracer.weave.WeaveTracer] is an experimental tracer that integrates with the [Weave Python SDK](https://docs.wandb.ai/weave). Use it as a substitute for [`AgentOpsTracer`][agentlightning.AgentOpsTracer] when the AgentOps SDK does not fit your environment.
+
+The Weave SDK instruments LLM calls and agent libraries directly. Unlike [`AgentOpsTracer`][agentlightning.AgentOpsTracer], Weave does not rely on OpenTelemetry to export spans; it routes everything through a dedicated Weave Trace Server. Agent-lightning implements a custom Weave Trace Server so every call captured by the Weave SDK can be persisted to the [`LightningStore`][agentlightning.LightningStore].
+
+!!! warning
+
+    [`WeaveTracer`][agentlightning.tracer.weave.WeaveTracer] remains experimental and has not been tested as thoroughly as [`AgentOpsTracer`][agentlightning.AgentOpsTracer]. It may conflict with libraries that ship OpenTelemetry instrumentation by default (for example, LiteLLM-based LLM proxies). Use the tracer with caution and report any issues to the Agent-lightning team.
+
+### LLM Proxy
+
+Sometimes the runner can’t observe the agent directly — because it’s in another language or running remotely. [`LLMProxy`][agentlightning.LLMProxy] bridges that gap by instrumenting the server side of LLM calls. It wraps [LiteLLM](https://docs.litellm.ai/) and adds middleware that accepts prefixed routes like `/rollout/{rid}/attempt/{aid}/v1/chat/completions`. Before forwarding, the middleware rewrites the path to `/v1/chat/completions`, fetches a monotonic `sequence_id` from the `LightningStore`, injects `x-rollout-id`, `x-attempt-id`, and `x-sequence-id` into the request headers, and then forwards the request to the backend LLM endpoint.
+
+LiteLLM produces OpenTelemetry spans for the request/response. A custom `LightningSpanExporter` reads the rollout/attempt/sequence identifiers from the recorded request headers and persists each span to the store. Because the `sequence_id` is allocated at the start of the request, traces stay in strict order even across machines with skewed clocks or asynchronous responses.
+
+```mermaid
+sequenceDiagram
+    participant Agent
+    participant Proxy as LLM Proxy
+    participant Backend as LLM Backend
+    participant Store as LightningStore
+
+    Agent->>Proxy: POST /rollout/{rid}/attempt/{aid}/v1/chat/completions
+    Proxy->>Store: get_next_span_sequence_id(rid, aid)
+    Store-->>Proxy: sequence_id
+    Proxy->>Backend: Forward /v1/chat/completions<br>(headers: rid, aid, sid)
+    Backend-->>Proxy: Response (tokens, usage, token_ids)
+    Proxy->>Store: Export OTEL spans (rid, aid, sequence_id)
+    Proxy-->>Agent: OpenAI-compatible response
+```
+
+[`LLMProxy`][agentlightning.LLMProxy] actually provides more functionalities than just the middleware for tracing. Read [Serving LLM](../deep-dive/serving-llm.md) for more details.
+
+[](){ #distributed-tracing }
+
+!!! note "Distributed Tracing"
+
+    Agent-lightning enforces deterministic span ordering by assigning a monotonic [`sequence_id`][agentlightning.Span.sequence_id] to every span within an attempt. Before calling [`LightningStore.add_span`][agentlightning.LightningStore.add_span] or [`LightningStore.add_otel_span`][agentlightning.LightningStore.add_otel_span], tracers are expected to call [`LightningStore.get_next_span_sequence_id`][agentlightning.LightningStore.get_next_span_sequence_id] to get the next sequence id. This removes clock skew and merges spans produced on different machines or threads. If you implement a custom tracer or exporter, make sure you do this (or respect the one provided in headers by components such as [`LLMProxy`][agentlightning.LLMProxy]); otherwise, adapters will struggle to properly reconstruct the execution tree.
+
+### Custom Tracer
+
+If none of the built-in tracers fit your environment, the first option to consider is to [return the spans](./write-agents.md) directly from your agent implementation. If that's not possible, or you want to support multiple agents in a unified effort, you can implement your own tracer by subclassing [`Tracer`][agentlightning.Tracer].
+
+Custom tracers must implement at least [`trace_context`][agentlightning.Tracer.trace_context]. The [`trace_context`][agentlightning.Tracer.trace_context] coroutine should install or activate whatever instrumentation you need, then yield a span processor that ultimately adds spans to the store. You can reuse the `LightningSpanProcessor` if you produce OpenTelemetry `ReadableSpan` objects, or call [`LightningStore.add_span`][agentlightning.LightningStore.add_span] directly if you generate [`Span`][agentlightning.Span] instances yourself.
+
+Advanced tracers often run auxiliary services (for example, starting a telemetry daemon or attaching to a container runtime) inside `init_worker` and tear them down in `teardown_worker`. The [`ParallelWorkerBase`][agentlightning.ParallelWorkerBase] lifecycle that `Tracer` inherits from ensures those hooks are executed in every runner subprocess.
+
+## Reading Traces
+
+Generally, there are two approaches to reading traces. When you only need a quick look, [`Tracer.get_last_trace`][agentlightning.Tracer.get_last_trace] returns the raw OpenTelemetry spans captured most recently. For historical analysis, use the [`LightningStore.query_spans`][agentlightning.LightningStore.query_spans] API, which yields normalized [`Span`][agentlightning.Span] objects keyed by rollout ID and attempt ID. Combine those queries with [`LightningStore.query_rollouts`][agentlightning.LightningStore.query_rollouts] to align spans with rollout status, retries, and timing information.
+
+Spans arrive asynchronously, originate from different processes, and form hierarchies rather than simple lists. The attributes of each span are tedious and unfriendly to human readers. This combination makes raw traces time-consuming to inspect, especially when you only care about specific signals such as rewards, LLM prompts, responses, or tool outputs. Understanding how the store exposes traces and how adapters reshape them will save hours when debugging or training.
+
+!!! note "Why traces can be difficult to read?"
+
+    The trace tree for a single rollout typically mixes multiple abstraction layers: a planner span may contain several LLM spans, each of which contains tool execution spans that can themselves trigger nested agent invocations. There are also instrumentations at different levels. For example, when a request delegates to another library (e.g., from LangChain to OpenAI), two libraries might emit spans for the same request. At the top level, there could be concurrently running agents that may flush spans slightly out of order. Sorting by `sequence_id` restores the chronological view, but interpreting the tree requires additional context about parent-child relationships and rollout metadata.
+
+### Adapter
+
+[Adapters][agentlightning.Adapter] transform lists of spans into higher-level data structures that training algorithms can consume directly. Agent-lightning provides several adapters out of the box:
+
+* [`TracerTraceToTriplet`][agentlightning.TracerTraceToTriplet] converts spans into `(prompt, response, reward)` triplets, which power reinforcement-learning algorithms such as [VERL](../algorithm-zoo/verl.md) and connect trace data to gradient updates.
+* [`TraceToMessages`][agentlightning.TraceToMessages] rewrites spans into OpenAI chat message JSON suitable for supervised fine-tuning or evaluation harnesses.
+* [`LlmProxyTraceToTriplet`][agentlightning.LlmProxyTraceToTriplet] mirrors [`TracerTraceToTriplet`][agentlightning.TracerTraceToTriplet] but understands spans emitted by [LLMProxy][agentlightning.LLMProxy]. It is experimental and might be merged with [`TracerTraceToTriplet`][agentlightning.TracerTraceToTriplet] in the future.
+
+Adapters are regular Python callable instances, so you can plug them into [`Trainer`][agentlightning.Trainer] via the `adapter` argument, or call them manually during exploration. When used in [`Trainer`][agentlightning.Trainer], adapters are bundled into the [`Algorithm`][agentlightning.Algorithm] before the algorithm runs, through the [`Algorithm.set_adapter`][agentlightning.Algorithm.set_adapter] method.
+
+You can also customize an [`Adapter`][agentlightning.Adapter] by extending the implementations above or subclassing the base class. If you need a bespoke format, subclass [`TraceAdapter`][agentlightning.TraceAdapter] (for store spans) or [`OtelTraceAdapter`][agentlightning.OtelTraceAdapter] (for raw OpenTelemetry spans) and implement `adapt` (these two classes can usually share the same implementation).
+
+### Reading Rewards
+
+Rewards are recorded as dedicated spans named [`agentlightning.annotation`][agentlightning.semconv.AGL_ANNOTATION]. Emitting a reward through [`emit_reward`][agentlightning.emit_reward] or [`emit_annotation`][agentlightning.emit_annotation] ensures the value is stored in the span’s `attributes`. To audit rewards, fetch spans from the store and use the helper utilities in [`agentlightning.emitter`](../reference/agent.md):
+
+```python
+from agentlightning.emitter import find_final_reward
+
+spans = await store.query_spans(rollout_id)
+reward = find_final_reward(spans)
+print(f"Final reward: {reward}")
+```
+
+[`find_reward_spans`][agentlightning.find_reward_spans] returns every reward span so you can visualize intermediate shaping signals, while [`find_final_reward`][agentlightning.find_final_reward] extracts the last non-null reward per attempt. While these helpers are convenient, they may not help you fully understand the chronological or hierarchical relationships between reward spans and other spans. Using an [`Adapter`][agentlightning.Adapter] — especially the same one used in the algorithm you’re working with — remains the recommended way to inspect your generated spans.

docs/tutorials/write-agents.md

@@ -0,0 +1,204 @@
+# Writing Agents
+
+This tutorial will focus on the heart of the system: the agent itself, guiding you through the different ways to define an agent's logic in Agent-lightning.
+
+The basic requirements for any agent are:
+
+1.  It must accept a single **task** as input.
+2.  It must accept a set of tunable **resources** (like a [PromptTemplate][agentlightning.PromptTemplate] or [LLM][agentlightning.LLM]).
+3.  It must **emit** trace span data so that algorithms can understand its behavior and learn from it. *The simplest way to do this is by returning a final reward.*
+
+In practice, please also bear in mind that tasks, resources, and spans have extra requirements, in order to make it *trainable* within Agent-lightning:
+
+1.  You will need a training dataset containing a set of tasks, of the same type that your agent expects as input.
+2.  The tunable resources are related to the algorithm. For example, the APO algorithm we've seen tunes a [PromptTemplate][agentlightning.PromptTemplate]. Other algorithms might tune model weights or other configurations.
+3.  The type of spans an algorithm can use varies. Almost all algorithms support a single, final reward span at the end of a rollout. However, not all algorithms support rewards emitted mid-rollout, let alone other kinds of spans like exceptions or log messages.
+
+This tutorial will show you how to write an agent that can handle various tasks and resources and emit all kinds of spans. However, you should understand that agents and algorithms are often co-designed. Supporting new types of resources or spans in an algorithm is often much more complex than just adding them to an agent.
+
+## [`@rollout`][agentlightning.rollout] Decorator
+
+The simplest way to create an agent is by writing a standard Python function and marking it with the [@rollout][agentlightning.rollout] decorator. This approach is perfect for agents with straightforward logic that doesn't require complex state management.
+
+Agent-lightning automatically inspects your function's signature and injects the required resources. For example, if your function has a parameter named `prompt_template`, Agent-lightning will find the [PromptTemplate][agentlightning.PromptTemplate] resource for the current rollout and pass it in.
+
+Let's revisit the `room_selector` agent from the first tutorial:
+
+```python
+from typing import TypedDict
+from agentlightning import PromptTemplate, rollout
+
+# Define a data structure for the task input
+class RoomSelectionTask(TypedDict):
+    # ... fields for the task ...
+    pass
+
+@rollout
+def room_selector(task: RoomSelectionTask, prompt_template: PromptTemplate) -> float:
+    # 1. Use the injected prompt_template to format the input for the LLM
+    prompt = prompt_template.format(**task)
+
+    # 2. Execute the agent's logic (e.g., call an LLM, use tools)
+    # ...
+
+    # 3. Grade the final choice to get a reward
+    reward = room_selection_grader(final_message, task["expected_choice"])
+
+    # 4. Return the final reward as a float
+    return reward
+```
+
+When you train this agent, the dataset is expected to be a list of `RoomSelectionTask` objects:
+
+```python
+from agentlightning import Dataset, Trainer
+
+dataset: Dataset[RoomSelectionTask] = [
+    RoomSelectionTask(date="2025-10-15", time="10:00", duration_min=60, attendees=10),
+    RoomSelectionTask(date="2025-10-16", time="10:00", duration_min=60, attendees=10),
+]
+
+Trainer().fit(agent=room_selector, train_dataset=dataset)
+```
+
+Behind the scenes, the [`@rollout`][agentlightning.rollout] decorator wraps your function in a `FunctionalLitAgent` object, which is a subclass of [LitAgent][agentlightning.LitAgent] introduced below, making it compatible with the [Trainer][agentlightning.Trainer] and [Runner][agentlightning.Runner]. It supports parameters like `task`, `prompt_template`, `llm`, and `rollout`, giving you flexible access to the execution context.
+
+Here is another example with more advanced usage with `llm` and `rollout` as parameters. The `llm` parameter gives you an OpenAI-compatible LLM endpoint to interact with, which can be tuned under the hood by algorithms. The `rollout` parameter gives you the full [Rollout][agentlightning.Rollout] object, which contains the rollout ID, rollout mode (training or validation), etc.
+
+```python
+from openai import OpenAI
+from agentlightning import LLM, Rollout
+
+class FlightBookingTask(TypedDict):
+    request: str
+    expected_booking: dict
+
+@rollout
+def flight_assistant(task: FlightBookingTask, llm: LLM, rollout: Rollout) -> float:
+    print(f"Rollout ID: {rollout.rollout_id}")
+    print(f"Rollout Mode: {rollout.mode}")
+
+    # Use the tuned LLM resource to create an OpenAI client
+    client = OpenAI(
+        # This endpoint could be a proxy to a proxy to a proxy ...
+        # It could be different every time `flight_assistant` is called
+        # But it should be OpenAI-API compatible
+        base_url=llm.endpoint,
+
+        # Use a dummy key if not provided
+        # Usually this does not matter because the training LLM is often not guarded by an API key
+        # But you can use `or os.environ["OPENAI_API_KEY"]` to make the function compatible with 3rd-party LLMs
+        api_key=llm.api_key or "dummy-key",
+    )
+
+    # Make an API call with the specified model
+    response = client.chat.completions.create(
+        model=llm.model,
+        messages=[{"role": "user", "content": task["request"]}],
+    )
+    # Whether the API supports features like streaming, tool calls, etc. depends on
+    # the endpoint that algorithms are serving to you.
+    final_message = response.choices[0].message.content
+
+    # Grade the result and return a reward
+    reward = grade_flight_booking(final_message, task["expected_booking"])
+    return reward
+```
+
+## Return Values from Agents
+
+The value your agent function returns (i.e., the return value of the function decorated by [`@rollout`][agentlightning.rollout]) is crucial, as it's the primary way to report the outcome of a rollout. Agent-lightning supports several return types to accommodate different scenarios, from simple rewards to detailed, custom traces.
+
+* **`float`**: This is the simplest and most common return type. The `float` is treated as the **final reward** for the entire rollout. Agent-lightning automatically creates a final reward span based on this value.
+
+* **`None`**: Returning `None` tells the runner that trace collection is being handled entirely by the [Tracer][agentlightning.Tracer] through auto-instrumentation (e.g., via AgentOps). In this case, the runner will simply retrieve the spans that the tracer has already captured.
+
+!!! important "Emitting the Final Reward"
+
+    When returning `None`, you must still ensure a final reward is logged. You can do this by using the [`emit_reward`][agentlightning.emit_reward] function (covered in the [Use Emitters](./emitter.md) documentation). Wrapping your reward calculation function with the `@reward` decorator is NOT the recommended approach any more.
+
+* **`list[ReadableSpan]`**, **`list[SpanCoreFields]`**, or **`list[Span]`**: For advanced use cases, you can manually construct and return a complete list of all spans for the rollout. This gives you full control over the trace data. You can return either a list of OpenTelemetry `ReadableSpan` objects or Agent-lightning's native `Span` objects.
+
+For most users, returning a **`float`** for simple agents or returning **`None`** and using the emitter for more complex ones are the recommended approaches.
+
+## Class-based Agents
+
+For more complex agents that require state, helper methods, or distinct logic for training versus validation, you can create a class that inherits from [`LitAgent`][agentlightning.LitAgent]. This object-oriented approach provides more structure and control over the agent's lifecycle.
+
+To create a class-based agent, you subclass [agentlightning.LitAgent][] and implement its [`rollout`][agentlightning.LitAgent.rollout] method.
+
+[](){ #introduction-to-named-resources }
+
+Here's how the `room_selector` could be implemented as a class. The rollout method has a slightly different signature than the function-based agent, mainly in how it handles the resources. Putting it simply, algorithms do not just send a [PromptTemplate][agentlightning.PromptTemplate] to the agents, they instead send [NamedResources][agentlightning.NamedResources], which is a mapping from resource key to [Resource][agentlightning.Resource]. This design is to allow for more advanced features like multi-resource tuning.
+
+With [`@rollout`][agentlightning.rollout] decorator, the resource with correctly matched type will be automatically injected into the rollout method. However, when you use a class-based agent, you need to manually access the resource from the `resources` dictionary. Built-in algorithms listed their resource key naming conventions [here](../algorithm-zoo/index.md).
+
+```python
+import agentlightning as agl
+
+class RoomSelectorAgent(agl.LitAgent[RoomSelectionTask]):
+    def rollout(self, task: RoomSelectionTask, resources: agl.NamedResources, rollout: agl.Rollout) -> float:
+        # 1. Access the prompt_template from the resources dictionary
+        prompt_template = resources["prompt_template"]
+
+        # 2. Execute the agent's logic
+        prompt = prompt_template.format(**task)
+        # ...
+
+        # 3. Grade the final choice
+        reward = room_selection_grader(final_message, task["expected_choice"])
+
+        # 4. Return the final reward
+        return reward
+
+# To use it with the trainer:
+# agent = RoomSelectorAgent()
+# trainer.fit(agent=agent, ...)
+```
+
+The [`LitAgent`][agentlightning.LitAgent] class provides several methods you can override for more fine-grained control:
+
+* [`rollout()`][agentlightning.LitAgent.rollout]: The primary method for the agent's logic. It's called for both training and validation by default.
+* [`training_rollout()`][agentlightning.LitAgent.training_rollout] / [`validation_rollout()`][agentlightning.LitAgent.validation_rollout]: Implement these if you need different behavior during training (e.g., with exploration) and validation (e.g., with deterministic choices).
+* [`rollout_async()`][agentlightning.LitAgent.rollout_async] / [`training_rollout_async()`][agentlightning.LitAgent.training_rollout_async] / [`validation_rollout_async()`][agentlightning.LitAgent.validation_rollout_async]: Implement the asynchronous versions of these methods if your agent uses `asyncio`.
+
+!!! note
+
+    Rollout is always executed in an asynchronous context no matter whether the agent is asynchronous or synchronous. If your synchronous agent contains some `asyncio.run()` calls, it might raise an error that there is already an event loop running. To avoid blocking the event loop, it's recommended to offload the inner async operations to a separate thread. Here is a sample code:
+
+    ```python
+    import asyncio
+    import queue
+    import threading
+
+    def run_sync_ephemeral(coro) -> Any:
+        """
+        Run an async coroutine from sync code.
+        - If no loop in this thread: use asyncio.run() directly.
+        - If already in an event loop: spawn a worker thread that calls asyncio.run()
+        (which creates and closes a brand-new event loop per call).
+        """
+        try:
+            asyncio.get_running_loop()
+        except RuntimeError:
+            # No running loop in this thread; safe to use asyncio.run
+            return asyncio.run(coro)
+
+        # Already in a running loop -> execute in a worker thread
+        q = queue.Queue[Any]()
+
+        def worker():
+            try:
+                result = asyncio.run(coro)  # creates & closes its own loop
+                q.put((True, result))
+            except BaseException as e:
+                q.put((False, e))
+
+        t = threading.Thread(target=worker, daemon=True)
+        t.start()
+        ok, payload = q.get()
+        t.join()
+        if ok:
+            return payload
+        raise payload
+    ```