chore: import upstream snapshot with attribution

docs/reference/agent.md

@@ -0,0 +1,51 @@
+# Agent Developer APIs
+
+## Agent Decorators
+
+!!! tip
+
+    These are convenient helpers for creating agents from functions. First-time users are recommended to use these decorators to create agents.
+
+::: agentlightning.rollout
+
+!!! warning
+
+    The following two decorators are implementations of [`agentlightning.rollout`][agentlightning.rollout]. They are not recommended for new users.
+
+::: agentlightning.llm_rollout
+
+::: agentlightning.prompt_rollout
+
+## Class-based Agents
+
+::: agentlightning.LitAgent
+
+## Emitter
+
+::: agentlightning.operation
+
+::: agentlightning.emit_annotation
+
+::: agentlightning.emit_reward
+
+::: agentlightning.emit_message
+
+::: agentlightning.emit_object
+
+::: agentlightning.emit_exception
+
+## Emitter Helpers
+
+::: agentlightning.get_message_value
+
+::: agentlightning.get_object_value
+
+::: agentlightning.find_final_reward
+
+::: agentlightning.find_reward_spans
+
+::: agentlightning.get_reward_value
+
+::: agentlightning.get_rewards_from_span
+
+::: agentlightning.is_reward_span

docs/reference/algorithm.md

@@ -0,0 +1,38 @@
+## Algorithm-side References
+
+!!! note
+
+    This reference covers APIs that are designed to be used at "Algorithm Side".
+    For built-in algorithms, see [Algorithm Zoo](../algorithm-zoo/index.md).
+
+## Base Class and Decorators
+
+::: agentlightning.Algorithm
+
+::: agentlightning.algo
+
+## Fast Algorithms (for Debugging)
+
+::: agentlightning.FastAlgorithm
+
+::: agentlightning.Baseline
+
+## Adapter
+
+::: agentlightning.Adapter
+
+::: agentlightning.TraceAdapter
+
+::: agentlightning.OtelTraceAdapter
+
+::: agentlightning.TraceToTripletBase
+
+::: agentlightning.TracerTraceToTriplet
+
+::: agentlightning.LlmProxyTraceToTriplet
+
+::: agentlightning.TraceToMessages
+
+## LLM Proxy
+
+::: agentlightning.LLMProxy

docs/reference/cli.md

@@ -0,0 +1,118 @@
+# Command Line Interface
+
+!!! warning
+
+    This document is a work in progress and might not be updated with the latest changes.
+    Try to use `agl -h` to get the latest help message.
+
+!!! tip
+
+    Agent-lightning also provides utilities to help you build your own CLI for [LitAgent][agentlightning.LitAgent] and [Trainer][agentlightning.Trainer]. See [Trainer](./trainer.md) for references.
+
+## agl
+
+```text
+usage: agl [-h] {vllm,store,prometheus,agentops}
+
+Agent Lightning CLI entry point.
+
+Available subcommands:
+  vllm        Run the vLLM CLI with Agent Lightning instrumentation.
+  store       Run a LightningStore server.
+  prometheus  Serve Prometheus metrics from the multiprocess registry.
+  agentops    Start the AgentOps server manager.
+
+positional arguments:
+  {vllm,store,prometheus,agentops}
+                        Subcommand to run.
+
+options:
+  -h, --help            show this help message and exit
+```
+
+## agl vllm
+
+Agent-lightning's instrumented vLLM CLI.
+
+```text
+usage: agl vllm [-h] [-v] {chat,complete,serve,bench,collect-env,run-batch} ...
+
+vLLM CLI
+
+positional arguments:
+  {chat,complete,serve,bench,collect-env,run-batch}
+    chat                Generate chat completions via the running API server.
+    complete            Generate text completions based on the given prompt via the running API server.
+    collect-env         Start collecting environment information.
+    run-batch           Run batch prompts and write results to file.
+
+options:
+  -h, --help            show this help message and exit
+  -v, --version         show program's version number and exit
+
+For full list:            vllm [subcommand] --help=all
+For a section:            vllm [subcommand] --help=ModelConfig    (case-insensitive)
+For a flag:               vllm [subcommand] --help=max-model-len  (_ or - accepted)
+Documentation:            https://docs.vllm.ai
+```
+
+## agl store
+
+Agent-lightning's LightningStore CLI. Use it to start an independent LightningStore server.
+
+Currently the store data are stored in memory and will be lost when the server is stopped.
+
+```text
+usage: agl store [-h] [--host HOST] [--port PORT] [--cors-origin CORS_ORIGINS] [--log-level {DEBUG,INFO,WARNING,ERROR}] [--tracker {prometheus,console} [{prometheus,console} ...]] [--n-workers N_WORKERS] [--backend {memory,mongo}]
+                 [--mongo-uri MONGO_URI]
+
+Run a LightningStore server
+
+options:
+  -h, --help            show this help message and exit
+  --host HOST           Host to bind the server to
+  --port PORT           Port to run the server on
+  --cors-origin CORS_ORIGINS
+                        Allowed CORS origin. Repeat for multiple origins. Use '*' to allow all origins.
+  --log-level {DEBUG,INFO,WARNING,ERROR}
+                        Configure the logging level for the store.
+  --tracker {prometheus,console} [{prometheus,console} ...]
+                        Enable metrics tracking. Repeat for multiple trackers.
+  --n-workers N_WORKERS
+                        Number of workers to run in the server. When it's greater than 1, the server will be run using `mp` launch mode. Only applicable for zero-copy stores such as MongoDB backend.
+  --backend {memory,mongo}
+                        Backend to use for the store.
+  --mongo-uri MONGO_URI
+                        MongoDB URI to use for the store. Applicable only if --backend is 'mongo'.
+```
+
+!!! tip
+
+    After launching the store via CLI, you can tell the [`Trainer`][agentlightning.Trainer] to use the store by passing the store address to the trainer.
+
+    ```python
+    store_client = agl.LightningStoreClient("http://localhost:4747")
+    trainer = agl.Trainer(store=store_client, ...)
+    ```
+
+    See [using external store][debug-with-external-store] for more details.
+
+## agl prometheus
+
+Expose the Prometheus multiprocess registry on a dedicated FastAPI server. This is useful when the main LightningStore service is under heavy load; exporters can scrape this auxiliary endpoint instead.
+
+```text
+usage: agl prometheus [-h] [--host HOST] [--port PORT] [--metrics-path METRICS_PATH] [--log-level {DEBUG,INFO,WARNING,ERROR}] [--access-log]
+
+Serve Prometheus metrics outside the LightningStore server.
+
+options:
+  -h, --help            show this help message and exit
+  --host HOST           Host to bind the metrics server to.
+  --port PORT           Port to expose the Prometheus metrics on.
+  --metrics-path METRICS_PATH
+                        HTTP path used to expose metrics. Must start with '/' and not be the root path.
+  --log-level {DEBUG,INFO,WARNING,ERROR}
+                        Configure the logging level for the metrics server.
+  --access-log          Enable uvicorn access logs. Disabled by default to reduce noise.
+```

docs/reference/instrumentation.md

@@ -0,0 +1,21 @@
+# Instrumentation API
+
+::: agentlightning.instrumentation.instrument_all
+
+::: agentlightning.instrumentation.uninstrument_all
+
+## AgentOps LangChain
+
+::: agentlightning.instrumentation.agentops_langchain
+
+## AgentOps
+
+::: agentlightning.instrumentation.agentops
+
+## LiteLLM
+
+::: agentlightning.instrumentation.litellm
+
+## vLLM
+
+::: agentlightning.instrumentation.vllm

docs/reference/internal.md

@@ -0,0 +1,77 @@
+# Internal API References
+
+!!! danger
+
+    The following APIs should be used with extra caution because they are very likely to change in the future.
+
+## Algorithms and Adapters
+
+::: agentlightning.adapter.messages.OpenAIMessages
+
+::: agentlightning.adapter.triplet.TraceTree
+
+::: agentlightning.adapter.triplet.Transition
+
+::: agentlightning.adapter.triplet.RewardMatchPolicy
+
+::: agentlightning.algorithm.decorator.FunctionalAlgorithm
+
+## LitAgent
+
+::: agentlightning.litagent.decorator.FunctionalLitAgent
+
+::: agentlightning.litagent.decorator.llm_rollout
+
+::: agentlightning.litagent.decorator.prompt_rollout
+
+::: agentlightning.emitter.annotation.OperationContext
+
+## LLM Proxy
+
+::: agentlightning.llm_proxy.ModelConfig
+
+::: agentlightning.llm_proxy.LightningSpanExporter
+
+::: agentlightning.llm_proxy.LightningOpenTelemetry
+
+::: agentlightning.llm_proxy.AddReturnTokenIds
+
+::: agentlightning.llm_proxy.StreamConversionMiddleware
+
+::: agentlightning.llm_proxy.MessageInspectionMiddleware
+
+::: agentlightning.llm_proxy.RolloutAttemptMiddleware
+
+## Store
+
+::: agentlightning.store.base.UNSET
+
+::: agentlightning.store.utils.rollout_status_from_attempt
+
+::: agentlightning.store.utils.scan_unhealthy_rollouts
+
+## Tracing and OpenTelemetry
+
+::: agentlightning.tracer.otel.LightningSpanProcessor
+
+## Deprecated APIs
+
+::: agentlightning.emitter.reward.reward
+
+::: agentlightning.server.AgentLightningServer
+
+::: agentlightning.server.ServerDataStore
+
+::: agentlightning.client.AgentLightningClient
+
+::: agentlightning.client.DevTaskLoader
+
+::: agentlightning.Task
+
+::: agentlightning.TaskInput
+
+::: agentlightning.TaskIfAny
+
+::: agentlightning.RolloutRawResultLegacy
+
+::: agentlightning.RolloutLegacy

docs/reference/restful.md

@@ -0,0 +1,19 @@
+# RESTful API References
+
+!!! note
+
+    Shown in the following is the RESTful API for Lightning Store.
+
+<div id="swagger-ui"></div>
+<link rel="stylesheet" href="https://unpkg.com/swagger-ui-dist/swagger-ui.css" />
+<script src="https://unpkg.com/swagger-ui-dist/swagger-ui-bundle.js"></script>
+<script>
+  window.onload = () => {
+    window.ui = SwaggerUIBundle({
+      url: '../../assets/store-openapi.json',
+      dom_id: '#swagger-ui',
+      presets: [SwaggerUIBundle.presets.apis],
+      layout: "BaseLayout"
+    });
+  }
+</script>

docs/reference/runner.md

@@ -0,0 +1,31 @@
+# Runner-side References
+
+!!! note
+
+    This reference covers APIs that are designed to be used at "Runner Side".
+
+## Runners
+
+::: agentlightning.LitAgentRunner
+
+::: agentlightning.Runner
+
+## Tracer
+
+::: agentlightning.AgentOpsTracer
+
+::: agentlightning.OtelTracer
+
+::: agentlightning.Tracer
+
+::: agentlightning.tracer.weave.WeaveTracer
+
+::: agentlightning.DummyTracer
+
+::: agentlightning.set_active_tracer
+
+::: agentlightning.get_active_tracer
+
+::: agentlightning.clear_active_tracer
+
+::: agentlightning.tracer.weave.WeaveTracer

docs/reference/semconv.md

@@ -0,0 +1,3 @@
+# Semantic Conventions
+
+::: agentlightning.semconv

docs/reference/store.md

@@ -0,0 +1,53 @@
+# Store References
+
+::: agentlightning.LightningStore
+
+::: agentlightning.LightningStoreCapabilities
+
+## Store Implementations
+
+::: agentlightning.InMemoryLightningStore
+
+::: agentlightning.store.mongo.MongoLightningStore
+
+::: agentlightning.CollectionBasedLightningStore
+
+## Client-Server and Thread-safe Wrappers
+
+::: agentlightning.LightningStoreServer
+
+::: agentlightning.LightningStoreClient
+
+::: agentlightning.LightningStoreThreaded
+
+## Collections and Collection Implementations
+
+::: agentlightning.store.collection.AtomicMode
+
+::: agentlightning.store.collection.AtomicLabels
+
+::: agentlightning.store.collection.Collection
+
+::: agentlightning.store.collection.Queue
+
+::: agentlightning.store.collection.KeyValue
+
+::: agentlightning.store.collection.LightningCollections
+
+::: agentlightning.store.collection.ListBasedCollection
+
+::: agentlightning.store.collection.DequeBasedQueue
+
+::: agentlightning.store.collection.DictBasedKeyValue
+
+::: agentlightning.store.collection.InMemoryLightningCollections
+
+::: agentlightning.store.collection.mongo.MongoBasedCollection
+
+::: agentlightning.store.collection.mongo.MongoBasedQueue
+
+::: agentlightning.store.collection.mongo.MongoBasedKeyValue
+
+::: agentlightning.store.collection.mongo.MongoClientPool
+
+::: agentlightning.store.collection.mongo.MongoLightningCollections

docs/reference/trainer.md

@@ -0,0 +1,33 @@
+# Agent-lightning Trainer
+
+::: agentlightning.Trainer
+
+::: agentlightning.build_component
+
+## Execution Strategy
+
+::: agentlightning.ExecutionStrategy
+
+::: agentlightning.ClientServerExecutionStrategy
+
+::: agentlightning.SharedMemoryExecutionStrategy
+
+## Events
+
+::: agentlightning.ExecutionEvent
+
+::: agentlightning.ThreadingEvent
+
+::: agentlightning.MultiprocessingEvent
+
+## CLI Builder
+
+::: agentlightning.lightning_cli
+
+## Logging
+
+::: agentlightning.configure_logger
+
+::: agentlightning.setup_module_logging
+
+::: agentlightning.setup_logging

docs/reference/types.md

@@ -0,0 +1,97 @@
+# Type References
+
+## Core Types
+
+::: agentlightning.Triplet
+
+::: agentlightning.RolloutRawResult
+
+::: agentlightning.RolloutMode
+
+::: agentlightning.GenericResponse
+
+::: agentlightning.ParallelWorkerBase
+
+::: agentlightning.Dataset
+
+::: agentlightning.AttemptStatus
+
+::: agentlightning.RolloutStatus
+
+::: agentlightning.RolloutConfig
+
+::: agentlightning.Rollout
+
+::: agentlightning.EnqueueRolloutRequest
+
+::: agentlightning.Attempt
+
+::: agentlightning.AttemptedRollout
+
+::: agentlightning.Worker
+
+::: agentlightning.WorkerStatus
+
+::: agentlightning.Hook
+
+::: agentlightning.PaginatedResult
+
+::: agentlightning.FilterOptions
+
+::: agentlightning.SortOptions
+
+::: agentlightning.FilterField
+
+## Resources
+
+::: agentlightning.Resource
+
+::: agentlightning.LLM
+
+::: agentlightning.ProxyLLM
+
+::: agentlightning.PromptTemplate
+
+::: agentlightning.ResourceUnion
+
+::: agentlightning.NamedResources
+
+::: agentlightning.ResourcesUpdate
+
+## Traces
+
+::: agentlightning.AttributeValue
+
+::: agentlightning.Attributes
+
+::: agentlightning.TraceState
+
+::: agentlightning.SpanContext
+
+::: agentlightning.TraceStatus
+
+::: agentlightning.Event
+
+::: agentlightning.Link
+
+::: agentlightning.Resource
+
+::: agentlightning.Span
+
+::: agentlightning.SpanAttributeNames
+
+::: agentlightning.SpanLike
+
+::: agentlightning.SpanCoreFields
+
+::: agentlightning.SpanRecordingContext
+
+## Environment Variables
+
+::: agentlightning.LightningEnvVar
+
+::: agentlightning.resolve_bool_env_var
+
+::: agentlightning.resolve_int_env_var
+
+::: agentlightning.resolve_str_env_var

docs/reference/utilities.md

@@ -0,0 +1,75 @@
+# Utility References
+
+## ID
+
+::: agentlightning.utils.id.generate_id
+
+## Metrics
+
+::: agentlightning.utils.metrics.MetricsBackend
+
+::: agentlightning.utils.metrics.ConsoleMetricsBackend
+
+::: agentlightning.utils.metrics.PrometheusMetricsBackend
+
+::: agentlightning.utils.metrics.MultiMetricsBackend
+
+::: agentlightning.utils.metrics.setup_multiprocess_prometheus
+
+::: agentlightning.utils.metrics.get_prometheus_registry
+
+::: agentlightning.utils.metrics.shutdown_metrics
+
+## Server Launcher
+
+::: agentlightning.utils.server_launcher.PythonServerLauncher
+
+::: agentlightning.utils.server_launcher.PythonServerLauncherArgs
+
+::: agentlightning.utils.server_launcher.LaunchMode
+
+## OpenTelemetry
+
+::: agentlightning.utils.otel.full_qualified_name
+
+::: agentlightning.utils.otel.get_tracer_provider
+
+::: agentlightning.utils.otel.get_tracer
+
+::: agentlightning.utils.otel.make_tag_attributes
+
+::: agentlightning.utils.otel.extract_tags_from_attributes
+
+::: agentlightning.utils.otel.make_link_attributes
+
+::: agentlightning.utils.otel.query_linked_spans
+
+::: agentlightning.utils.otel.extract_links_from_attributes
+
+::: agentlightning.utils.otel.filter_attributes
+
+::: agentlightning.utils.otel.filter_and_unflatten_attributes
+
+::: agentlightning.utils.otel.flatten_attributes
+
+::: agentlightning.utils.otel.unflatten_attributes
+
+::: agentlightning.utils.otel.sanitize_attribute_value
+
+::: agentlightning.utils.otel.sanitize_attributes
+
+::: agentlightning.utils.otel.sanitize_list_attribute_sanity
+
+::: agentlightning.utils.otel.check_attributes_sanity
+
+::: agentlightning.utils.otel.format_exception_attributes
+
+## OTLP
+
+::: agentlightning.utils.otlp.handle_otlp_export
+
+::: agentlightning.utils.otlp.spans_from_proto
+
+## System Snapshot
+
+::: agentlightning.utils.system_snapshot.system_snapshot