lightseekorg--tokenspeed/docs/index.md

layout, hero, features

layout	hero	features
home	name text tagline actions TokenSpeed Speed-of-light LLM inference Production-oriented docs for launching, tuning, and operating low-latency OpenAI-compatible serving. theme text link brand Get Started /guides/getting-started theme text link alt Launch Recipes /recipes/models theme text link alt Server Parameters /configuration/server	title details Launch First Start with concrete commands, then tune the exact knobs that affect memory, scheduling, parallelism, and kernels. title details Familiar Parameters TokenSpeed keeps familiar parameter names where the runtime semantics match, with TokenSpeed-specific knobs documented separately. title details Model Recipes Recipes collect the launch patterns used for Kimi and GPT-OSS deployments. title details Operational Surface Parallelism and configuration guidance stay close to the serving paths operators actually use.

Start Here

• Getting Started
• Launching a Server
• Model Recipes
• Server Parameters
• Compatible Parameters
• Parallelism

Common Workflow

1. Install the runtime and kernel packages.
2. Pick a launch recipe close to your model family and hardware.
3. Set model loading, memory, scheduler, and parallelism parameters explicitly.
4. Validate correctness and throughput together before changing more than one tuning dimension.

Minimal Server

tokenspeed serve openai/gpt-oss-20b \
  --host 0.0.0.0 \
  --port 8000 \
  --tensor-parallel-size 1

The server exposes an OpenAI-compatible API under /v1.

High-Performance Shape

Large MoE deployments usually make the same decisions:

• model path and revision
• context length and KV cache dtype
• scheduler token and sequence budgets
• attention and MoE backends
• tensor, data, and expert parallelism
• reasoning, tool-call, and speculative decoding parsers

See Model Recipes for concrete examples and Server Parameters for the parameter reference.