mudler--localai/docs/content/features/video-generation.md

+++ disableToc = false title = "Video Generation" weight = 18 url = "/features/video-generation/" +++

LocalAI can generate videos from text prompts and optional image or audio conditioning via the /video endpoint. Supported backends include diffusers, stablediffusion, vllm-omni, and the dedicated longcat-video backend.

API

• Method: POST
• Endpoint: /video

Request

The request body is JSON with the following fields:

Parameter	Type	Required	Default	Description
model	string	Yes		Model name to use
prompt	string	Yes		Text description of the video to generate
negative_prompt	string	No		What to exclude from the generated video
start_image	string	No		Starting image as base64 string or URL
end_image	string	No		Ending image for guided generation
audio	string	No		Audio conditioning as base64, a data URI, or URL
width	int	No	512	Video width in pixels
height	int	No	512	Video height in pixels
num_frames	int	No		Number of frames
fps	int	No		Frames per second
seconds	string	No		Duration in seconds
size	string	No		Size specification (alternative to width/height)
input_reference	string	No		Input reference for the generation
seed	int	No		Random seed for reproducibility
cfg_scale	float	No		Classifier-free guidance scale
step	int	No		Number of inference steps
response_format	string	No	url	url to return a file URL, b64_json for base64 output
params	object	No		Backend-specific string parameters

Response

Returns an OpenAI-compatible JSON response:

Field	Type	Description
created	int	Unix timestamp of generation
id	string	Unique identifier (UUID)
data	array	Array of generated video items
data[].url	string	URL path to video file (if response_format is url)
data[].b64_json	string	Base64-encoded video (if response_format is b64_json)

Usage

Generate a video from a text prompt

curl http://localhost:8080/video \
  -H "Content-Type: application/json" \
  -d '{
    "model": "video-model",
    "prompt": "A cat playing in a garden on a sunny day",
    "width": 512,
    "height": 512,
    "num_frames": 16,
    "fps": 8
  }'

Example response

{
  "created": 1709900000,
  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "data": [
    {
      "url": "/generated-videos/abc123.mp4"
    }
  ]
}

Generate with a starting image

curl http://localhost:8080/video \
  -H "Content-Type: application/json" \
  -d '{
    "model": "video-model",
    "prompt": "A timelapse of flowers blooming",
    "start_image": "https://example.com/flowers.jpg",
    "num_frames": 24,
    "fps": 12,
    "seed": 42,
    "cfg_scale": 7.5,
    "step": 30
  }'

Get base64-encoded output

curl http://localhost:8080/video \
  -H "Content-Type: application/json" \
  -d '{
    "model": "video-model",
    "prompt": "Ocean waves on a beach",
    "response_format": "b64_json"
  }'

LongCat-Video and Avatar 1.5

The dedicated longcat-video backend serves the official base and Avatar 1.5 checkpoints, including CUDA 13 ARM64 systems such as DGX Spark. See [LongCat Video and Avatar]({{%relref "features/longcat-video" %}}) for gallery installation, Studio instructions, complete model YAML, API examples, tuning options, and hardware requirements.

Error Responses

Status Code	Description
400	Missing or invalid model or request parameters
412	The selected backend cannot run on the available hardware
500	Backend error during video generation