Video Tool

Enable Video Tool on a POST /threads/messages turn and the thread LLM decides whether to call the built-in generate_video tool. llm_provider / model_name select the conversation model; video_model_provider / video_model_name select the separate video generation model. Enabling the tool does not force a generation.

​

Turn-level parameters

For one-shot requests with explicit frame/reference roles, see stateless video calls. The video model and config fields stay the same.

Pass these settings on every turn that needs video generation, including follow-ups. They are not permanent assistant configuration.

Discover models with authenticated GET /models/video/all, or inspect one with GET /models/video/{model_name}. These endpoints expose OpenRouter’s live video catalog, architecture, generation options, and pricing SKUs without a database sync. Check capabilities and supported values rather than assuming every model supports every mode.

Video models also belong to the shared catalog as model_type: "video". The OpenRouter catalog sync imports their capabilities and pricing metadata; GET /models?model_type=video and the model-library page expose the synchronized snapshot. Operators can sync just video models with python update_openrouter_models.py videos --no-email. Catalog prices are informational: billing uses the actual provider-reported request cost, never an estimated replacement when that cost is missing.

video_config accepts:

Before the first conversation-model call, Backboard builds the video tool schema from the selected model’s supported values. Explicit API configuration is fixed; tool arguments can fill omitted settings, not override them. Invalid caller configurations are rejected before generation. Tool schema and execution validation share a capability snapshot, including continuation calls within the turn. These rules are identical for streaming and non-streaming.

A model advertising video input does not establish unlimited file size, duration, reference count, or codec support. Use inputs supported by the selected model.

​

Text-to-video

Replace VIDEO_MODEL_ID with a current model ID from OpenRouter’s video catalog.

curl -X POST "https://app.backboard.io/api/threads/messages" \
  -H "X-API-Key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "thread_id": "THREAD_ID",
    "content": "Generate a video of a sunflower opening in the morning light.",
    "llm_provider": "openai",
    "model_name": "gpt-4.1",
    "video_generation": "auto",
    "video_model_provider": "openrouter",
    "video_model_name": "VIDEO_MODEL_ID",
    "stream": true
  }'

​

Media inputs and supported modes

The assistant calls generate_video internally with a prompt and, when appropriate, media document IDs from uploads or previous generations:

Reference images guide style or content rather than specifying exact frames. Ask the assistant to use the IDs in your message; these are tool arguments, not additional top-level message fields. For example:

Generate a video of the flower opening. Use image document FIRST_IMAGE_ID as
input_image_document_id and LAST_IMAGE_ID as input_last_frame_document_id.

Each mode, image combination, and audio-output option depends on the actual video model. These are capability-dependent patterns, not a claim that every permutation or model has been tested.

Providers also impose input constraints not included in modality flags. For example, live tests of Seedance 2.0 Mini required reference video of at least 407,696 pixels and an image or video alongside any audio reference. Backboard supplies HTTPS URLs for audio/video references; inline data URIs are not accepted for those video-generation inputs.

Video input for chat is separate: understanding an uploaded video requires a conversation model with supports_video_input; this does not imply video generation support. For generation, audio/video references are forwarded only when the selected video model advertises those input modalities. Frame images and references cannot be combined: OpenRouter would ignore the references. Generated audio is audio within the output clip, where supported.

​

Results, lifecycle, and cost

Backboard returns stored videos through the existing generated_media message metadata and media_generated SSE events, with document_id, url, media_type: "video", and mime_type: "video/mp4":

{
  "type": "media_generated",
  "media": {
    "document_id": "VIDEO_DOCUMENT_ID",
    "media_type": "video",
    "mime_type": "video/mp4",
    "url": "https://..."
  }
}

Generated clips use existing document/media storage; no schema migration is required.

Underneath the tool, OpenRouter uses asynchronous POST /api/v1/videos jobs and polling. States include pending, in_progress, completed, failed, cancelled, and expired. These are upstream job states, not new Backboard SSE event types. Generation can take minutes; a failed or otherwise unsuccessful job does not provide a completed clip. Billing uses the actual usage.cost returned by polling, not an estimated cost.

Provider job IDs and completed-job costs are retained in thread metadata for reconciliation. Polling is bounded to 30 minutes. A process interruption or polling timeout does not cancel the upstream job; automatic recovery/reconciliation is not implemented. Do not blindly resubmit a failed turn.

See the official OpenRouter video generation guide for model discovery and upstream options.

​

Related pages

• SDK Video Tool Guide
• Image Tool
• Tool Calling