Music generation

# Async behavior

Session-backed music generation runs as a background task:

• Background task: music_generate creates a background task, returns a started/task response immediately, and posts the finished track later in a follow-up agent message.
• Duplicate prevention: while a task is queued or running, later music_generate calls in the same session return task status instead of starting another generation. Use action: "status" to check explicitly.
• Status lookup: openclaw tasks list or openclaw tasks show <taskId> inspects queued, running, and terminal status.
• Completion wake: OpenClaw injects an internal completion event back into the same session so the model can write the user-facing follow-up itself.
• Prompt hint: later user/manual turns in the same session get a small runtime hint when a music task is already in flight, so the model does not blindly call music_generate again.
• No-session fallback: direct/local contexts without a real agent session run inline and return the final audio result in the same turn.

# Task lifecycle

Check status from the CLI:

openclaw tasks list
openclaw tasks show <taskId>
openclaw tasks cancel <taskId>

# Configuration

# Model selection

{
  agents: {
    defaults: {
      musicGenerationModel: {
        primary: "google/lyria-3-clip-preview",
        fallbacks: ["minimax/music-2.6"],
      },
    },
  },
}

# Provider selection order

OpenClaw tries providers in this order:

1. model parameter from the tool call (if the agent specifies one).
2. musicGenerationModel.primary from config.
3. musicGenerationModel.fallbacks in order.
4. Auto-detection using auth-backed provider defaults only:
   • current default provider first;
   • remaining registered music-generation providers in provider-id order.

If a provider fails, the next candidate is tried automatically. If all fail, the error includes details from each attempt.

Set agents.defaults.mediaGenerationAutoProviderFallback: false to use only explicit model, primary, and fallbacks entries.

# Provider notes

# Choosing the right path

• Shared provider-backed when you want model selection, provider failover, and the built-in async task/status flow.
• Plugin path (ComfyUI) when you need a custom workflow graph or a provider that is not part of the shared bundled music capability.

If you are debugging ComfyUI-specific behavior, see ComfyUI. If you are debugging shared provider behavior, start with Google (Gemini) or MiniMax.

# Provider capability modes

The shared music-generation contract supports explicit mode declarations:

• generate for prompt-only generation.
• edit when the request includes one or more reference images.

New provider implementations should prefer explicit mode blocks:

capabilities: {
  generate: {
    maxTracks: 1,
    supportsLyrics: true,
    supportsFormat: true,
  },
  edit: {
    enabled: true,
    maxTracks: 1,
    maxInputImages: 1,
    supportsFormat: true,
  },
}

Legacy flat fields such as maxInputImages, supportsLyrics, and supportsFormat are not enough to advertise edit support. Providers should declare generate and edit explicitly so live tests, contract tests, and the shared music_generate tool can validate mode support deterministically.

# Live tests

Opt-in live coverage for the shared bundled providers:

OPENCLAW_LIVE_TEST=1 pnpm test:live -- extensions/music-generation-providers.live.test.ts

Repo wrapper:

pnpm test:live:media music

This live file loads missing provider env vars from ~/.profile, prefers live/env API keys ahead of stored auth profiles by default, and runs both generate and declared edit coverage when the provider enables edit mode. Coverage today:

• google: generate plus edit
• minimax: generate only
• comfy: separate Comfy live coverage, not the shared provider sweep

Opt-in live coverage for the bundled ComfyUI music path:

OPENCLAW_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts

The Comfy live file also covers comfy image and video workflows when those sections are configured.

# Related

• Background tasks — task tracking for detached music_generate runs
• ComfyUI
• Configuration reference — musicGenerationModel config
• Google (Gemini)
• MiniMax
• Models — model configuration and failover
• Tools overview