Control UI

• Chat with the model via Gateway WS (chat.history, chat.send, chat.abort, chat.inject).
• Chat history refreshes request a bounded recent window with per-message text caps so large sessions do not force the browser to render a full transcript payload before the chat becomes usable.
• Talk through browser realtime sessions. OpenAI uses direct WebRTC, Google Live uses a constrained one-use browser token over WebSocket, and backend-only realtime voice plugins use the Gateway relay transport. Client-owned provider sessions start with talk.client.create; Gateway relay sessions start with talk.session.create. The relay keeps provider credentials on the Gateway while the browser streams microphone PCM through talk.session.appendAudio and forwards openclaw_agent_consult provider tool calls through talk.client.toolCall for Gateway policy and the larger configured OpenClaw model.
• Stream tool calls + live tool output cards in Chat (agent events).

• Channels: built-in plus bundled/external plugin channels status, QR login, and per-channel config (channels.status, web.login.*, config.patch).
• Channel probe refreshes keep the previous snapshot visible while slow provider checks finish, and partial snapshots are labeled when a probe or audit exceeds its UI budget.
• Instances: presence list + refresh (system-presence).
• Sessions: list configured-agent sessions by default, fall back from stale unconfigured agent session keys, and apply per-session model/thinking/fast/verbose/trace/reasoning overrides (sessions.list, sessions.patch).
• Dreams: dreaming status, enable/disable toggle, and Dream Diary reader (doctor.memory.status, doctor.memory.dreamDiary, config.patch).

• Cron jobs: list/add/edit/run/enable/disable + run history (cron.*).
• Skills: status, enable/disable, install, API key updates (skills.*).
• Nodes: list + caps (node.list).
• Exec approvals: edit gateway or node allowlists + ask policy for exec host=gateway/node (exec.approvals.*).

• View/edit ~/.openclaw/openclaw.json (config.get, config.set).
• Apply + restart with validation (config.apply) and wake the last active session.
• Writes include a base-hash guard to prevent clobbering concurrent edits.
• Writes (config.set/config.apply/config.patch) preflight active SecretRef resolution for refs in the submitted config payload; unresolved active submitted refs are rejected before write.
• Schema + form rendering (config.schema / config.schema.lookup, including field title / description, matched UI hints, immediate child summaries, docs metadata on nested object/wildcard/array/composition nodes, plus plugin + channel schemas when available); Raw JSON editor is available only when the snapshot has a safe raw round-trip.
• If a snapshot cannot safely round-trip raw text, Control UI forces Form mode and disables Raw mode for that snapshot.
• Raw JSON editor "Reset to saved" preserves the raw-authored shape (formatting, comments, $include layout) instead of re-rendering a flattened snapshot, so external edits survive a reset when the snapshot can safely round-trip.
• Structured SecretRef object values are rendered read-only in form text inputs to prevent accidental object-to-string corruption.

• Debug: status/health/models snapshots + event log + manual RPC calls (status, health, models.list).
• The event log includes Control UI refresh/RPC timings, slow chat/config render timings, and browser responsiveness entries for long animation frames or long tasks when the browser exposes those PerformanceObserver entry types.
• Logs: live tail of gateway file logs with filter/export (logs.tail).
• Update: run a package/git update + restart (update.run) with a restart report, then poll update.status after reconnect to verify the running gateway version.

• For isolated jobs, delivery defaults to announce summary. You can switch to none if you want internal-only runs.
• Channel/target fields appear when announce is selected.
• Webhook mode uses delivery.mode = "webhook" with delivery.to set to a valid HTTP(S) webhook URL.
• For main-session jobs, webhook and none delivery modes are available.
• Advanced edit controls include delete-after-run, clear agent override, cron exact/stagger options, agent model/thinking overrides, and best-effort delivery toggles.
• Form validation is inline with field-level errors; invalid values disable the save button until fixed.
• Set cron.webhookToken to send a dedicated bearer token, if omitted the webhook is sent without an auth header.
• Deprecated fallback: stored legacy jobs with notify: true can still use cron.webhook until migrated.

• chat.send is non-blocking: it acks immediately with { runId, status: "started" } and the response streams via chat events.
• Chat uploads accept images plus non-video files. Images keep the native image path; other files are stored as managed media and shown in history as attachment links.
• Re-sending with the same idempotencyKey returns { status: "in_flight" } while running, and { status: "ok" } after completion.
• chat.history responses are size-bounded for UI safety. When transcript entries are too large, Gateway may truncate long text fields, omit heavy metadata blocks, and replace oversized messages with a placeholder ([chat.history omitted: message too large]).
• Assistant/generated images are persisted as managed media references and served back through authenticated Gateway media URLs, so reloads do not depend on raw base64 image payloads staying in the chat history response.
• When rendering chat.history, the Control UI strips display-only inline directive tags from visible assistant text (for example [[reply_to_*]] and [[audio_as_voice]]), plain-text tool-call XML payloads (including <tool_call>...</tool_call>, <function_call>...</function_call>, <tool_calls>...</tool_calls>, <function_calls>...</function_calls>, and truncated tool-call blocks), and leaked ASCII/full-width model control tokens, and omits assistant entries whose whole visible text is only the exact silent token NO_REPLY / no_reply or the heartbeat acknowledgement token HEARTBEAT_OK.
• During an active send and the final history refresh, the chat view keeps local optimistic user/assistant messages visible if chat.history briefly returns an older snapshot; the canonical transcript replaces those local messages once the Gateway history catches up.
• Live chat events are delivery state, while chat.history is rebuilt from the durable session transcript. After tool-final events the Control UI reloads history and merges only a small optimistic tail; the transcript boundary is documented in WebChat.
• chat.inject appends an assistant note to the session transcript and broadcasts a chat event for UI-only updates (no agent run, no channel delivery).
• The chat header shows the agent filter before the session picker, and the session picker is scoped by the selected agent. Switching agents shows only sessions tied to that agent and falls back to that agent's main session when it has no saved dashboard sessions yet.
• On desktop widths, chat controls stay on one compact row and collapse while scrolling down the transcript; scrolling up, returning to the top, or reaching the bottom restores the controls.
• Consecutive duplicate text-only messages render as one bubble with a count badge. Messages that carry images, attachments, tool output, or canvas previews are left uncollapsed.
• The chat header model and thinking pickers patch the active session immediately through sessions.patch; they are persistent session overrides, not one-turn-only send options.
• Typing /new in the Control UI creates and switches to the same fresh dashboard session as New Chat. Typing /reset keeps the Gateway's explicit in-place reset for the current session.
• The chat model picker requests the Gateway's configured model view. If agents.defaults.models is present, that allowlist drives the picker. Otherwise the picker shows explicit models.providers.*.models entries plus providers with usable auth. The full catalog stays available through the debug models.list RPC with view: "all".
• When fresh Gateway session usage reports include current context tokens, the chat composer area shows a compact context usage indicator. It switches to warning styling at high context pressure and, at recommended compaction levels, shows a compact button that runs the normal session compaction path. Stale token snapshots are hidden until the Gateway reports fresh usage again.

Talk mode uses a registered realtime voice provider. Configure OpenAI with talk.realtime.provider: "openai" plus talk.realtime.providers.openai.apiKey, or configure Google with talk.realtime.provider: "google" plus talk.realtime.providers.google.apiKey. The browser never receives a standard provider API key. OpenAI receives an ephemeral Realtime client secret for WebRTC. Google Live receives a one-use constrained Live API auth token for a browser WebSocket session, with instructions and tool declarations locked into the token by the Gateway. Providers that only expose a backend realtime bridge run through the Gateway relay transport, so credentials and vendor sockets stay server-side while browser audio moves through authenticated Gateway RPCs. The Realtime session prompt is assembled by the Gateway; talk.client.create does not accept caller-provided instruction overrides.

In the Chat composer, the Talk control is the waves button next to the microphone dictation button. When Talk starts, the composer status row shows Connecting Talk..., then Talk live while audio is connected, or Asking OpenClaw... while a realtime tool call is consulting the configured larger model through talk.client.toolCall.

Maintainer live smoke: OPENAI_API_KEY=... GEMINI_API_KEY=... node --import tsx scripts/dev/realtime-talk-live-smoke.ts verifies the OpenAI browser WebRTC SDP exchange, Google Live constrained-token browser WebSocket setup, and the Gateway relay browser adapter with fake microphone media. The command prints provider status only and does not log secrets.

• Click Stop (calls chat.abort).
• While a run is active, normal follow-ups queue. Click Steer on a queued message to inject that follow-up into the running turn.
• Type /stop (or standalone abort phrases like stop, stop action, stop run, stop openclaw, please stop) to abort out-of-band.
• chat.abort supports { sessionKey } (no runId) to abort all active runs for that session.

• When a run is aborted, partial assistant text can still be shown in the UI.
• Gateway persists aborted partial assistant text into transcript history when buffered output exists.
• Persisted entries include abort metadata so transcript consumers can tell abort partials from normal completion output.

{
  gateway: {
    controlUi: { allowInsecureAuth: true },
    bind: "tailnet",
    auth: { mode: "token", token: "replace-me" },
  },
}

allowInsecureAuth is a local compatibility toggle only:

• It allows localhost Control UI sessions to proceed without device identity in non-secure HTTP contexts.
• It does not bypass pairing checks.
• It does not relax remote (non-localhost) device identity requirements.

{
  gateway: {
    controlUi: { dangerouslyDisableDeviceAuth: true },
    bind: "tailnet",
    auth: { mode: "token", token: "replace-me" },
  },
}

• Successful trusted-proxy auth can admit operator Control UI sessions without device identity.
• This does not extend to node-role Control UI sessions.
• Same-host loopback reverse proxies still do not satisfy trusted-proxy auth; see Trusted proxy auth.

• gatewayUrl is stored in localStorage after load and removed from the URL.
• If you pass a full ws:// or wss:// endpoint via gatewayUrl, URL-encode the gatewayUrl value so the browser parses the query string correctly.
• token should be passed via the URL fragment (#token=...) whenever possible. Fragments are not sent to the server, which avoids request-log and Referer leakage. Legacy ?token= query params are still imported once for compatibility, but only as a fallback, and are stripped immediately after bootstrap.
• password is kept in memory only.
• When gatewayUrl is set, the UI does not fall back to config or environment credentials. Provide token (or password) explicitly. Missing explicit credentials is an error.
• Use wss:// when the Gateway is behind TLS (Tailscale Serve, HTTPS proxy, etc.).
• gatewayUrl is only accepted in a top-level window (not embedded) to prevent clickjacking.
• Non-loopback Control UI deployments must set gateway.controlUi.allowedOrigins explicitly (full origins). This includes remote dev setups.
• Gateway startup may seed local origins such as http://localhost:<port> and http://127.0.0.1:<port> from the effective runtime bind and port, but remote browser origins still need explicit entries.
• Do not use gateway.controlUi.allowedOrigins: ["*"] except for tightly controlled local testing. It means allow any browser origin, not "match whatever host I am using."
• gateway.controlUi.dangerouslyAllowHostHeaderOriginFallback=true enables Host-header origin fallback mode, but it is a dangerous security mode.