title: 'Spice Cloud Platform' sidebar_label: 'Spice Cloud' description: 'Connect a self-hosted Spice runtime to the Spice Cloud Platform to centralize task history and runtime observability across deployments.' keywords: [ spice.ai, spice cloud, monitoring, observability, task history, self-hosted, management, api key, ] tags:
A self-hosted Spice runtime — running in Kubernetes, Docker, on a VM, or on a laptop — can connect to the Spice Cloud Platform (opens in a new tab) to centralize task history (opens in a new tab) across one or many runtimes. Once connected, query logs, refresh activity, AI tool invocations, and runtime errors from every connected instance are visible in a single SCP app, queryable as a standard runtime.task_history table, and retained beyond each runtime's local in-memory window.
The connection is configured declaratively in spicepod.yaml with a top-level management: block. No CLI command, sidecar, or extra agent is required — the runtime itself streams task history to SCP over Arrow Flight.
The management: block enables a single export path: rows from the local runtime.task_history (opens in a new tab) accelerated table are appended every 5 seconds to the runtime.task_history dataset of the target Spice Cloud app. Each row is a span representing one unit of execution — a SQL query, an AI chat completion, a dataset refresh, a tool call — including its inputs, outputs, duration, and error status.
The export covers a rolling 3-day window from the runtime's current time. Records older than 3 days are not backfilled when a runtime first connects.
The following are not sent through this path:
For a full observability stack on self-hosted deployments, combine management: with the metrics and tracing exporters above. management: covers task history; the others cover everything else.
From the Spice Cloud Console, create a new app (any name and visibility). Copy the API key shown on the app's settings page — it is only displayed at creation time and on demand from the API Keys tab.
To create the app from the CLI:
The output includes the app's primary API key. Treat the key as a secret — anyone with the key can write task history to (and read it from) the app.
For an existing app, retrieve the current key with:
Add a management: block to spicepod.yaml on each self-hosted runtime. The API key should be sourced from a secret store (opens in a new tab) rather than hard-coded.
With SPICEAI_API_KEY set in the environment (or .env file picked up by the env secret store (opens in a new tab)), the runtime connects to the default Spice Cloud region on startup. On success, the runtime logs:
INFO runtime::management: Connected to Spice Cloud for management and monitoring
By default, the runtime targets the Spice Cloud default region. To target a specific region, set region in params:
The region must match the region the API key was issued in. Available regions are listed in the Spice Cloud Console.
For VPC-peered or otherwise-routed Cloud endpoints, override the Flight endpoint explicitly with data_endpoint:
data_endpoint takes precedence over region. The scheme must be https:// for production; http:// is accepted for local testing only.
Set enabled: false to turn off the export without removing the block (useful for staging vs. production overrides):
The management: block lives at the top level of spicepod.yaml, alongside runtime:, datasets:, and models:.
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
enabled | boolean | No | true | Whether the management export is active. When false, the block is parsed but no connection is established. |
api_key | string | Yes | — | API key for the target Spice Cloud app. Resolved through any secret store (opens in a new tab) via ${secrets:KEY}. |
params.region | string | No | — | Spice Cloud region (e.g. us-east-1). Used to build the Flight endpoint when data_endpoint is not set. |
params.data_endpoint | string | No | Built from region | Flight endpoint URL override. Must be https:// or http://. Takes precedence over region. |
end_time within the last 3 days. Older rows are not backfilled.runtime.task_history.enabled is false, the management export stays initialized but never sends data. Task history is enabled by default; see the task history reference (opens in a new tab) for retention and capture options.After restarting the runtime, confirm the connection is established:
Check the runtime log for the connection line:
INFO runtime::management: Connected to Spice Cloud for management and monitoring
For deeper logging during setup, run with SPICED_LOG="spiced=INFO,runtime=INFO,runtime::management=DEBUG". The runtime reads its log filter from SPICED_LOG, not RUST_LOG (see Tracing (opens in a new tab)).
Run a query against the local runtime to generate task history:
Wait 5–10 seconds, then query the target Spice Cloud app's task history. From any client logged in to Spice Cloud:
The SELECT 1 query issued on the self-hosted runtime appears in the result set.
A single Spice Cloud app can receive task history from any number of self-hosted runtimes — share the same api_key across them. To distinguish runtimes when querying the consolidated table, tag each runtime in spicepod.yaml:
The name field is recorded with every task history span and is the simplest way to filter:
| Symptom | Likely cause | Resolution |
|---|---|---|
Startup error: Missing required secret: api_key. Specify a value. | api_key is empty or the referenced secret is not defined. | Ensure the secret store has the key and the ${secrets:...} reference matches. Test secret resolution with spice run and the env secret store (opens in a new tab). |
Startup error: Failed to create data connector for cloud management | The Flight endpoint cannot be reached, or the API key is rejected. | Verify outbound TLS connectivity to https://<region>-prod-aws-flight.spiceai.io. Regenerate the API key from the Spice Cloud Console; confirm region matches the key's region. |
| No connection log line after restart | The management: block was not parsed — typically a YAML indentation issue. | Confirm management: is at the top level of spicepod.yaml, not nested under runtime:. Check spice run startup logs for spicepod-parse warnings. |
| Records never appear in the Cloud app |
For detailed export logging, set SPICED_LOG="spiced=INFO,runtime=INFO,runtime::management=TRACE" and watch for per-flush log lines like Exported {n} task history records.
api_key securely.| Task history is disabled, or the runtime hasn't run any spans yet. |
Confirm runtime.task_history.enabled is true (the default). Issue a query against the local runtime, then wait at least 5 seconds for the next export tick. |
UNAUTHENTICATED on Flight handshake | Wrong API key, key from a different app, or key from a different region. | Regenerate the API key from the target app's API Keys tab and update the secret store. Verify params.region matches the app's region. |
Repeated Failed to export runtime task history records warnings | Transient network failure between the runtime and Spice Cloud. | The runtime retries automatically on the next 5-second tick. If errors persist, check outbound network policy, DNS resolution, and the Spice.ai status page (opens in a new tab). |
Records appear, but spicepod_name is empty for some runtimes | The runtime's spicepod has no name: field. | Set name: at the top of spicepod.yaml on each runtime so consolidated rows can be filtered. |