Spice Cloud Update: Up to 93% Lower P99 Latency During Refreshes and AI Decisions in SQL

Over the past month, we strengthened security across Spice Cloud and the runtime, made it easier to integrate AI with your development workflow, and significantly improved runtime performance.

The new Management MCP server lets coding agents like Claude Code, Codex, and Cursor create and deploy Spice Cloud projects and manage secrets on your behalf. In the runtime, Spice v2.4 cuts P99 latency during data refreshes by up to 93%, and uncached primary-key lookups now run twice as fast. It also brings AI decisions to SQL for classifying, scoring, and filtering rows with any model in your spicepod.

New in Spice Cloud

Security updates

Attacks on developer tools and infrastructure are rapidly evolving in both scale and sophistication. Over the past few weeks, we strengthened how Spice Cloud protects your credentials and how we operate the service.

In Spice Cloud:

• Secrets stay out of spicepods. When a plain-text credential like a database password is added to a spicepod.yaml, the editor now displays a warning and asks you to move it before saving. Move to secrets stores each value as a project or org secret and puts a ${secrets:NAME} reference in its place.
• Clearer OAuth consent. When a client such as a coding agent asks to connect, the consent screen now names the host that receives your approval, and marks clients that registered themselves as unverified.
• Patched sign-in and web dependencies. We applied the November 3 security releases for Better Auth, which tightens passkey origin checks, and Next.js, which fixes cross-user cache disclosure and cookie exposure issues.

In how we run Spice Cloud:

• Short-lived credentials. Our deployment pipelines and internal service calls now authenticate with short-lived tokens issued for each run.

In the Spice runtime, recent releases stop trusting the compromised eMudhra certificate authorities (read how Spice responded). v2.4.0 and v2.4.1 also block requests to internal network addresses, cap the memory that malformed files and oversized responses can use, harden Flight SQL, and patch two published advisories. Visit the v2.4.0 and v2.4.1 release notes for more details, and the Spice Security page for our full security posture. To report a vulnerability, email security@spice.ai.

Manage Spice from your coding agent

Figure 1: The Management MCP server URL in Project Settings > Endpoints

The Spice Management API now has an MCP server at https://api.spice.ai/mcp. It exposes every public Management API operation as its own MCP tool, so Claude Code, Codex, Cursor, and other agents can create projects, deploy, manage secrets, and read instance logs.

In Project Settings > Endpoints, Connect a client gives you a one-click install for Claude and Cursor, and a ready-to-run command for Claude Code, Codex, Grok, and OpenCode. Approve the connection in your browser and choose the org, and the agent works with only the scopes you grant.

To manage the clients your agents use, open your account Settings > User OAuth from the avatar menu.

AI decisions in Spice Cloud

Spice Cloud now supports AI decisions in two ways: decision functions in SQL, and the Decisions API over HTTP.

A decision function asks a model a question about each row and returns the answer as a column you can filter, group, and join on.

In this example, ai_classify asks a model to label each of NYC’s roughly 260 taxi zones by its name, and SQL joins those labels back onto every trip:

Figure 2: ai_classify labels each taxi zone, then SQL aggregates trips by label

The model sees each zone name once, and the join fans the labels out across every trip.

To try the Decisions API, open the Decisions tab in the Playground and ask a model about pasted text, with no code. The Spice Runtime v2.4 Updates cover the Decisions API, reranking, and the full set of SQL decision functions.

Read the full Spice Cloud changelog for every update.

Spice Runtime v2.4 Updates

Spice v2.4 significantly improves latency during data refreshes, and brings AI decisions to SQL and HTTP.

Spice Cayenne performance improvements

Figure 3: P99 point-lookup latency during full-refresh commits: 130 to 690 ms on v2.3, and 44 to 50 ms on v2.4

Spice Cayenne is faster across lookups, loads, and refreshes in v2.4.

Steady latency during refreshes. Full and append refreshes now write on dedicated threads, so queries keep their capacity while a refresh commits. In a test with 500 point-lookup joins per second against a 2.2M-row, 8 GB table and a full refresh every 5 minutes, P99 latency in 15-second windows during each refresh commit fell from between 130 and 690 ms to a steady 44 to 50 ms.

2x faster primary-key lookups. Uncached primary-key point lookups run 2.11x to 2.41x faster at P50 on a 10M-row file-mode table, from 1 to 128 clients.

The release also speeds up first loads, append visibility, append-only ingestion, metastore writes, and caching mode. The v2.4.0 release notes list each test.

Spice Cayenne is now the default accelerator

Datasets and views that don’t set acceleration.engine now use Spice Cayenne by default instead of Arrow, except on native Windows, which keeps Arrow. On Windows, run Spice in Docker or WSL to use Cayenne.

The other accelerators are now deprecated, with removal planned for v2.5. To move an existing acceleration, follow the Cayenne migration guide.

Decisions API

Figure 4: A Decisions API request with a predicate, a choice, and a score question

The new OpenAI-compatible POST /v1/decisions API asks a model a set of questions about an input and returns an answer for each one. It works with TypeSafe Jev, OpenAI’s GPT-6 Luna, or any chat model in your spicepod. TypeSafe decision models return calibrated probabilities, while other models return their own estimates.

Predicate answers return a single probability, while choice and score answers return a probability for each option, plus a confidence:

Figure 5: The response: the message is likely urgent (0.9), belongs to billing (0.8), and reads as annoyed to furious

Rerank with decisions

Decision models also work as search rerankers. For each candidate, the model judges how likely it is to be relevant to the query, and that probability becomes the candidate’s rerank_score.

Decisions API docs · Rerank docs

AI functions for decisions in SQL

Figure 6: Classify, score, and filter rows with any model in SQL

Spice’s AI functions now cover decision work. You can classify, score, and filter data with any model directly in SQL, and get back probabilities and a confidence score:

AI functions reference

OpenTelemetry log export

Figure 7: Export runtime logs alongside metrics over OTLP

The runtime can now send structured logs through its existing OpenTelemetry Protocol (OTLP) exporter, alongside metrics. The runtime batches log delivery on a dedicated export runtime, so exporting logs stays off the query path.

The runtime.query.log_errors setting controls which query failures get logged, and failure records leave out the SQL text.

Observability docs

DataFusion v55 and distributed queries

Spice v2.4 upgrades to Apache DataFusion v55 and Arrow v59.3, alongside Ballista v55, Vortex v0.86, and Iceberg v0.11. ORDER BY ... LIMIT queries now skip row groups that can’t change the result, and the topk_tpch suite ran about 43% faster. Distributed queries use adaptive query execution by default: at TPC-H scale factor 1,000, Ballista finished all 22 queries in 502 s, against 898 s for Apache Spark 4.1.3 on the same cluster.

New data connectors and formats

• Hugging Face datasets: The new Hugging Face connector queries and accelerates Parquet, CSV, TSV, JSON, and ORC files straight from Hugging Face, with from: hf://datasets/....
• ORC files: Listing connectors such as S3 now read file_format: orc. Optional _location and _last_modified metadata columns show which object each row came from and when it last changed.
• Iceberg without a catalog: Point an Iceberg dataset at a table’s root directory on local or object storage. Spice picks up the latest metadata file on each query and refresh, so new commits show up without a restart. Append refreshes now follow the table’s snapshots, and Spice reads Iceberg v3 deletion vectors.

Additional runtime updates

• SQS now supported through S3 notifications: With refresh_mode: changes, an accelerated S3 dataset now loads new objects as soon as SQS delivers their S3 event notifications.
• Results-cache warmup: After a restart, Spice replays recorded query shapes, so the first queries read from a warm SQL results cache.
• MySQL connection pools (v2.4.1): Pools replace each connection after mysql_connection_max_lifetime (5m by default), so they spread across every server behind a load-balanced hostname such as an Aurora reader endpoint. mysql_pool_min and mysql_pool_max now apply with mysql_connection_string.
• v2.4.0 includes breaking changes, including to SQL results-cache expiry and on_conflict on Cayenne. Read the breaking changes before you upgrade, then see the v2.4.0 and v2.4.1 release notes for more details.

SDK Updates

• spice.js: Upgraded from v3.2.0, v3.3.0 adds query cancellation with an AbortSignal, runtimeStatus(), and refreshDataset(), and deprecates refreshAcceleration(). v3.3.0 also fixes intermittent HTTP query failures on Node, sends API keys on plaintext gRPC, and binds typed Flight parameters correctly. If you set only one of flightUrl or httpUrl, the client now uses the matching address for the other, and an empty URL or a Flight authentication failure now throws an error.
• spicepy: Upgraded from v4.0.0, v4.0.1 sends API keys as a Bearer token on Flight and ADBC calls to self-managed runtimes, and binds numpy and pandas scalars as query parameters. refresh_dataset() and runtime_status() now raise SpiceAIError with the runtime’s error message instead of requests.HTTPError. v4.0.1 requires certifi 2026.10.19 or later.
• spice-rs: Upgraded from v4.0.0, v5.0.0 moves to Arrow 59 and binds chrono and std::time values as query parameters. Parameterized queries now work with API keys, and queries retry after a connection reset. Applications that also depend on arrow must move to Arrow 59.
• spice-java: Upgraded from v0.8.0, v0.8.1 accepts any non-empty API key and sends it on refreshDataset. It also fixes the NoClassDefFoundError that v0.8.0 raised on every TLS connection, including to Spice Cloud.
• spice-dotnet: Upgraded from v0.4.0, v0.5.0 adds GetRuntimeStatusAsync() for /v1/status, which all six SDKs now support. It also accepts any non-empty API key and supports grpc:// Flight addresses.
• gospice: Upgraded from v9.0.0, v9.1.0 binds time.Time, time.Duration, pointers, sql.Null* types, and named types as query parameters. It also accepts any non-empty API key and uses one Flight session for each client. v9.1.0 requires Go 1.26 or later, and Decimal128Param and Decimal256Param now read Arrow little-endian bytes, so check those calls before you upgrade.

We’ll be back next month with another round of updates. Join us in the Spice Community Slack to follow along with the latest news or if you have any questions!