date: 2026-06-05 title: 'Spice v2.0-stable (Jun 5, 2026)' type: blog authors: [phillipleblanc] tags: [release, distributed-query, cayenne, datafusion, cdc, cedar, postgres, mongodb, security, snowflake, udf, http, kafka, arrow, ducklake, data-connector]
53 releases since Spice 1.0-stable, Spice.ai OSS has reached the 2.0-stable milestone! 🎉
Spice v2.0.0 is the next major release of Spice and a major milestone in the project's development, advancing Spice from a single-node engine into a distributed data and query platform built for enterprise AI agents. These agents need low-latency, governed access to data spread across many production systems, and because they generate their own queries autonomously, that access has to be sandboxed, observable, and able to absorb occasional heavy analytical queries without overwhelming the underlying systems. The release is headlined by multi-node distributed query, now generally available — multi-active, highly-available, and object-store-native, built on Apache Ballista — distributing both query execution and ingestion across executors with data-local routing and per-executor statistics for distributed join planning. Alongside it, the Spice Cayenne data accelerator is generally available, built on the Vortex compressed columnar format, with a high-throughput CDC write path, MERGE INTO, SQL-defined partitioning, inline writes, a dedicated compaction runtime, and write-path statistics for distributed join sizing. The engine also moves to DataFusion v52 with sort pushdown, a rewritten merge join, and dynamic filters, and the Spice CLI is rewritten in Rust as a single self-contained binary.
v2.0 also expands real-time and write-path capabilities across the platform: native CDC from MongoDB Change Streams and PostgreSQL WAL logical replication, durable Kafka CDC offsets, DML write-back for PostgreSQL, Snowflake, DynamoDB, Arrow, and DuckLake, DDL and MERGE INTO for Iceberg catalogs, mutual TLS across server endpoints and outbound connectors, HashiCorp Vault and Azure Key Vault secret stores, user-defined functions, hybrid search with Elasticsearch and DuckDB HNSW vector indexes, provider-aware LLM prompt caching, and the Responses API across all model providers.
/v1/queriesCREATE TABLE/DROP TABLE and MERGE INTO for Iceberg catalogsSpice v2.0 includes several breaking changes. Review the breaking changes section before upgrading.
AI/ML support including local LLM/ML model and hosted LLM inference is now included in the default Spice build and image. The separate models build variant has been removed.
With models now included by default, the data-only distribution (without AI/ML support) is only published in nightly builds. Official production-ready data-only distributions are available exclusively through Spice Cloud and the Enterprise release.
A new Network Attached Storage (NAS) distribution with built-in SMB and NFS data connector support is also available in nightly builds and with Spice.ai Enterprise.
| Distribution / Variant | Open Source | Spice Cloud | Enterprise |
|---|---|---|---|
| Default | ✅ | ✅ | ✅ |
| Data | Nightly only | ✅ | ✅ |
| NAS (SMB + NFS) | Nightly only | ❌ | ✅ |
| Metal (macOS) | ✅ | ✅ | ✅ |
| CUDA (Linux) | Nightly only | ✅ | ✅ |
| Allocator variants | Nightly only | ✅ | ✅ |
| ODBC connector | Local build only | ✅ | ✅ |
Native Windows builds are no longer provided; use WSL for local development. For more details, see the Distributions documentation.
The Spice Cayenne data accelerator is generally available in v2.0, with a major focus across the release candidates on write-path throughput, correctness, and distributed operation.
Write path & ingest:
DELETE statements whose filters identify primary keys directly — including composite keys expressed as (k1, k2) IN ((...), (...)) — skip the table scan entirely.Query & planning:
JoinSelection can correctly size joins without rescans.SQL & catalog:
MERGE INTO for Cayenne catalog tables, distributed across executors in cluster mode.PARTITION BY in SQL: Define partitioning directly in CREATE TABLE ... PARTITION BY (...); metadata is persisted in the catalog and survives restarts.partition_by: [col1, col2] with hierarchical path-like keys.Correctness: Synchronized partition commits, correct NULL-sentinel handling for nullable partition expressions, tombstoned inline-checkpointed rows on upsert (preventing duplicate primary keys), and live reads through expired protected snapshots.
Spice.ai Enterprise feature. See High Availability.
Distributed Query is generally available. Built on Apache Ballista, it distributes query execution across multiple active executor nodes with no single point of failure, reading directly from object storage rather than relying on a central cluster.
Distributed query supports two execution modes:
/v1/queries API materialize results to object storage for later retrieval — best for long-running analytical and batch workloads.Key capabilities:
DoPut writes to the responsible executors.JoinSelection can size joins correctly, fixing out-of-memory conditions on large semi-joins./v1/ready gates on a configurable executor quorum for safe rolling deployments; scheduler readiness additionally waits for executor partition loads; executor heartbeat timeout reduced from 180s to 30s.MERGE INTO.scheduler_active_executors_count), distributed runtime.task_history replication, and a Grafana dashboard.runtime.params.shuffle_location: s3://... complete reliably with executor-environment-derived S3 clients.Several capabilities in this section are Spice.ai Enterprise features. See Enterprise Security.
Mutual TLS across the platform:
client_auth_mode: request (optional, for migration windows) or required (strict) client-certificate verification.SIGHUP for zero-downtime rotation.spice sql REPL supports mTLS client auth.Authentication & Authorization (Spice.ai Enterprise):
runtime.authorization governs allow/deny access across datasets, models, tools, and endpoints. Combined with identity SQL functions (current_principal(), current_principal_email(), current_principal_groups()), policies enforce per-principal row-level filtering and column masking.New Secret Stores: HashiCorp Vault (KV v1/v2; token, approle, kubernetes, and jwt auth with automatic lease renewal) and Azure Key Vault (service principal, managed identity, workload identity, Azure CLI, or auto-detect; sovereign cloud support).
Hardening:
DoGet path and async query endpoints.allowed_hosts: MCP servers can be restricted to an explicit allowlist of upstream hosts.See Change Data Capture (CDC) for an overview of CDC in Spice.
refresh_mode: changes stream changes natively into any local accelerator — no Debezium or Kafka required.pgoutput decoding, with automatic per-replica slot management, an initial REPEATABLE READ bootstrap snapshot, and durable LSN acknowledgement.Spice v2.0 turns more connectors and catalogs into full read/write tables:
INSERT, UPDATE, and DELETE write-back on PostgreSQL datasets, with foreign-key metadata exposed via the PostgreSQL catalog connector.INSERT, UPDATE, and DELETE write-back on Snowflake datasets.INSERT, UPDATE, and DELETE for DynamoDB, complementing read and CDC streaming.CREATE TABLE and DROP TABLE via FlightSQL and /v1/sql for Iceberg, with .See the SQL Reference for the full SQL surface area.
ST_* UDFs for geometry workloads.flatten_json, json_tree, and flatten_json_properties table-valued functions for JSON transformation and schema decomposition (with options such as expand_maps). See JSON Functions and Operators.obj_description, col_description), so BI tools and psql surface Spice metadata.CommandStatementSubstraitPlan support for clients submitting Substrait-encoded plans.\x for a vertical key-value layout on wide result sets.JOIN ON conditions for Spice Cloud federation and correct EXISTS/ subquery handling in the federation analyzer.columns[].type, columns[].nullable) and fully resolved on first reference, reducing startup time and memory for large spicepods.Ctrl-C, and cancelled HTTP requests cancel the query end-to-end.acceleration.storage_profile (auto, local_ssd, ebs, tmpfs) applies storage-aware defaults across DuckDB, SQLite, Turso, and Cayenne file-mode accelerators; auto detects the backing storage.refresh_mode: snapshot (Spice.ai Enterprise): Point-in-time snapshot acceleration with SQLite/Turso WAL flushing and Cayenne metastore slice integration, now reporting accurate readiness when no snapshot exists yet./v1/datasets?status=true and /v1/models?status=true return structured error objects (category, , ) and human-readable fields; the CLI shows an column.Spicepods now support version: v2, the default for spice init, while v1 spicepods continue to work with automatic migration of deprecated fields.
| Version | Status |
|---|---|
v2 | Default. Used by spice init. |
v1 | Supported. Deprecated fields auto-migrate. |
v1beta1 | Removed. No longer accepted. |
| v1 (deprecated) | v2 (preferred) | Notes |
|---|---|---|
runtime.results_cache | runtime.caching.sql_results | All fields migrate automatically. cache_max_size → max_size. |
runtime.memory_limit | runtime.query.memory_limit | Auto-migrated. query.memory_limit takes priority if both set. |
runtime.temp_directory | runtime.query.temp_directory | Auto-migrated. query.temp_directory takes priority if both set. |
dataset.invalid_type_action | dataset.unsupported_type_action | Auto-migrated. v2 adds a new string variant. |
New v2 fields include runtime.ready_state, runtime.query.spill_compression, runtime.caching.sql_results.stale_while_revalidate_ttl, runtime.caching.sql_results.encoding, scheduler partition-assignment configuration, and catalog.access: read_write_create.
New connectors:
vector_search() kNN, text_search() BM25, and rrf() fusion — plus Elasticsearch as a backing vector engine, direct FTS engine configuration, and index lifecycle controls.New catalog connectors for PostgreSQL, MySQL, MSSQL, and Snowflake, using native metadata catalogs for schema and table discovery. Unity Catalog compatibility extends to OSS Unity Catalog deployments, and DDL-defined catalogs can expose and query views.
HTTP connector: OAuth2 refresh-token authentication, query-parameter and no-limit pagination, dynamic request headers parameterised from query predicates, subquery-driven request parameters for fan-out queries, response metadata as queryable columns, map-to-array conversion, shared and persistent rate-control state across restarts and replicas, no caching of transient 429/5xx errors, and a correctly populated fetched_at column.
JSON ingestion: Single-object documents, JSONL, BOM-prefixed input, Socrata SODA responses, format auto-detection, and RFC 6901 json_pointer extraction of nested payloads.
Databricks: Resilience controls, Unity Catalog-aware permission prechecks with structured advisory errors, Classic SQL Warehouse foreign-table compatibility, connect_timeout/client_timeout parameters, a Databricks SQL dialect for federation, and Delta Lake column mapping (Name and Id modes).
Other connector improvements: MongoDB SRV support; MySQL mysql_zero_date_behavior; Snowflake OBJECT, MAP, GEOGRAPHY, GEOMETRY, VECTOR, and TIMESTAMP_LTZ types plus key-pair auth; ClickHouse Date32; S3 s3_url_style for path-style addressing and faster Parquet reads; GraphQL custom auth headers; Oracle and MSSQL sort/limit pushdown; GitHub GraphQL resilience; and improved Kafka reliability.
response.output_text.delta events and Authorization: Bearer header support._match column identifying the best-matching element.rerank() UDTF: Reorder results from vector_search, text_search, or rrf using any registered chat model as a reranker, with automatic query propagation and pushdown support./v1/mcp) on rmcp v1.5.0, native auth for streamable HTTP tools (mcp_auth_token, mcp_headers), external MCP server tool calls traced in task history, and configurable allowed_hosts.vector_engine: duckdb uses DuckDB's HNSW index for fast approximate nearest-neighbor search without an external vector store. In v2.0.0, the DuckDB VSS extension is statically linked into the bundled DuckDB, so HNSW vector search works out-of-the-box on clean machines with no extension download. HNSW indexes are preserved across data refresh, and cosine_distance pushes down via array_cosine_distance.rrf()), backed by Tantivy, Elasticsearch, or DuckDB.row_id columns are validated during dataset initialization.Improvements across Caching:
runtime.caching.sql_results.stale_while_revalidate_ttl serves stale results while revalidating in the background.zstd) for SQL results cache entries.Apache DataFusion is upgraded to v52.5 over the course of the release cycle, bringing:
DESC on ASC-sorted files.MIN/MAX aggregates and hash-join build sides prune files, row groups, and rows during execution.CASE Expressions, statistics caching, and prefix-aware list-files caching for faster planning.TableProvider DELETE/UPDATE hooks and the RelationPlanner API for extensible SQL planning.try_cast_to errors on overflow instead of silently producing NULLs.Additional engine work: default query memory limit raised from 70% to 90% with GreedyMemoryPool, partial aggregation optimization for FlightSQLExec, improved partitioned query planning, and metastore transaction support to prevent concurrent conflicts.
The Spice CLI is completely rewritten from Go to Rust — a single spice binary built from the same codebase as spiced, with full feature parity across 27+ commands.
spice query: Interactive REPL for async queries with multi-line SQL, progress indication, and cancellation.spice dataset configure: Non-interactive flag-based configuration (--from, --description, --param KEY=VALUE, --set) alongside interactive prompts.spice completions: Shell completion script generation.--output=json: Machine-readable output for scripting; spice login --output adds env, json, and keychain modes.spice init writes a yaml-language-server schema directive for IDE completions.runtime.telemetry.metric_prefix), delta temporality by default, and OTLP resource attributes via runtime.telemetry.properties.query_executions metric gains a datasets dimension for per-dataset query attribution.rows_written, bytes_written, and dataset_acceleration_size_bytes for acceleration refresh and Flight DoPut/ADBC ingestion, and EXPLAIN ANALYZE metrics in FlightSQLExec.localpod child datasets correctly track parent refreshes when the parent uses the in-memory Arrow accelerator.JOIN ON conditions, fixing rejected federated queries.refresh_mode: snapshot: No longer reports Ready with empty data when no snapshot exists.fetched_at column is correctly populated.TOP N pushdown; DuckDB DELETE/UPDATE on full and caching refresh modes; Turso checked arithmetic for timestamp conversions; ODBC queries no longer silently return 0 rows on failure; Flight GetFlightInfo/DoGet schema parity.| Dependency / Component | Version |
|---|---|
| DataFusion | v52.5 |
| Ballista | v52 |
| Arrow (arrow-rs) | v57.2 |
| DuckDB | v1.5.3 (with statically linked VSS) |
| iceberg-rust | v0.9.1 |
| Turso (libsql) | v0.6.1 |
| Vortex | v0.69.0 |
| delta_kernel | v0.18.2 |
| rmcp (MCP) | v1.5.0 |
| mistral.rs | v0.8.x (candle v0.10.1) |
| ADBC Core | v0.23 |
| Rust toolchain | v1.94.1 |
Models included by default: The separate models build variant has been removed. Local LLM inference is always included in the default build and image.
Windows native builds removed: Use WSL for local development.
Spicepod version defaults to v2: spice init creates version: v2 spicepods. v1 remains supported with auto-migration; v1beta1 is no longer accepted.
Flattened runtime.scheduler configuration: The nested runtime.scheduler.partition_management block is flattened and renamed:
S3 metadata columns renamed: location, last_modified, size → _location, _last_modified, _size.
Default query memory limit changed: Increased from 70% to 90%.
Most v1 spicepods continue to work on v2.0 — v1 remains supported and deprecated fields auto-migrate at load time — so many deployments can upgrade by updating the binary or image alone. The steps below cover the breaking changes that may require manual action. Review each before upgrading a production deployment.
models build variant (and the corresponding -models image tags) has been removed; local LLM inference is always included in the default build and image. If your deployment pinned a models build or -models-tagged image, switch to the default build/image.v2 (recommended)spice init now creates version: v2 spicepods. v1 spicepods remain supported with automatic migration, but v1beta1 is no longer accepted. To move to v2, set version: v2 and update the following fields — each auto-migrates from v1, but updating now clears the deprecation:
| v1 (deprecated) | v2 (preferred) |
|---|---|
runtime.results_cache | runtime.caching.sql_results (cache_max_size → max_size) |
runtime.memory_limit | runtime.query.memory_limit |
runtime.temp_directory | runtime.query.temp_directory |
dataset.invalid_type_action | dataset.unsupported_type_action |
partitioned_write_flush_threshold → partitioned_write_flush_threshold_rows.runtime.query.memory_limit.location, last_modified, size → _location, _last_modified, _size. Update any queries that reference these columns./v1/search always returns an array in matches, even for a single result. Update clients that assumed a scalar value./v1/evals API removed. Remove integrations that depend on it./v1/responses endpoint exclusively. Ensure x.ai integrations target the Responses API.accelerated_refresh → acceleration_refresh, and the last_refresh_time gauge is renamed to include the milliseconds unit. Update dashboards and alerts that reference these metric names.After updating, restart the runtime and verify datasets and models report ready via /v1/datasets?status=true and /v1/models?status=true (the CLI shows a Ready/ERROR column).
New Spice Cookbook recipes added during the v2.0 release cycle:
The Spice Cookbook includes more than 100 recipes to help you get started with Spice quickly and easily.
To upgrade to v2.0.0, use one of the following methods:
CLI:
Homebrew:
Docker:
Pull the spiceai/spiceai:2.0.0 image:
For available tags, see DockerHub.
Helm:
AWS Marketplace:
Spice is available in the AWS Marketplace.
Full Changelog: https://github.com/spiceai/spiceai/compare/v1.11.6...v2.0.0
ST_*refresh_mode: snapshot — storage-aware acceleration defaults and point-in-time snapshot accelerationrerank() UDTFspice CLI with spice query async REPL, shell completions, and --output=jsoncatalog.access: read_write_createread_write access support INSERT.NOT EXISTStypecodeerror_messageERRORMetric renames: accelerated_refresh metrics renamed to acceleration_refresh; last_refresh_time gauge renamed to include the milliseconds unit.
DuckDB parameter rename: partitioned_write_flush_threshold → partitioned_write_flush_threshold_rows.
/v1/search API: Always returns an array in matches, even for single results.
/v1/evals API removed.
Perplexity model provider removed.
x.ai model endpoint: x.ai models exclusively use the /v1/responses endpoint.
time_interval and fix snapshots_creation_policy: on_change by @sgrebnov in #9026PartitionManagementTask for scheduler to update accelerated table partition assignments by @Jeadie in #9378spice query CLI response deserialization by @phillipleblanc in #9588spice upgrade by @phillipleblanc in #10194runtime-cluster crate + Run partition discovery before forwarding refresh to executors by @krinart in #10490private_key in connector PARAMETERS (fixes #10517) by @claudespice in #10559cosine_distance pushdown to DuckDB accelerator via array_cosine_distance by @sgrebnov in #10564full and caching refresh mode datasets by @phillipleblanc in #10632include filter being ignored by @phillipleblanc in #10738spice cloud metrics table output by @claudespice in #10784CayennePropagateFilterAcrossEquiJoinKeys by @sgrebnov in #10863refresh_mode: snapshot reports Ready with empty data when no snapshot exists by @sgrebnov in #10979group_into_sub_batches to reduce sub-batch splits by @sgrebnov in #11059datasets dimension to the query_executions metric by @phillipleblanc in #11138