date: 2026-09-02 title: 'Spice v2.2.1 (Sep 2, 2026)' type: blog authors: [bjchambers] tags: [release, cayenne, adbc, duckdb, sql, data-connector]
Spice v2.2.1 is now available! 🛠️
Spice v2.2.1 is a patch release that improves accelerated data management, query federation, and search. Spice Cayenne now reclaims the storage that upserts and compaction leave behind, and new metrics report what each maintenance pass did. Retention policies apply to more Cayenne datasets. Federated queries push more work down to BigQuery. DuckDB and HTTP datasets keep their memory bounded. Vector search with HuggingFace embedding models returns more relevant results.
A Spice Cayenne (opens in a new tab) acceleration now reclaims the space that a write or a compaction supersedes. Each maintenance pass removes four kinds of state that earlier releases kept:
Cayenne also releases the metastore pages that a pass frees. On a table with frequent refreshes or CDC upserts, disk and metastore usage now stay bounded. No configuration change is required.
Cayenne now applies retention_sql on datasets with refresh_mode: full (opens in a new tab) and on CDC-accelerated datasets with refresh_mode: changes (opens in a new tab). Before this release, only the append refresh path ran the retention pass. A full refresh reloaded every source row, and the rows that a policy had removed came back.
A retention_sql predicate that calls now() also works now. Cayenne prepared the predicate once when the table opened, at a point where DataFusion cannot evaluate now(), so the pass failed and removed nothing. Cayenne now resolves now() at the start of each pass. See Data Refresh (opens in a new tab) for the retention settings.
Cayenne also accepts refresh_append_overlap (opens in a new tab) now. It warns when a dataset sets retention_sql together with mode: memory, or sets indexes, because it cannot apply either setting.
Cayenne now reports its maintenance decisions and its storage footprint as metrics on the cayenne meter. The runtime serves them at /metrics when spiced starts with --metrics <addr>. An operator can now tell a pass that found nothing from a pass that declined. The metrics also show where disk and metastore space goes. See Metrics (opens in a new tab) for the metrics endpoint.
cayenne_compaction_outcome_total and cayenne_maintenance_outcome_total count each pass by outcome, such as committed, no_op, or declined_<reason>.cayenne_compaction_trigger_total names the threshold that requested a pass.cayenne_maintenance_reclaimed_files_total, cayenne_maintenance_reclaimed_bytes_total, and cayenne_maintenance_reclaimed_rows_total count what each pass reclaimed.cayenne_storage_files, cayenne_storage_bytes, and cayenne_storage_rows report the live footprint by storage tier.cayenne_data_dir_files, cayenne_data_dir_bytes, and cayenne_data_dir_snapshot_dirs report what the data directory holds on disk.cayenne_metastore_db_bytes, cayenne_metastore_wal_bytes, and cayenne_metastore_table_rows report the size of the metastore.json_get_str out of the SQL it sends to a source, and it honours query_federation: disabled (opens in a new tab). The v2.2.0 move of the connector into its own crate dropped both settings. A query with a Spice-only function against BigQuery then failed with an unknown-function error.json_get_int and json_get_float (opens in a new tab) into BigQuery SQL. A filter or a join on a JSON value runs in BigQuery instead of in Spice.json_get_int and json_get_float now read a negative JSON number. Every earlier release returned NULL for one.INNER or FULL join between two datasets that both set a description (opens in a new tab) now plans. DataFusion carries the description as schema metadata, and the join failed with a schema mismatch error. This release updates DataFusion with the fix.DELETE (opens in a new tab) with a non-primary-key predicate on a Cayenne upsert table now removes only the rows that match. The delete matched the predicate against superseded row versions as well as live rows. A key whose old version matched therefore lost its live row too. The delete now applies the same visibility rules as a query.CAST(date_col AS TIMESTAMP) comparison failed the scan for a Date32 column. For a Date64 column, the same comparison skipped files that held matching rows, so the query returned fewer rows. The scan cast the file's date statistic through the wrong storage type. It now converts the statistic before it prunes on it.COUNT(*) that Cayenne answered from its maintained row count could miss the rows of a commit that had just become visible. Cayenne now claims a commit's row-count delta before it publishes the rows.ORDER BY on the same key failed to plan on a CDC-accelerated (opens in a new tab) Cayenne dataset after its first transactional commit. An example is WHERE id = 7 ORDER BY id. The runtime rejected the query and returned an error. The scan now forwards its constant columns across the schema cast, and the query plans again.Map columns in file mode: A Cayenne accelerator with mode: file (opens in a new tab) now stores Arrow Map columns, such as the response_headers column of an HTTP dataset. Every write of such a column failed, and the acceleration stayed empty.DuckDB: The bundled DuckDB (opens in a new tab) moves from v1.5.5 to v1.4.4. On v1.5.5, a DuckDB-accelerated dataset that refreshed through an ON CONFLICT ... DO UPDATE upsert grew its memory by about 6 MiB per refresh. The row count stayed steady while the memory grew, and the write eventually failed with an out-of-memory error. The DuckDB project tracks the regression as duckdb/duckdb#25162 (opens in a new tab), and it is present in every v1.5.x release. v1.4.4 holds a steady footprint. Queries that run during a file-mode DuckDB swap also now recover when the swap invalidates their statements. The configured database path stays attached across the swap.
HTTP: The HTTP Data Connector (opens in a new tab) now bounds its response cache. The cache kept every fetched response for the life of the process. An API-proxy workload with request-shaped keys therefore grew by about 1 MB per distinct request.
The cache now holds at most response_cache_max_size_bytes per dataset, 64 MiB by default, and evicts the oldest entries first. It expires an entry by the origin's max-age and Age headers, and it refuses to store a response that sends no-store or no-cache. The new response_cache_fallback_ttl setting keeps a response from an origin that sends no Cache-Control header at all. Two new metrics, response_cache_size_bytes and response_cache_items_count, report occupancy per dataset.
Vector search (opens in a new tab) with a HuggingFace (opens in a new tab) sentence-transformer embedding model, such as all-MiniLM-L6-v2, now returns relevant results. The runtime kept the fixed padding that some models declare in tokenizer.json, so it padded every input to 128 tokens. The padding then dominated the vector of a short input. The runtime now clears that padding and lets the embedding layer own the attention mask. On the MTEB SciFact benchmark with all-MiniLM-L6-v2, nDCG@10 rises from 0.018 to 0.640 (published: 0.645). Full-text search and model2vec static models were not affected.
A search filter on a timezone-aware timestamp column now matches the exact instant when the runtime pushes it down to a DuckDB-backed search index (opens in a new tab). The filter compared whole milliseconds, so a filter inside a millisecond returned different rows than requested.
BIGINT primary keys: Durable write-back (opens in a new tab) now delivers for a table whose primary key is a single BIGINT column. Such a table acknowledged writes but never delivered them to the source.mysql_replication_invalid_checkpoint_behavior: restart now rebuilds when the source has purged its binlog position, even when mysql_replication_initial_snapshot: disabled is set. It previously resumed from the current head and kept the rows that the source had deleted in the gap.cayenne_write_concurrency (opens in a new tab) is raised.No breaking changes.
One behavior change to note before you upgrade:
No new cookbook recipes.
The Spice Cookbook (opens in a new tab) includes more than 104 recipes to help you get started with Spice quickly and easily.
To upgrade to v2.2.1, use one of the following methods:
CLI:
Homebrew:
Docker:
Pull the spiceai/spiceai:2.2.1 image:
For available tags, see DockerHub (opens in a new tab).
Helm:
AWS Marketplace:
Spice is available in the AWS Marketplace (opens in a new tab).
no-store (fixes #13460) by @bjchambers (opens in a new tab) in #13462 (opens in a new tab)**Full Changelog: https://github.com/spiceai/spiceai/compare/v2.2.0...v2.2.1 (opens in a new tab)