date: 2025-12-03 title: 'Spice v1.10.0-rc.1 (Dec 2, 2025)' type: blog authors: [mach-kernel] tags: [release, caching, acceleration, dynamodb, cayenne, data-connector, distributed-query, performance]

Announcing the release of Spice v1.10.0-rc.1! ⚡

v1.10.0-rc1 is a release candidate for early testing of v1.10 features including an all new caching acceleration mode, tiny_lfu caching policy, a new DynamoDB Streams connector (Preview), improvements to the DynamoDB connector, faster distributed query execution, S3 connector improvements, and security hardening for v1.10.0-stable.

What's New in v1.10.0-rc1

Caching Acceleration Mode with SWR and TinyLFU

This release introduces a new caching acceleration mode (opens in a new tab) that implements the stale-while-revalidate (SWR) pattern using Data Accelerators such as DuckDB or Cayenne, enabling queries to return file-persisted cached results immediately while asynchronously refreshing data in the background. Combined with the new TinyLFU cache eviction policy (opens in a new tab), Spice can now maintain higher cache hit rates while keeping memory usage predictable.

Key Features:

• Stale-While-Revalidate (SWR): Returns cached data immediately while refreshing in the background
• Data Accelerator Support: Cached accelerators can persist data to disk using DuckDB, SQLite, or Cayenne file modes.
• TinyLFU Cache Policy: Probabilistic cache admission policy that maintains high hit rates with minimal overhead
• Predictable Memory Usage: Configurable memory limits with automatic eviction of less frequently used entries

Example Spicepod.yml configuration:

runtime:
  caching:
    sql_results:
      enabled: true
      eviction_policy: tiny_lfu # default lru

datasets:
  - from: s3://my-bucket/data.parquet
    name: cached_data
    acceleration:
      enabled: true
      engine: duckdb
      mode: file # Persist cache to disk
      refresh_mode: caching
      refresh_check_interval: 10m

runtime:
  caching:
    sql_results:
      enabled: true
      eviction_policy: tiny_lfu # default lru

datasets:
  - from: s3://my-bucket/data.parquet
    name: cached_data
    acceleration:
      enabled: true
      engine: duckdb
      mode: file # Persist cache to disk
      refresh_mode: caching
      refresh_check_interval: 10m

For more details, refer to the Data Acceleration Documentation (opens in a new tab) and Caching Documentation (opens in a new tab).

DynamoDB Streams Data Connector in Preview

DynamoDB Connector now integrates with DynamoDB Streams which enables real-time streaming with support for both table bootstrapping and continuous change data capture (CDC). This connector automatically detects changes in DynamoDB tables and streams them into Spice for real-time query, search, and LLM-inference.

Key Features:

• Real-Time CDC: Automatically captures inserts, updates, and deletes from DynamoDB tables
• Table Bootstrapping: Initial full table load before streaming changes

Example Spicepod.yml configuration:

datasets:
  - from: dynamodb:my_table
    name: orders_stream
    acceleration:
      enabled: true
      refresh_mode: changes

datasets:
  - from: dynamodb:my_table
    name: orders_stream
    acceleration:
      enabled: true
      refresh_mode: changes

For more details, refer to the DynamoDB Connector Documentation (opens in a new tab).

Cayenne Accelerator Enhancements

The Cayenne data accelerator (opens in a new tab) now supports:

• Sort Columns Configuration: Optimize inserts by pre-sorting data on specified columns for improved query performance

Example Spicepod.yml configuration:

datasets:
  - from: s3://my-bucket/data.parquet
    name: sorted_data
    acceleration:
      enabled: true
      engine: cayenne
      mode: file_create
      params:
        sort_columns: timestamp,region

datasets:
  - from: s3://my-bucket/data.parquet
    name: sorted_data
    acceleration:
      enabled: true
      engine: cayenne
      mode: file_create
      params:
        sort_columns: timestamp,region

For more details, refer to the Cayenne Documentation (opens in a new tab).

S3 Connector Improvements

S3 Location Predicate Pruning: The S3 data connector (opens in a new tab) now supports location-based predicate pruning, dramatically reducing data scanned by pushing down predicates to S3 listing operations. This optimization is especially effective for partitioned datasets stored in S3.

AWS S3 Tables Write Support: Full read/write capability for AWS S3 Tables (opens in a new tab), enabling fast integration with AWS's table format for S3.

For more details, refer to the S3 Tables Data Connector Documentation (opens in a new tab) and Glue Data Connection Documentation (opens in a new tab).

Faster Distributed Query Execution

Distributed query (opens in a new tab) planning and execution have been significantly improved:

• Fixed executor registration in cluster mode for more reliable distributed deployments
• Improved hostname resolution for Flight server binding, enabling better executor discovery
• Distributed accelerator registration: Data accelerators now properly register in distributed mode
• Optimized query planning: DistributeFileScanOptimizer improvements for faster planning with large datasets

For more details, refer to the Distributed Query Documentation (opens in a new tab).

Search Improvements

Search (opens in a new tab) capabilities have been improved with several performance and reliability enhancements:

• Fixed FTS query blocking: Full-text search queries no longer block unnecessarily, improving query responsiveness
• Optimized vector index operations: Eliminated unnecessary list_vectors calls for better performance
• Improved limit pushdown: IndexerExec now properly handles limit pushdown for more efficient searches

For more details, refer to the Search Documentation (opens in a new tab).

Security Hardening

Multiple security improvements have been implemented:

• SQL identifier quoting: Hardened SQL identifier quoting across all connectors to prevent injection attacks
• Token redaction: Sensitive tokens are now fully redacted in debug output to prevent credential leakage
• Path traversal prevention: Fixed tar extraction to prevent path traversal vulnerabilities
• Input sanitization: Added validation for top_n_sample order_by parsing
• Improved credential handling: Improved credential management in Glue connector

Developer Experience Improvements

• Health probe metrics: Added health probe latency metrics for better observability
• CLI improvements: Fixed .clear history command in the REPL to fully clear persisted history

Contributors

• @lukekim (opens in a new tab)
• @sgrebnov (opens in a new tab)
• @peasee (opens in a new tab)
• @krinart (opens in a new tab)
• @Jeadie (opens in a new tab)
• @mach-kernel (opens in a new tab)
• @phillipleblanc (opens in a new tab)
• @ewgenius (opens in a new tab)

Breaking Changes

No breaking changes.

Cookbook Updates

No major cookbook updates. The Spice Cookbook (opens in a new tab) still offers 82+ recipes to help you prototype quickly.

Upgrading

To try v1.10.0-rc1, use one of the following methods:

CLI:

spice upgrade --version 1.10.0-rc1

spice upgrade --version 1.10.0-rc1

Homebrew:

brew upgrade spiceai/spiceai/spice

brew upgrade spiceai/spiceai/spice

Docker:

Pull the spiceai/spiceai:1.10.0-rc1 image:

docker pull spiceai/spiceai:1.10.0-rc1

docker pull spiceai/spiceai:1.10.0-rc1

For available tags, see DockerHub (opens in a new tab).

Helm:

helm repo update
helm upgrade spiceai spiceai/spiceai --version 1.10.0-rc1

helm repo update
helm upgrade spiceai spiceai/spiceai --version 1.10.0-rc1

AWS Marketplace:

🎉 Spice is available in the AWS Marketplace (opens in a new tab).

What's Changed

Changelog

• Test-operator: Add tpcds_q8 to the default row-count validation skip list by @sgrebnov (opens in a new tab) in #8185 (opens in a new tab)
• fix: Remove unwrap_used from test by @peasee (opens in a new tab) in #8212 (opens in a new tab)
• Run glue_iceberg_integration_test_catalog as part of main integration tests by @sgrebnov (opens in a new tab) in #8222 (opens in a new tab)
• Add TPCH sf100 testoperator spicepods with dispatch by @Jeadie (opens in a new tab) in #8192 (opens in a new tab)
• Build with CPU native flags by @lukekim (opens in a new tab) in #8224 (opens in a new tab)
• Make copilot check for empty copyright header by @krinart (opens in a new tab) in #8245 (opens in a new tab)
• fix: Apply assertion clippy in CI/Makefile only by @peasee (opens in a new tab) in #8229 (opens in a new tab)
• feat: Support running queries only in testoperator by @peasee (opens in a new tab) in #8211 (opens in a new tab)
• DuckDB query planning: aggregate pushdown by @mach-kernel (opens in a new tab) in #8174 (opens in a new tab)
• install.sh improvements by @lukekim (opens in a new tab) in #8252 (opens in a new tab)
• Fix .clear history by @lukekim (opens in a new tab) in #8254 (opens in a new tab)
• fix: Pushdown dynamic filters to partition scans by @peasee (opens in a new tab) in #8240 (opens in a new tab)
• Harden SQL identifier quoting in connectors by @phillipleblanc (opens in a new tab) in #8276 (opens in a new tab)
• Cayenne sort_columns on insert by @lukekim (opens in a new tab) in #8091 (opens in a new tab)
• Redact token debug output by @phillipleblanc (opens in a new tab) in #8280 (opens in a new tab)
• fix: Cayenne configuration options by @lukekim (opens in a new tab) in #8281 (opens in a new tab)
• Prevent path traversal in untar by @phillipleblanc (opens in a new tab) in #8284 (opens in a new tab)
• Fix cluster mode executor registration by @mach-kernel (opens in a new tab) in #8292 (opens in a new tab)
• Unignore s3_vectors_kafka_stream test by @Jeadie (opens in a new tab) in #8289 (opens in a new tab)
• Post-release house keeping by @krinart (opens in a new tab) in #8293 (opens in a new tab)
• Improve generate_changelog script by @krinart (opens in a new tab) in #8273 (opens in a new tab)
• Acceleration mode caching by @lukekim (opens in a new tab) in #8237 (opens in a new tab)
• Sanitization and security checks by @lukekim (opens in a new tab) in #7854 (opens in a new tab)
• Add health probe latency metric by @phillipleblanc (opens in a new tab) in #8300 (opens in a new tab)
• Add distributed registration for data accelerators by @phillipleblanc (opens in a new tab) in #8299 (opens in a new tab)
• Pass IndexedTableProvider down in 'changes_stream' and 'append_stream' by @Jeadie (opens in a new tab) in #8295 (opens in a new tab)
• Add dynamodb-streams crate by @krinart (opens in a new tab) in #8283 (opens in a new tab)
• Distributed query: resolve executor hostname when determining Flight server binding by @mach-kernel (opens in a new tab) in #8304 (opens in a new tab)
• Return computed embeddings from index for partitioned S3Vectors by @Jeadie (opens in a new tab) in #8306 (opens in a new tab)
• [DDB Streams] Skeleton for DynamoDB Streams by @krinart (opens in a new tab) in #8296 (opens in a new tab)
• DistributeFileScanOptimizer: Improve planning performance by @mach-kernel (opens in a new tab) in #8305 (opens in a new tab)
• feat: Add an ExactLeftAccumulator implementation by @peasee (opens in a new tab) in #8302 (opens in a new tab)
• deps: Upgrade Vortex to 0.56 by @peasee (opens in a new tab) in #8311 (opens in a new tab)
• DynamoDB table bootstrapping + streaming by @krinart (opens in a new tab) in #8312 (opens in a new tab)
• Avoid calling S3Vector list_vectors (or equivalent) when indexing into VectorIndexs by @Jeadie (opens in a new tab) in #8282 (opens in a new tab)
• Add on_conflict testing support to append benchmark by @sgrebnov (opens in a new tab) in #8314 (opens in a new tab)
• docker: Add valid home directory to fix duckdb extension loading issue by @phillipleblanc (opens in a new tab) in #8318 (opens in a new tab)
• Add GH Workflow to run Append benchmark test by @sgrebnov (opens in a new tab) in #8321 (opens in a new tab)
• Exclude MySQL SF100 from test-operator dispatch by @sgrebnov (opens in a new tab) in #8320 (opens in a new tab)
• feat: Update clippy lints by @peasee (opens in a new tab) in #8317 (opens in a new tab)
• Add S3 location predicate pruning to listing connector by @phillipleblanc (opens in a new tab)
• Review feedback for caching mode accelerator by @phillipleblanc (opens in a new tab) in #8326 (opens in a new tab)
• Also include Dockerfile home changes for release build by @phillipleblanc (opens in a new tab) in #8327 (opens in a new tab)
• Change communication channel from Discord to Slack by @Jeadie (opens in a new tab) in #8330 (opens in a new tab)
• Replace Discord link with Slack link in README by @Jeadie (opens in a new tab) in #8331 (opens in a new tab)
• fix(glue): Prevent OpenDAL from automatic loading of AWS credentials from environment by @sgrebnov (opens in a new tab) in #8337 (opens in a new tab)
• Block on index read for FTS queries by @Jeadie (opens in a new tab) in #8339 (opens in a new tab)
• Fix search query provider by @Jeadie (opens in a new tab) in #8343 (opens in a new tab)
• Support for writing into AWS S3 Tables by @sgrebnov (opens in a new tab) in #8344 (opens in a new tab)
• Acceleration file_create mode by @lukekim (opens in a new tab) in #8347 (opens in a new tab)
• Don't block on lock in FTS query path by @Jeadie (opens in a new tab) in #8348 (opens in a new tab)
• feat: Add an optimizer rule to replace join accumulator for Cayenne by @peasee (opens in a new tab) in #8316 (opens in a new tab)
• S3 Vectors limit updates by @lukekim (opens in a new tab) in #8352 (opens in a new tab)
• Sanitize top_n_sample order_by parsing by @phillipleblanc (opens in a new tab) in #8356 (opens in a new tab)
• Update version to v1.10.0-rc.1 by @ewgenius (opens in a new tab) in #8362 (opens in a new tab)
• Improve IndexerExec to properly handle limit pushdown by @sgrebnov (opens in a new tab) in #8366 (opens in a new tab)
• Fix Cayenne partition_by metadata flaky integration test by @phillipleblanc (opens in a new tab) in #8367 (opens in a new tab)
• Rework caching accelerator to use the stale-while-revalidate pattern. by @phillipleblanc (opens in a new tab) in #8365 (opens in a new tab)
• Add TinyLFU caching policy by @lukekim (opens in a new tab) in #8370 (opens in a new tab)