date: 2025-11-04 title: 'Spice v1.9.0-rc.1 (Nov 4, 2025)' type: blog authors: [peasee] tags: [release, cayenne, acceleration, datafusion, dynamodb, duckdb, full-text-search, vector-search, performance]
This is the first release candidate for v1.9.0, which introduces Cayenne, a new high-performance data accelerator built on the Vortex columnar format that delivers DuckDB-comparable performance without scaling limitations. This release also upgrades to DataFusion v50 for improved query performance, expands search capabilities with full-text search on views and multi-column embeddings, includes significant DynamoDB and DuckDB accelerator improvements, and delivers security and reliability enhancements.
Introducing Cayenne: SQL as an Acceleration Format: A new high-performance data accelerator that simplifies multi-file data acceleration by using an embedded database (SQLite) for metadata while storing data in the Vortex columnar format. Cayenne delivers query and ingestion performance comparable or better to DuckDB's file-based acceleration without DuckDB's memory overhead and the scaling challenges of single DuckDB files.
Cayenne uses SQLite to manage acceleration metadata (schemas, snapshots, statistics, file tracking) through simple SQL transactions, while storing actual data in Vortex's compressed columnar format. This architecture provides:
Key Features:
Example Spicepod.yml configuration:
Note, the Cayenne Data Accelerator is in Alpha with limitations.
For more details, refer to the Cayenne Documentation, the Vortex project, and the DuckLake announcement that partly inspired this design.
Spice.ai is built on the DataFusion query engine. The v50 release brings significant performance improvements and enhanced reliability:
Performance Improvements 🚀:
ExecutionPlans, ensuring filters propagate correctly through all physical operators for improved query performance.Bug Fixes & Reliability: Resolved issues with partition name validation and empty execution plans when vector index lists are empty. Fixed timestamp support for partition expressions, enabling better partitioning for time-series data.
See the Apache DataFusion 50.0.0 Release for more details.
Improved Query Performance: The DynamoDB Data Connector now includes improved filter handling for edge cases, parallel scan support for faster data ingestion, and better error handling for misconfigured queries. These improvements enable more reliable and performant access to DynamoDB data.
Example Spicepod.yml configuration:
Full-Text Search on Views: Full-text search indexes are now supported on views, enabling advanced search scenarios over pre-aggregated or transformed data. This extends the power of Spice's search capabilities beyond base datasets.
Multi-Column Embeddings on Views: Views now support embedding columns, enabling vector search and semantic retrieval on view data. This is useful for search over aggregated or joined datasets.
Vector Engines on Views: Vector search engines are now available for views, enabling similarity search over complex queries and transformations.
Example Spicepod.yml configuration:
Parquet Buffering for Partitioned Writes: DuckDB partitioned writes in table mode now support Parquet buffering, reducing memory usage and improving write performance for large datasets.
Retention SQL on Refresh Commit: DuckDB accelerations now support running retention SQL on refresh commit, enabling automatic data cleanup and lifecycle management during refresh operations.
UTC Timezone for DuckDB: DuckDB now uses UTC as the default timezone, ensuring consistent behavior for time-based queries across different environments.
Example Spicepod.yml configuration:
Optimized Prepared Statements: Prepared statement handling has been optimized for better performance with parameterized queries, reducing planning overhead and improving execution time for repeated queries.
Large RecordBatch Chunking: Large Arrow RecordBatch objects are now automatically chunked to control memory usage during query execution, preventing memory exhaustion for queries returning large result sets.
Enhanced HTTP Client Security: HTTP client usage across the runtime has been hardened with improved TLS validation, certificate pinning for critical endpoints, and better error handling for network failures.
ODBC Connector Improvements: Removed unwrap calls from the ODBC connector, improving error handling and reliability. Fixed secret handling and Kubernetes secret integration.
CLI Permissions Hardening: Tightened file permissions for the CLI and install script, ensuring secure defaults for configuration files and credentials.
Oracle Instant Client Pinning: Oracle Instant Client downloads are now pinned to specific SHAs, ensuring reproducible builds and preventing supply chain attacks.
DataFusion Log Emission: The Spice runtime now emits DataFusion internal logs, providing deeper visibility into query planning and execution for debugging and performance analysis.
AI Completions Tracing: Fixed tracing so that ai_completions operations are correctly parented under sql_query traces, improving observability for AI-powered queries.
Version-Controlled Data Access: The new Git Data Connector (Alpha) enables querying datasets stored in Git repositories. This connector is ideal for use cases involving configuration files, documentation, or any data tracked in version control.
Example Spicepod.yml configuration:
For more details, refer to the Git Data Connector Documentation.
spicepod.yaml files, providing actionable feedback for misconfiguration.ListingTable partitions are pruned correctly when filters are not used.regexp_match function for DuckDB datasets.No breaking changes.
No major cookbook updates.
The Spice Cookbook includes 81 recipes to help you get started with Spice quickly and easily.
To upgrade to v1.9.0-rc.1, use one of the following methods:
CLI:
Homebrew:
Docker:
Pull the spiceai/spiceai:1.9.0-rc.1 image:
For available tags, see DockerHub.
Helm:
AWS Marketplace:
🎉 Spice is now available in the AWS Marketplace!
runtime-secrets crate by @phillipleblanc in #7474runtime-parameters crate by @phillipleblanc in #7475