spiceai/docs

Help Login

trunk

Edit on GitHub

Fork

/docs/website/blog/releases/v1.7.0.md

spiceai/docs | Spice Cloud Platform

trunk

Edit on GitHub

Fork

/docs/website/blog/releases/v1.7.0.md

spiceai/docs/README.md

date: 2025-09-23 title: 'Spice v1.7.0 (Sep 23, 2025)'

type: blog authors: [sgrebnov] tags: [release, datafusion, openai, search, embeddings, embeddinggemma]

Announcing the release of Spice v1.7.0! ⚡

Spice v1.7.0 upgrades to DataFusion v49 for improved performance and query optimization, introduces real-time full-text search indexing for CDC streams, EmbeddingGemma support for high-quality embeddings, new search table functions powering the /v1/search API, embedding request caching for faster and cost-efficient search and indexing, and OpenAI Responses API tool calls with streaming. This release also includes numerous bug fixes across CDC streams, vector search, the Kafka Data Connector, and error reporting.

What's New in v1.7.0

DataFusion v49 Highlights

DataFusion Clickbench Performance Graph Source: DataFusion 49.0.0 Release Blog.

Performance Improvements 🚀

Equivalence System Upgrade: Faster planning for queries with many columns, enabling more sophisticated sort-based optimizations.
Dynamic Filters & TopK Pushdown: Queries with ORDER BY and LIMIT now use dynamic filters and physical filter pushdown, skipping unnecessary data reads for much faster top-k queries.
Compressed Spill Files: Intermediate files written during sort/group spill to disk are now compressed, reducing disk usage and improving performance.
WITHIN GROUP for Ordered-Set Aggregates: Support for ordered-set aggregate functions (e.g., percentile_disc) with WITHIN GROUP.
REGEXP_INSTR Function: Find regex match positions in strings.

Spice Runtime Highlights

EmbeddingGemma Support: Spice now supports EmbeddingGemma, Google's state-of-the-art embedding model for text and documents. EmbeddingGemma provides high-quality, efficient embeddings for semantic search, retrieval, and recommendation tasks. You can use EmbeddingGemma via HuggingFace in your Spicepod configuration:

Example spicepod.yml snippet:

Learn more about EmbeddingGemma in the official documentation.

POST /v1/search API Use Search Table Functions: The /v1/search API now uses the new text_search and vector_search Table Functions for improved performance.

Embedding Request Caching: The runtime now supports caching embedding requests, reducing latency and cost for repeated content and search requests.

Example spicepod.yml snippet:

See the Caching documentation for details.

Real-Time Indexing for Full Text Search: Full Text search indexing is now supported for connectors that enable real-time changes, such as Debezium CDC streams. Adding a full-text index on a column with refresh_mode: changes works as it does for full/append-mode refreshes, enabling instant search on new data.

Example spicepod.yml snippet:

OpenAI Responses API Tool Calls with Streaming: The OpenAI Responses API now supports tool calls with streaming, enabling advanced model interactions such as web_search and code_interpreter with real-time response streaming. This allows you to invoke OpenAI-hosted tools and receive results as they are generated.

Learn more in the OpenAI Model Provider documentation.

Runtime Output Level Configuration: You can now set the output_level parameter in the Spicepod runtime configuration to control logging verbosity in addition to the existing CLI and environment variable support. Supported values are info, verbose, and very_verbose. The value is applied in the following priority: CLI, environment variables, then YAML configuration.

Example spicepod.yml snippet:

For more details on configuring output level, see the Troubleshooting documentation.

Bug Fixes

Several bugs and issues have been resolved in this release, including:

CDC Streams: Fixed issues where refresh_mode: changes could prevent the Spice runtime from becoming Ready, and improved support for full-text indexing on CDC streams.
Vector Search: Fixed bugs where vector search HTTP pipeline could not find more than one IndexedTableProvider, and resolved errors with field mismatches in vector_search UDTF.
Kafka Integration: Improved Kafka schema inference with configurable sample size, improved consumer group persistence for SQLite and Postgres accelerations, and added cooperative mode support.
Perplexity Web Search: Fixed bug where Perplexity web search sometimes used incorrect query schema (limit).
Databricks: Fixed issue with unparsing embedded columns.
Error Reporting: ThrottlingException is now reported correctly instead of as InternalError.
Iceberg Data Connector: Added support for LIMIT pushdown.
Amazon S3 Vectors: Fixed ingestion issues with zero-vectors and improved handling when vector index is full.
Tracing: Fixed vector search tracing to correctly report SQL status.

Contributors

New Contributors

@ChrisTomAlxHitachi made their first contribution in github.com/spiceai/spiceai/pull/6932 🎉

Breaking Changes

No breaking changes.

Cookbook Updates

New Spice with Dotnet SDK Recipe - The recipe shows how to query Spice using the Dotnet SDK.

The Spice Cookbook includes 78 recipes to help you get started with Spice quickly and easily.

Upgrading

To upgrade to v1.7.0, use one of the following methods:

CLI:

Homebrew:

Docker:

Pull the spiceai/spiceai:1.7.0 image:

For available tags, see DockerHub.

Helm:

AWS Marketplace:

🎉 Spice is now available in the AWS Marketplace!

What's Changed

Dependencies

Rust: Upgraded from 1.88.0 to 1.89.0
DataFusion: Upgraded from 48.0.1 to 49.0.0
text-embeddings-inference: Upgraded from 1.7.3 to 1.8.2
twox-hash: Upgraded from 1.6.3 to 2.1.0.

Changelog

Fix parameterised query planning in DataFusion by @Jeadie in #6942
fix: Update benchmark snapshots by @app/github-actions in #6944
refactor: Decouple full text search candidate from UDTF by @peasee in #6940
fix: Re-enable search integration tests by @peasee in #6930
Update acknowledgements and spicepod.schema.json by @sgrebnov in #6948
Add enabling the responses API by @lukekim in #6949
Post-release housekeeping by @sgrebnov in #6951
Add missing param in release notes by @Advayp in #6959
Create comprehensive S3vectors test by @Jeadie in #6903
Update ROADMAP after v1.6 release by in

spiceai/docs/README.md

date: 2025-09-23 title: 'Spice v1.7.0 (Sep 23, 2025)'

type: blog authors: [sgrebnov] tags: [release, datafusion, openai, search, embeddings, embeddinggemma]

Announcing the release of Spice v1.7.0! ⚡

What's New in v1.7.0

DataFusion v49 Highlights

DataFusion Clickbench Performance Graph Source: DataFusion 49.0.0 Release Blog.

Performance Improvements 🚀

Equivalence System Upgrade: Faster planning for queries with many columns, enabling more sophisticated sort-based optimizations.
Dynamic Filters & TopK Pushdown: Queries with ORDER BY and LIMIT now use dynamic filters and physical filter pushdown, skipping unnecessary data reads for much faster top-k queries.
Compressed Spill Files: Intermediate files written during sort/group spill to disk are now compressed, reducing disk usage and improving performance.
WITHIN GROUP for Ordered-Set Aggregates: Support for ordered-set aggregate functions (e.g., percentile_disc) with WITHIN GROUP.
REGEXP_INSTR Function: Find regex match positions in strings.

Spice Runtime Highlights

Example spicepod.yml snippet:

Learn more about EmbeddingGemma in the official documentation.

POST /v1/search API Use Search Table Functions: The /v1/search API now uses the new text_search and vector_search Table Functions for improved performance.

Embedding Request Caching: The runtime now supports caching embedding requests, reducing latency and cost for repeated content and search requests.

Example spicepod.yml snippet:

See the Caching documentation for details.

Example spicepod.yml snippet:

Learn more in the OpenAI Model Provider documentation.

Example spicepod.yml snippet:

For more details on configuring output level, see the Troubleshooting documentation.

Bug Fixes

Several bugs and issues have been resolved in this release, including:

CDC Streams: Fixed issues where refresh_mode: changes could prevent the Spice runtime from becoming Ready, and improved support for full-text indexing on CDC streams.
Vector Search: Fixed bugs where vector search HTTP pipeline could not find more than one IndexedTableProvider, and resolved errors with field mismatches in vector_search UDTF.
Kafka Integration: Improved Kafka schema inference with configurable sample size, improved consumer group persistence for SQLite and Postgres accelerations, and added cooperative mode support.
Perplexity Web Search: Fixed bug where Perplexity web search sometimes used incorrect query schema (limit).
Databricks: Fixed issue with unparsing embedded columns.
Error Reporting: ThrottlingException is now reported correctly instead of as InternalError.
Iceberg Data Connector: Added support for LIMIT pushdown.
Amazon S3 Vectors: Fixed ingestion issues with zero-vectors and improved handling when vector index is full.
Tracing: Fixed vector search tracing to correctly report SQL status.

Contributors

New Contributors

@ChrisTomAlxHitachi made their first contribution in github.com/spiceai/spiceai/pull/6932 🎉

Breaking Changes

No breaking changes.

Cookbook Updates

New Spice with Dotnet SDK Recipe - The recipe shows how to query Spice using the Dotnet SDK.

The Spice Cookbook includes 78 recipes to help you get started with Spice quickly and easily.

Upgrading

To upgrade to v1.7.0, use one of the following methods:

CLI:

Homebrew:

Docker:

Pull the spiceai/spiceai:1.7.0 image:

For available tags, see DockerHub.

Helm:

AWS Marketplace:

🎉 Spice is now available in the AWS Marketplace!

What's Changed

Dependencies

Rust: Upgraded from 1.88.0 to 1.89.0
DataFusion: Upgraded from 48.0.1 to 49.0.0
text-embeddings-inference: Upgraded from 1.7.3 to 1.8.2
twox-hash: Upgraded from 1.6.3 to 2.1.0.

Changelog

Fix parameterised query planning in DataFusion by @Jeadie in #6942
fix: Update benchmark snapshots by @app/github-actions in #6944
refactor: Decouple full text search candidate from UDTF by @peasee in #6940
fix: Re-enable search integration tests by @peasee in #6930
Update acknowledgements and spicepod.schema.json by @sgrebnov in #6948
Add enabling the responses API by @lukekim in #6949
Post-release housekeeping by @sgrebnov in #6951
Add missing param in release notes by @Advayp in #6959
Create comprehensive S3vectors test by @Jeadie in #6903
Update ROADMAP after v1.6 release by in

embeddings:
  - from: huggingface:huggingface.co/google/embeddinggemma-300m
    name: embeddinggemma
    params:
      hf_token: ${secrets:HUGGINGFACE_TOKEN}

embeddings:
  - from: huggingface:huggingface.co/google/embeddinggemma-300m
    name: embeddinggemma
    params:
      hf_token: ${secrets:HUGGINGFACE_TOKEN}

runtime:
  caching:
    embeddings:
      enabled: true
      max_size: 128mb
      item_ttl: 5s

runtime:
  caching:
    embeddings:
      enabled: true
      max_size: 128mb
      item_ttl: 5s

datasets:
  - from: debezium:cdc.public.question
    name: questions
    acceleration:
      enabled: true
      engine: duckdb
      primary_key: id
      refresh_mode: changes # Use 'changes'
    params: *kafka_params
    columns:
      - name: title
        full_text_search:
          enabled: true # Enable full-text-search indexing
          row_id:
            - id

datasets:
  - from: debezium:cdc.public.question
    name: questions
    acceleration:
      enabled: true
      engine: duckdb
      primary_key: id
      refresh_mode: changes # Use 'changes'
    params: *kafka_params
    columns:
      - name: title
        full_text_search:
          enabled: true # Enable full-text-search indexing
          row_id:
            - id

runtime:
  output_level: info # or verbose, very_verbose

runtime:
  output_level: info # or verbose, very_verbose

spice upgrade

spice upgrade

brew upgrade spiceai/spiceai/spice

brew upgrade spiceai/spiceai/spice

docker pull spiceai/spiceai:1.7.0

docker pull spiceai/spiceai:1.7.0

helm repo update
helm upgrade spiceai spiceai/spiceai

helm repo update
helm upgrade spiceai spiceai/spiceai