spiceai/docs

Help Login

trunk

Edit on GitHub

Fork

/docs/website/releases/v1.11.0-rc.1.md

spiceai/docs | Spice Cloud Platform

trunk

Edit on GitHub

Fork

/docs/website/releases/v1.11.0-rc.1.md

spiceai/docs/README.md

date: 2026-01-07 title: 'Spice v1.11.0-rc.1 (Jan 6, 2026)' type: blog authors: [ewgenius] tags: [release, distributed-query, cayenne, acceleration, data-connector, llm-tool]

Announcing the release of Spice v1.11.0-rc.1! :star:

v1.11.0-rc.1 is the first release candidate for early testing of v1.11 features including Distributed Query with mTLS for enterprise-grade secure cluster communication, new SMB and NFS Data Connectors for direct network-attached storage access, Prepared Statements for improved query performance and security, Cayenne Accelerator Enhancements with Key-based deletion vectors and Amazon S3 Express One Zone support, Google LLM Support for expanded AI inference capabilities, and Spice Java SDK v0.5.0 with parameterized query support.

What's New in v1.11.0-rc.1

Distributed Query with mTLS

Enterprise-Grade Secure Cluster Communication: Distributed query cluster mode now enables mutual TLS (mTLS) by default for secure communication between schedulers and executors. Internal cluster communication includes highly privileged RPC calls like fetching Spicepod configuration and expanding secrets. mTLS ensures only authenticated nodes can join the cluster and access sensitive data.

Key Features:

Mutual TLS Authentication: All executor-to-scheduler and executor-to-executor gRPC connections on the internal cluster port (50052) are secured with mTLS, securing communication, and preventing unauthorized nodes from joining the cluster
Certificate Management CLI: New developer spice cluster tls init and spice cluster tls add commands for generating CA certificates and node certificates with proper SANs (Subject Alternative Names)
Simplified CLI Arguments: Renamed cluster arguments for clarity (--role, --scheduler-address, --node-mtls-*) with --scheduler-address implying --role executor
Port Separation: Public services (Flight queries, HTTP API, Prometheus metrics) remain on ports 50051, 8090, and 9090 respectively, while internal cluster services (SchedulerGrpcServer, ClusterService) are isolated on port 50052 with mTLS enforced
Development Mode: Use --allow-insecure-connections flag to disable mTLS requirement for local development and testing

Quick Start:

For more details, refer to the Distributed Query Documentation.

SMB and NFS Data Connectors

Network-Attached Storage Connectors: New data connectors for SMB (Server Message Block) and NFS (Network File System) protocols enable direct federated queries against network-attached storage without requiring data movement to cloud object stores.

Key Features:

SMB Protocol Support: Connect to Windows file shares and Samba servers with authentication support
NFS Protocol Support: Connect to Unix/Linux NFS exports for direct data access
Federated Queries: Query Parquet, CSV, JSON, and other file formats directly from network storage with full SQL support
Acceleration Support: Accelerate data from SMB/NFS sources using DuckDB, Spice Cayenne, or other accelerators

Example spicepod.yaml configuration:

For more details, refer to the Data Connectors Documentation.

Prepared Statements

Improved Query Performance and Security: Spice now supports prepared statements, enabling parameterized queries that improve both performance through query plan caching and security by preventing SQL injection attacks.

Key Features:

Query Plan Caching: Prepared statements cache query plans, reducing planning overhead for repeated queries
SQL Injection Prevention: Parameters are safely bound, preventing SQL injection vulnerabilities
Arrow Flight SQL Support: Full prepared statement support via Arrow Flight SQL protocol

SDK Support:

SDK	Support	Min Version	Method
gospice (Go)	✅ Full	v8.0.0+	`SqlWithParams()` with typed constructors (`Int32Param`, `StringParam`, `TimestampParam`, etc.)
spice-rs (Rust)	✅ Full	v3.0.0+	`query_with_params()` with `RecordBatch` parameters
spice-dotnet (.NET)	❌ Not yet	-	Coming soon
spice-java (Java)	✅ Full	v0.5.0+	`queryWithParams()` with typed `Param` constructors (`Param.int64()`, `Param.string()`, etc.)
spice.js (JavaScript)	❌ Not yet	-	Coming soon

Example (Go):

Example (Java):

For more details, refer to the Parameterized Queries Documentation.

Spice Cayenne Accelerator Enhancements

The Spice Cayenne data accelerator has been improved with several key enhancements:

KeyBased Deletion Vectors: Improved deletion vector support using key-based lookups for more efficient data management and faster delete operations. KeyBased deletion vectors are more memory-efficient than positional vectors for sparse deletions.
S3 Express One Zone Support: Store Cayenne data files in S3 Express One Zone for single-digit millisecond latency, ideal for latency-sensitive query workloads that require persistence.

Example spicepod.yaml configuration:

For more details, refer to the Cayenne Documentation.

Google LLM Support

Expanded AI Provider Support: Spice now supports Google embedding and chat models via the Google AI provider, expanding the available LLM options for AI inference workloads alongside existing providers like OpenAI, Anthropic, and AWS Bedrock.

Key Features:

Google Chat Models: Access Google's Gemini models for chat completions
Google Embeddings: Generate embeddings using Google's text embedding models
Unified API: Use the same OpenAI-compatible API endpoints for all LLM providers

Example spicepod.yaml configuration:

For more details, refer to the Google LLM Documentation (see docs PR #1286).

Spice Java SDK v0.5.0

Parameterized Query Support for Java: The Spice Java SDK v0.5.0 introduces parameterized queries using ADBC (Arrow Database Connectivity), providing a safer and more efficient way to execute queries with dynamic parameters.

Key Features:

SQL Injection Prevention: Parameters are safely bound, preventing SQL injection vulnerabilities
Automatic Type Inference: Java types are automatically mapped to Arrow types (e.g., double → Float64, String → Utf8)
Explicit Type Control: Use the new Param class with typed factory methods (Param.int64(), Param.string(), Param.decimal128(), etc.) for precise control over Arrow types
Updated Dependencies: Apache Arrow Flight SQL upgraded to 18.3.0, plus new ADBC driver support

Example:

Maven:

For more details, refer to the Spice Java SDK Repository.

OpenTelemetry Improvements

Unified Telemetry Endpoint: OTel metrics ingestion has been consolidated to the Flight port (50051), simplifying deployment by removing the separate OTel port (50052). The push-based metrics exporter continues to support integration with OpenTelemetry collectors.

Note: This is a breaking change. Update your configurations if you were using the dedicated OTel port 50052. Internal cluster communication now uses port 50052 exclusively.

Developer Experience Improvements

Turso v0.3.2 Upgrade: Upgraded Turso accelerator for improved performance and reliability
Rust 1.91 Upgrade: Updated to Rust 1.91 for latest language features and performance improvements
Spice Cloud CLI: Added spice cloud CLI commands for cloud deployment management
Improved Spicepod Schema: Enhanced JSON schema generation for better IDE support and validation
Acceleration Snapshots: Added configurable snapshots_create_interval for periodic acceleration snapshots independent of refresh cycles
Tiered Caching with Localpod: The Localpod connector now supports caching refresh mode, enabling multi-layer acceleration where a persistent cache feeds a fast in-memory cache
GitHub Data Connector: Added workflows and workflow runs support for GitHub repositories
NDJSON/LDJSON Support: Added support for Newline Delimited JSON and Line Delimited JSON file formats

Additional Improvements & Bug Fixes

Reliability: Fixed DynamoDB IAM role authentication with new dynamodb_auth: iam_role parameter
Reliability: Fixed cluster executors to use scheduler's temp_directory parameter for shuffle files
Reliability: Initialize secrets before object stores in cluster executor mode
Reliability: Added page-level retry with backoff for transient GitHub GraphQL errors
Performance: Improved statistics for rewritten DistributeFileScanOptimizer plans
Developer Experience: Added max_message_size configuration for Flight service

Contributors

Breaking Changes

OTel Ingestion Port Change

OTel ingestion has been moved to the Flight port (50051), removing the separate OTel port 50052. Port 50052 is now used exclusively for internal cluster communication. Update your configurations if you were using the dedicated OTel port.

Distributed Query Cluster Mode Requires mTLS

Distributed query cluster mode now requires mTLS for secure communication between cluster nodes. This is a security enhancement to prevent unauthorized nodes from joining the cluster and accessing secrets.

Migration Steps:

Generate certificates using spice cluster tls init and spice cluster tls add
Update scheduler and executor startup commands with --node-mtls-* arguments
For development/testing, use --allow-insecure-connections to opt out of mTLS

Renamed CLI Arguments:

Old Name	New Name
`--cluster-mode`	`--role`
`--cluster-ca-certificate-file`	`--node-mtls-ca-certificate-file`
`--cluster-certificate-file`	`--node-mtls-certificate-file`
`--cluster-key-file`	`--node-mtls-key-file`
`--cluster-address`	`--node-bind-address`
`--cluster-advertise-address`	`--node-advertise-address`
`--cluster-scheduler-url`	`--scheduler-address`

Removed CLI Arguments:

--cluster-api-key: Replaced by mTLS authentication

Cookbook Updates

No major cookbook updates.

The Spice Cookbook includes 84 recipes to help you get started with Spice quickly and easily.

Upgrading

To try v1.11.0-rc.1, use one of the following methods:

CLI:

Homebrew:

Docker:

Pull the spiceai/spiceai:1.11.0-rc.1 image:

For available tags, see DockerHub.

Helm:

AWS Marketplace:

🎉 Spice is available in the AWS Marketplace!

What's Changed

Changelog

OTel exporter for push metrics by @lukekim in #8442
fix: Update benchmark snapshots by @app/github-actions in #8448
Add TPCH append tests to scheduled dispatch workflow by @sgrebnov in #8451
Add snapshot creation logging by @krinart in #8469
Fix PeriodicReader panic by @krinart in #8471
Benchmarks: increase readiness timeout for turso acceleration (TPC-H) by @sgrebnov in #8470
fix: Pin CUDA build actions to commits by @peasee in #8477
Add Criterion benchmarking to chunking crate. by @Jeadie in #8431
DuckDB agg pushdown: gate behind accelerator parameter by @mach-kernel in

spiceai/docs/README.md

date: 2026-01-07 title: 'Spice v1.11.0-rc.1 (Jan 6, 2026)' type: blog authors: [ewgenius] tags: [release, distributed-query, cayenne, acceleration, data-connector, llm-tool]

Announcing the release of Spice v1.11.0-rc.1! :star:

What's New in v1.11.0-rc.1

Distributed Query with mTLS

Key Features:

Mutual TLS Authentication: All executor-to-scheduler and executor-to-executor gRPC connections on the internal cluster port (50052) are secured with mTLS, securing communication, and preventing unauthorized nodes from joining the cluster
Certificate Management CLI: New developer spice cluster tls init and spice cluster tls add commands for generating CA certificates and node certificates with proper SANs (Subject Alternative Names)
Simplified CLI Arguments: Renamed cluster arguments for clarity (--role, --scheduler-address, --node-mtls-*) with --scheduler-address implying --role executor
Port Separation: Public services (Flight queries, HTTP API, Prometheus metrics) remain on ports 50051, 8090, and 9090 respectively, while internal cluster services (SchedulerGrpcServer, ClusterService) are isolated on port 50052 with mTLS enforced
Development Mode: Use --allow-insecure-connections flag to disable mTLS requirement for local development and testing

Quick Start:

For more details, refer to the Distributed Query Documentation.

SMB and NFS Data Connectors

Key Features:

SMB Protocol Support: Connect to Windows file shares and Samba servers with authentication support
NFS Protocol Support: Connect to Unix/Linux NFS exports for direct data access
Federated Queries: Query Parquet, CSV, JSON, and other file formats directly from network storage with full SQL support
Acceleration Support: Accelerate data from SMB/NFS sources using DuckDB, Spice Cayenne, or other accelerators

Example spicepod.yaml configuration:

For more details, refer to the Data Connectors Documentation.

Prepared Statements

Key Features:

Query Plan Caching: Prepared statements cache query plans, reducing planning overhead for repeated queries
SQL Injection Prevention: Parameters are safely bound, preventing SQL injection vulnerabilities
Arrow Flight SQL Support: Full prepared statement support via Arrow Flight SQL protocol

SDK Support:

SDK	Support	Min Version	Method
gospice (Go)	✅ Full	v8.0.0+	`SqlWithParams()` with typed constructors (`Int32Param`, `StringParam`, `TimestampParam`, etc.)
spice-rs (Rust)	✅ Full	v3.0.0+	`query_with_params()` with `RecordBatch` parameters
spice-dotnet (.NET)	❌ Not yet	-	Coming soon
spice-java (Java)	✅ Full	v0.5.0+	`queryWithParams()` with typed `Param` constructors (`Param.int64()`, `Param.string()`, etc.)
spice.js (JavaScript)	❌ Not yet	-	Coming soon

Example (Go):

Example (Java):

For more details, refer to the Parameterized Queries Documentation.

Spice Cayenne Accelerator Enhancements

The Spice Cayenne data accelerator has been improved with several key enhancements:

KeyBased Deletion Vectors: Improved deletion vector support using key-based lookups for more efficient data management and faster delete operations. KeyBased deletion vectors are more memory-efficient than positional vectors for sparse deletions.
S3 Express One Zone Support: Store Cayenne data files in S3 Express One Zone for single-digit millisecond latency, ideal for latency-sensitive query workloads that require persistence.

Example spicepod.yaml configuration:

For more details, refer to the Cayenne Documentation.

Google LLM Support

Key Features:

Google Chat Models: Access Google's Gemini models for chat completions
Google Embeddings: Generate embeddings using Google's text embedding models
Unified API: Use the same OpenAI-compatible API endpoints for all LLM providers

Example spicepod.yaml configuration:

For more details, refer to the Google LLM Documentation (see docs PR #1286).

Spice Java SDK v0.5.0

Key Features:

SQL Injection Prevention: Parameters are safely bound, preventing SQL injection vulnerabilities
Automatic Type Inference: Java types are automatically mapped to Arrow types (e.g., double → Float64, String → Utf8)
Explicit Type Control: Use the new Param class with typed factory methods (Param.int64(), Param.string(), Param.decimal128(), etc.) for precise control over Arrow types
Updated Dependencies: Apache Arrow Flight SQL upgraded to 18.3.0, plus new ADBC driver support

Example:

Maven:

For more details, refer to the Spice Java SDK Repository.

OpenTelemetry Improvements

Note: This is a breaking change. Update your configurations if you were using the dedicated OTel port 50052. Internal cluster communication now uses port 50052 exclusively.

Developer Experience Improvements

Turso v0.3.2 Upgrade: Upgraded Turso accelerator for improved performance and reliability
Rust 1.91 Upgrade: Updated to Rust 1.91 for latest language features and performance improvements
Spice Cloud CLI: Added spice cloud CLI commands for cloud deployment management
Improved Spicepod Schema: Enhanced JSON schema generation for better IDE support and validation
Acceleration Snapshots: Added configurable snapshots_create_interval for periodic acceleration snapshots independent of refresh cycles
Tiered Caching with Localpod: The Localpod connector now supports caching refresh mode, enabling multi-layer acceleration where a persistent cache feeds a fast in-memory cache
GitHub Data Connector: Added workflows and workflow runs support for GitHub repositories
NDJSON/LDJSON Support: Added support for Newline Delimited JSON and Line Delimited JSON file formats

Additional Improvements & Bug Fixes

Reliability: Fixed DynamoDB IAM role authentication with new dynamodb_auth: iam_role parameter
Reliability: Fixed cluster executors to use scheduler's temp_directory parameter for shuffle files
Reliability: Initialize secrets before object stores in cluster executor mode
Reliability: Added page-level retry with backoff for transient GitHub GraphQL errors
Performance: Improved statistics for rewritten DistributeFileScanOptimizer plans
Developer Experience: Added max_message_size configuration for Flight service

Contributors

Breaking Changes

OTel Ingestion Port Change

Distributed Query Cluster Mode Requires mTLS

Migration Steps:

Generate certificates using spice cluster tls init and spice cluster tls add
Update scheduler and executor startup commands with --node-mtls-* arguments
For development/testing, use --allow-insecure-connections to opt out of mTLS

Renamed CLI Arguments:

Old Name	New Name
`--cluster-mode`	`--role`
`--cluster-ca-certificate-file`	`--node-mtls-ca-certificate-file`
`--cluster-certificate-file`	`--node-mtls-certificate-file`
`--cluster-key-file`	`--node-mtls-key-file`
`--cluster-address`	`--node-bind-address`
`--cluster-advertise-address`	`--node-advertise-address`
`--cluster-scheduler-url`	`--scheduler-address`

Removed CLI Arguments:

--cluster-api-key: Replaced by mTLS authentication

Cookbook Updates

No major cookbook updates.

The Spice Cookbook includes 84 recipes to help you get started with Spice quickly and easily.

Upgrading

To try v1.11.0-rc.1, use one of the following methods:

CLI:

Homebrew:

Docker:

Pull the spiceai/spiceai:1.11.0-rc.1 image:

For available tags, see DockerHub.

Helm:

AWS Marketplace:

🎉 Spice is available in the AWS Marketplace!

What's Changed

Changelog

OTel exporter for push metrics by @lukekim in #8442
fix: Update benchmark snapshots by @app/github-actions in #8448
Add TPCH append tests to scheduled dispatch workflow by @sgrebnov in #8451
Add snapshot creation logging by @krinart in #8469
Fix PeriodicReader panic by @krinart in #8471
Benchmarks: increase readiness timeout for turso acceleration (TPC-H) by @sgrebnov in #8470
fix: Pin CUDA build actions to commits by @peasee in #8477
Add Criterion benchmarking to chunking crate. by @Jeadie in #8431
DuckDB agg pushdown: gate behind accelerator parameter by @mach-kernel in

# Generate certificates for development
spice cluster tls init
spice cluster tls add scheduler1
spice cluster tls add executor1

# Start scheduler
spiced --role scheduler \
  --node-mtls-ca-certificate-file ca.crt \
  --node-mtls-certificate-file scheduler1.crt \
  --node-mtls-key-file scheduler1.key

# Start executor
spiced --role executor \
  --scheduler-address https://scheduler1:50052 \
  --node-mtls-ca-certificate-file ca.crt \
  --node-mtls-certificate-file executor1.crt \
  --node-mtls-key-file executor1.key

# Generate certificates for development
spice cluster tls init
spice cluster tls add scheduler1
spice cluster tls add executor1

# Start scheduler
spiced --role scheduler \
  --node-mtls-ca-certificate-file ca.crt \
  --node-mtls-certificate-file scheduler1.crt \
  --node-mtls-key-file scheduler1.key

# Start executor
spiced --role executor \
  --scheduler-address https://scheduler1:50052 \
  --node-mtls-ca-certificate-file ca.crt \
  --node-mtls-certificate-file executor1.crt \
  --node-mtls-key-file executor1.key

datasets:
  # SMB share
  - from: smb://fileserver/share/data.parquet
    name: smb_data
    params:
      smb_username: ${secrets:SMB_USER}
      smb_password: ${secrets:SMB_PASS}

  # NFS export
  - from: nfs://nfsserver/export/data.parquet
    name: nfs_data

datasets:
  # SMB share
  - from: smb://fileserver/share/data.parquet
    name: smb_data
    params:
      smb_username: ${secrets:SMB_USER}
      smb_password: ${secrets:SMB_PASS}

  # NFS export
  - from: nfs://nfsserver/export/data.parquet
    name: nfs_data

import ai.spice.SpiceClient;
import ai.spice.Param;
import org.apache.arrow.adbc.core.ArrowReader;

try (SpiceClient client = new SpiceClient()) {
    // With automatic type inference
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM products WHERE price > $1 AND category = $2",
        10.0, "electronics");

    // With explicit typed parameters
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM products WHERE price > $1 AND category = $2",
        Param.float64(10.0),
        Param.string("electronics"));
}

import ai.spice.SpiceClient;
import ai.spice.Param;
import org.apache.arrow.adbc.core.ArrowReader;

try (SpiceClient client = new SpiceClient()) {
    // With automatic type inference
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM products WHERE price > $1 AND category = $2",
        10.0, "electronics");

    // With explicit typed parameters
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM products WHERE price > $1 AND category = $2",
        Param.float64(10.0),
        Param.string("electronics"));
}

datasets:
  - from: s3://my-bucket/data.parquet
    name: fast_data
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      params:
        # Use S3 Express One Zone for data files
        cayenne_s3express_bucket: my-express-bucket--usw2-az1--x-s3

datasets:
  - from: s3://my-bucket/data.parquet
    name: fast_data
    acceleration:
      enabled: true
      engine: cayenne
      mode: file
      params:
        # Use S3 Express One Zone for data files
        cayenne_s3express_bucket: my-express-bucket--usw2-az1--x-s3

models:
  - from: google:gemini-2.0-flash
    name: gemini
    params:
      google_api_key: ${secrets:GOOGLE_API_KEY}

embeddings:
  - from: google:text-embedding-004
    name: google_embeddings
    params:
      google_api_key: ${secrets:GOOGLE_API_KEY}

models:
  - from: google:gemini-2.0-flash
    name: gemini
    params:
      google_api_key: ${secrets:GOOGLE_API_KEY}

embeddings:
  - from: google:text-embedding-004
    name: google_embeddings
    params:
      google_api_key: ${secrets:GOOGLE_API_KEY}

import ai.spice.SpiceClient;
import ai.spice.Param;

try (SpiceClient client = new SpiceClient()) {
    // With automatic type inference
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM taxi_trips WHERE trip_distance > $1 LIMIT 10",
        5.0);

    // With explicit typed parameters for precise control
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM orders WHERE order_id = $1 AND amount >= $2",
        Param.int64(12345),
        Param.decimal128(new BigDecimal("99.99"), 10, 2));
}

import ai.spice.SpiceClient;
import ai.spice.Param;

try (SpiceClient client = new SpiceClient()) {
    // With automatic type inference
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM taxi_trips WHERE trip_distance > $1 LIMIT 10",
        5.0);

    // With explicit typed parameters for precise control
    ArrowReader reader = client.queryWithParams(
        "SELECT * FROM orders WHERE order_id = $1 AND amount >= $2",
        Param.int64(12345),
        Param.decimal128(new BigDecimal("99.99"), 10, 2));
}

spice upgrade --version 1.11.0-rc.1

spice upgrade --version 1.11.0-rc.1

brew upgrade spiceai/spiceai/spice

brew upgrade spiceai/spiceai/spice

docker pull spiceai/spiceai:1.11.0-rc.1

docker pull spiceai/spiceai:1.11.0-rc.1

helm repo update
helm upgrade spiceai spiceai/spiceai --version 1.11.0-rc.1

helm repo update
helm upgrade spiceai spiceai/spiceai --version 1.11.0-rc.1

import "github.com/spiceai/gospice/v8"

client, _ := spice.NewClient()
defer client.Close()

// Parameterized query with typed parameters
results, _ := client.SqlWithParams(ctx,
    "SELECT * FROM products WHERE price > $1 AND category = $2",
    spice.Float64Param(10.0),
    spice.StringParam("electronics"),
)

import "github.com/spiceai/gospice/v8"

client, _ := spice.NewClient()
defer client.Close()

// Parameterized query with typed parameters
results, _ := client.SqlWithParams(ctx,
    "SELECT * FROM products WHERE price > $1 AND category = $2",
    spice.Float64Param(10.0),
    spice.StringParam("electronics"),
)

<dependency>
  <groupId>ai.spice</groupId>
  <artifactId>spiceai</artifactId>
  <version>0.5.0</version>
</dependency>

<dependency>
  <groupId>ai.spice</groupId>
  <artifactId>spiceai</artifactId>
  <version>0.5.0</version>
</dependency>