This hands-on guide is designed to help developers quickly build an understanding of Spice: what it is (an AI-native query engine that federates queries, accelerates data, and integrates search and AI), when to use it (data-intensive applications and AI agents), and how it can be leveraged to solve enterprise-scale data challenges.  

*Note: This guide was last updated on February 5, 2026. Please see the docs for the latest updates.  

This guide is for developers who want to understand why, how, and when to use Spice.ai.  

If you are new to Spice, you might also be wondering how Spice is different to other query engines or data and AI platforms. Most developers exploring Spice are generally doing one of the following: 

Let’s start with the problem Spice is solving to anchor the discussion.  

Modern applications face a distributed data challenge. 

Enterprise data is spread across operational databases, data lakes, warehouses, third-party APIs, and more. Each source has its own interface, latency characteristics, and access patterns. 

AI workloads amplify the problem. RAG applications generally require: 

This can be a lot of complexity, even for a simple application. 

Spice is an open-source SQL query, search, and LLM-inference engine written in Rust, purpose-built for data-driven applications and AI agents. At its core, Spice is a high-performance compute engine that federates, searches, and processes data across your existing infrastructure - querying & accelerating data where it lives and integrating search and AI capabilities through SQL. 

Unlike databases that require migrations & maintenance, Spice takes a declarative configuration approach: datasets, views, models, tools are defined in declarative YAML, and Spice handles the operations of fetching, caching, and serving that data.  

This makes Spice ideal when: 

What Spice is not: 

Think of Spice as the operational data & AI layer between your applications and your data infrastructure. 

How this guide works 

We'll start with a hands-on quickstart to get Spice running, then progressively build your mental model through the core concepts: 

By the end, you'll understand how these primitives are used together to solve enterprise-scale data challenges.

To install and get Spice started, run: 

Or using Homebrew: 

Next, in any folder, create a spicepod.yaml file with the following content: 

In the same folder, run:

And, finally, in another terminal, run:

In that quickstart, you: 

You can run the same Spicepod configuration in Spice.ai Cloud, the fully managed version of Spice that extends the open-source runtime with enterprise capabilities: built-in observability, elastic scaling, and team collaboration. 

In the quickstart, you queried taxi_trips stored in a remote S3 bucket using standard SQL without copying or moving that data. That's federation in action - querying data where it lives, not where you've moved it to. 

This is foundational to Spice's architecture. Federation in Spice enables you to query data across multiple heterogeneous sources using a single SQL interface, without moving data or building ETL pipelines. 

Traditional approaches force you to build ETL pipelines that extract data from these sources, transform it, and load it into a centralized database or warehouse. Every new data source means building and maintaining another pipeline. 

Spice connects directly to your existing data sources and provides a unified SQL interface across all of them. You configure datasets declaratively in YAML, and Spice handles the connection, query translation, and result aggregation.  

Spice supports query federation across: 

When you configure multiple datasets from different sources, Spice's query planner (built on Apache DataFusion) optimizes and routes queries appropriately: 

Without additional configuration, each query fetches data directly from the underlying sources. Spice optimizes this as much as possible using filter pushdown and column projection.  

📚 Docs: Spice Federation and Data Connectors 

Federation solves the data movement problem, but alone often isn't enough for production applications. Querying remote S3 buckets for every request introduces latency - even with query pushdown and optimization, round-trips to distributed data sources can take seconds (or tens of seconds) for large datasets. 

Spice data acceleration materializes working sets of data locally, reducing query latency from seconds to milliseconds. When enabled, Spice syncs data from connected sources and stores it in local stores, like DuckDB or Vortex - giving you the speed of local data with the flexibility of federated access. 

You can think of acceleration as an intelligent caching layer that understands your data access patterns. Hot data gets materialized locally for instant access and cold data remains federated. Unlike traditional caches that just store query results or static database materializations, Spice accelerates entire datasets with configurable refresh strategies, with the flexible compute of an embedded database.  

To enable acceleration, add the acceleration block to your dataset configuration: 

With this configuration, Spice fetches the events dataset from S3 and stores it in a local Spice Cayenne Vortex files. Queries to events are then served from the local disk instead of making remote calls to S3.  

While DuckDB and SQLite are general purpose engines, Spice Cayenne is purpose-built for modern data lake workloads. It's built on Vortex - a next-generation columnar format under the Linux Foundation - designed for the scale and access patterns of object storage. 

Learn more: Introducing the Spice Cayenne Data Accelerator  

📚 Docs: Data Accelerators 

Spice offers multiple strategies for keeping accelerated data synchronized with sources: 

📚 Docs: Refresh Modes 

While refresh modes control how acceleration is populated, retention policies prevent unbounded growth. As data continuously flows into an accelerated dataset—especially in append or streaming modes—storage can grow indefinitely. Retention policies automatically evict stale data using time-based or custom SQL strategies. 
 
Retention is particularly useful for time-series workloads like logs, metrics, and event streams where only recent data is relevant for queries. For example, an application monitoring dashboard might only need the last 7 days of logs for troubleshooting, while a real-time analytics pipeline processing IoT sensor data might retain just 24 hours of readings. By defining retention policies, you ensure accelerated datasets stay bounded and performant without manual intervention. 
 
Spice supports two retention strategies: time-based, which removes records older than a specified period, and custom SQL-based, which executes arbitrary DELETE statements for more complex eviction logic. Once defined, Spice runs retention checks automatically at the configured interval: 

📚 Docs: Retention 

Accelerated datasets support primary key constraints and indexes for optimized query performance and data integrity: 

📚 Docs: Constraints & Indexes 

Views are virtual tables defined by SQL queries - useful for pre-aggregations, transformations, and simplified access patterns: 

📚 Docs: Views 

Spice provides in-memory caching for SQL query results, search results, and embeddings - all enabled by default. Caching eliminates redundant computation for repeated queries and improves performance for non-accelerated datasets. 

Spice also supports HTTP cache-control headers (no-cache, max-stale, only-if-cached) for fine-grained control over caching behavior per request. 

📚 Docs: Results Caching 

Snapshots allow file-based acceleration engines (DuckDB, SQLite, or Cayenne) to bootstrap from pre-stored snapshots in object storage. This dramatically reduces cold-start latency in distributed deployments. 

Snapshot triggers vary by refresh mode: 

📚 Docs: Snapshots 

AI is a first-class capability in the Spice runtime - not a bolt-on integration. Instead of wiring external APIs, you call LLMs directly from SQL queries using the `ai()` function. Embeddings generate automatically during data ingestion, eliminating separate pipeline infrastructure. Text-to-SQL is schema-aware with direct data access, preventing the hallucinations common in external tools that don’t understand your table structure.  

This SQL-first approach means you can query your federated and accelerated data, pipe results to an LLM for analysis, and get synthesized answers in a single SQL statement.  

You can connect to hosted providers (OpenAI, Anthropic, Bedrock) or serve models locally with GPU acceleration. Spice provides an OpenAI-compatible AI Gateway, so existing applications using OpenAI SDKs can swap endpoints without code changes. 

Connect to hosted models or serve locally: 

Use via the OpenAI-compatible API or the spice chat CLI: 

The /v1/nsql endpoint converts natural language to SQL and executes it: 

Spice uses tools like table_schema, random_sample, and sample_distinct_columns to help models write accurate, contextual SQL. 

Transform text into vectors for similarity search. These embeddings power the vector search capabilities covered in the 'search' section coming up next: 

Configure columns for automatic embedding generation: 

📚 Docs: Models & Embeddings 

In the previous section, we configured embeddings to generate automatically during data ingestion. Those embeddings enable vector search - one of three search methods Spice provides as native SQL functions. 

Spice takes the same integrated approach with search as it does with AI. Search indexes are built on top of accelerated datasets - the same data you're querying and piping to LLMs. Full-text search uses Tantivy with BM25 scoring for keyword matching. Vector search uses the embeddings you've already configured to generate during ingestion. Hybrid search combines both methods with Reciprocal Rank Fusion (RRF) to merge rankings - all via SQL functions like`text_search()`, `vector_search()`, and `rrf()`. Search in Spice powers retrieval-augmented generation (RAG), recommendation systems, and content discovery: 

Full-text search performs keyword-driven retrieval optimized for text data. Powered by Tantivy with BM25 scoring, it excels at finding exact phrases, specific terms, and keyword combinations. Enable it by indexing the columns you want to search: 

Vector search uses embeddings to find documents based on semantic similarity rather than exact keyword matches. This is particularly useful when users search with different wording than the source content—a query for "how to fix login issues" can match documents about "authentication troubleshooting." 
 
Spice supports both local embedding models (like sentence-transformers from Hugging Face) and remote providers (OpenAI, Anthropic, etc.). Embeddings are configured as top-level components and referenced in dataset columns:

Vector search is also available via the `/v1/search` HTTP API for direct integration with applications. 

Neither vector nor full-text search alone produces optimal results for every query. A search for "Python error 403" benefits from both semantic understanding ("error" relates to "exception," "failure") and exact keyword matching ("403," "Python"). Hybrid search combines results from multiple search methods using Reciprocal Rank Fusion (RRF), merging rankings to improve relevance across diverse content types: 

📚 Docs: Search & Vector Search 

Spice supports writing to Apache Iceberg tables and Amazon S3 Tables via standard INSERT INTO statements. 

Spice offers full read/write capability for Amazon S3 Tables, enabling direct integration with AWS' managed table format for S3: 

Note: Write support requires access: read_write configuration.

📚 Docs: Write-Capable Connectors 

Spice is designed for deployment flexibility and optionality – from edge devices to multi-node distributed clusters. It ships as a single file ~140MB binary with no external dependencies beyond your configured data sources.  

This portability means you can deploy the same Spicepod configuration on a Raspberry Pi at the edge, as a sidecar in your Kubernetes cluster, or as a fully-managed cloud service - without code changes: 

Spice makes data fast, federated, and AI-ready - through configuration, not code. The flexibility of this architecture means you can start simple and evolve incrementally.  

Whether you're replacing complex ETL pipelines, building AI-powered applications, or deploying intelligent agents at the edge—Spice provides the primitives to deliver fast, context-aware access to data wherever it lives. 

📚 Docs: Use Cases 

Now that you have a mental model for Spice, check out the cookbook recipes for 80+ examples, the GitHub repo, the full docs, and join us on Slack to connect directly with the team and other Spice users.

And, remember these principles: