Query across sources
without ETL

Spice joins data from operational databases, warehouses, data lakes, and APIs in one SQL query. Connect a new source without building another pipeline.

Get a demo (opens in a new tab)

View the cookbook (opens in a new tab)

Diagram of apps, AI agents, and dashboards querying Spice federation, with an accelerator, over a data store of Databricks, PostgreSQL, Snowflake, and Amazon S3

Enterprise AI apps need data across many systems

With traditional data systems, every new app, feature, or dashboard requires another ETL pipeline to build and maintain. Latency depends on when the last run took place. Agents and latency-sensitive apps demand infrastructure that returns results faster.

Query every source through a single SQL endpoint

Spice queries each source where it lives, and materializes the hot-path data next to the app.

Federated SQL

Query and join data where it lives

Join data from databases, warehouses, data lakes, and APIs in one SQL query. Spice plans each query with Apache DataFusion, sending filters and other supported operations to each source, and then joins the results.

Read the query federation docs (opens in a new tab)

Query and join data where it lives: an agent sends one SQL query to Spice. Spice reads MySQL and Databricks over SQL, Parquet files in Amazon S3, and an HTTP API, then joins the rows into one answer, with no data copied ahead of timeAgentquery{ }{ }MySQLDatabaseDatabricksWarehouseAmazon S3Data lakeHTTP APIWeb serviceSQLSQLParquetHTTPOne SQL query reaches every kind of source.Spice reads each source through its own interface.Each answer becomes rows in one SQL result.One joined answer, with no data copied ahead of time.
Local acceleration

Fast answers on the hot path

Accelerate frequently queried datasets locally with Spice Cayenne. Queries read the local acceleration, so they do not wait on the source.

Read the data acceleration docs (opens in a new tab)

Infrastructure simplification

Replace separate engines, ETL jobs, and caches

Replace separate query engines, ETL jobs, and custom caches with one runtime. The same runtime handles federation, acceleration, and hybrid search.

Read the features docs (opens in a new tab)

CDC and refresh

Keep local accelerations current

Reload a small table on a schedule, append new rows to a log table, or stream each change from an orders table over CDC. Each local acceleration stays current with its source.

Read the data refresh docs (opens in a new tab)

40+ integrations

Connectors to databases, warehouses, data lakes, HTTP APIs, and more.

View integrations

Query across sources in minutes

Join Parquet in S3 with a PostgreSQL table in one federated query.

# spicepod.yamldatasets:  - from: s3://spiceai-demo-datasets/taxi_trips/2024/    name: taxi_trips    params: { file_format: parquet, s3_auth: public }  - from: postgres:taxi_zones    name: taxi_zones    params:      pg_host: localhost      pg_port: 15432      pg_db: taxi      pg_user: postgres      pg_pass: ${secrets:SPICE_PG_PASS}      pg_sslmode: disablesql> SELECT z.zone, COUNT(*) AS trips     FROM taxi_trips t                 -- S3     JOIN taxi_zones z                 -- PostgreSQL       ON t."PULocationID" = z.location_id     GROUP BY z.zone ORDER BY trips DESC LIMIT 3;JFK Airport                 145240Midtown Center              143471Upper East Side South       142708Time: 2.705 seconds

Examples and timings from Spice cookbooks: Federation (opens in a new tab), Acceleration (opens in a new tab)

  1. Connect your sources

    In a spicepod.yaml, the Spice configuration file, list each source table. Spice queries each source where it lives.

  2. Accelerate the tables you read most

    Set acceleration to enabled. Spice keeps a local acceleration and refreshes it, with no ETL job.

  3. Deploy and query

    Start Spice with spice run, or deploy it as a sidecar, at the edge, in your cluster, or on Spice Cloud. Then query from agents, dashboards, or your own apps.

  1. /plugin marketplace add spiceai/skills
  2. /plugin install spiceai@spiceai

Proven in production

Teams that build agents and apps on data from many systems run Spice as one SQL layer.

Ramachandra Ramarathinam

“Spice is the data plane behind Summation. Every connector we ship, from Snowflake to a generic REST-as-a-table, collapses into one SQL surface, which is what makes our AI agents portable across a customer's stack.”

Ramachandra Ramarathinam

CTO, Summation

Peter Janovsky

“Spice opened the door to take these critical control-plane datasets and move them next to our services in the runtime path.”

Peter Janovsky

Software Architect, Twilio

Darin Douglass

100x

Faster queries

“It just spins up and works, which is really nice. The responsiveness is amazing, which is a huge gain for the customer.”

Darin Douglass

Principal Software Engineer, Barracuda

Learn how Barracuda gained 100x faster query responses and reduced costs 50% with Spice

FAQs

Answers for teams that query data across many systems.

Query federation runs one SQL query across many data sources, as if they were one database. Spice sends filters and other supported operations to each source, and then joins the results. No data is copied ahead of time. For more, read SQL federation and acceleration.

For agents and apps, often not. A batch job copies data on a schedule, so answers are as old as the last run. Spice reads each source directly, or refreshes a local acceleration, so no separate pipeline runs.

Spice pushes filters down to each source, so less data crosses the network. For data that queries read often, Spice Cayenne keeps a local acceleration next to the app. Queries on it do not wait on the source.

Spice has more than 40 data connectors. They include databases such as PostgreSQL, MySQL, and SQL Server, warehouses such as Snowflake and Databricks, and object stores such as Amazon S3. Table formats include Apache Iceberg and Delta Lake. For the full list, read the data connector docs.

An ETL pipeline fits when you reshape data heavily before anyone queries it. It also fits a warehouse that keeps years of history, or a report that runs once a day. Spice fits queries that need current data from many systems. Many teams use both.