Spice benchmarks
Explore how Spice performs on TPC-H, TPC-DS, ClickBench, and CH-benCHmark. Filter by scale factor and by single-engine, distributed, or Ballista execution. Every Spice figure comes from a published Spice.ai OSS run, with the hardware and runtime version it was measured on, and comparison figures for other engines name their source.
Headline results
88.3s
TPC-H SF-100 query set on one node, Spice 2.0 with Cayenne
26x
faster than Spice 1.x on TPC-DS SF-100 with Cayenne, same hardware
2.9x
faster on TPC-H SF-100 moving from one node to three Ballista executors
Explore the results
Pick a benchmark, a scale factor, and an execution mode. The panel shows the published result for that combination, the hardware and runtime version behind it, and a link to the post where it was first reported.
Combinations without a published run are marked as such rather than estimated. A figure measured at one scale factor is not carried to another, because the shape of a query plan changes with the size of the data.
TPC-H, SF-100, Single engine. Query-set duration, TPC-H SF-100.
TPC-H Decision-support analytics on data at rest. 22 queries over a star-schema sales dataset. Single engine: One Spice node with a local acceleration engine (Cayenne, DuckDB).
Query-set duration, TPC-H SF-100
One node, local acceleration. Spice 1.x and Spice 2.0 on identical hardware. Lower is better.
- Fastest configuration
- 88.3 s
- Spice 2.0 with Cayenne, 1.57x faster than DuckDB at 138.3 s
- Peak memory, DuckDB
- 40.7 GB
- Down from 70.7 GB on Spice 1.x, 42% less
- Steady-state memory
- 3.6x lower
- Cayenne median memory versus DuckDB on TPC-H
Hardware: Single node under a 256 GB memory limit.
Runtime: Spice.ai OSS 2.0 release-candidate builds versus the final 1.x release (v1.11.6).
Source: Spice 2.0 launch benchmarks
What the modes mean
Spice runs the same SQL engine in three configurations, and the difference between them is where the data sits and how many nodes touch it.
- Single engine. One Spice node with a local acceleration engine, either Spice Cayenne or DuckDB. The working set is materialized on the node, so queries never cross the network. This is the fastest configuration when the data fits.
- Distributed. One Spice node querying federated object storage directly, with no local copy. Compute stays on a single node and the data stays remote. This is the baseline that scale-out is measured against.
- Ballista. A multi-node cluster built on Apache Ballista, with a scheduler and separate executors reading the same federated source. Spice's distributed query engine is Ballista, so this is the multi-node form of the mode above.
Published results
The headline result for each published cell, with the supporting figures from the explorer below the table.
| Benchmark | Scale | Mode | Configuration | Result |
|---|---|---|---|---|
| TPC-H | SF-10 | Single engine | CDC freshness, Spicebench | 2.0 s |
| TPC-H | SF-10 | Single engine | Append-stream freshness | 7.0 s |
| TPC-H | SF-100 | Single engine | Spice 2.0, Cayenne | 88.3 s |
| TPC-H | SF-100 | Single engine | Spice 2.0, DuckDB | 138.3 s |
| TPC-H | SF-100 | Single engine | Spice 1.x, Cayenne | 133.6 s |
| TPC-H | SF-100 | Single engine | Spice 1.x, DuckDB | 253.0 s |
| TPC-H | SF-100 | Distributed | One node, federated S3 Parquet | 2,741 s |
| TPC-H | SF-100 | Ballista | Three Ballista executors | 935 s |
| TPC-DS | SF-100 | Single engine | Spice 2.0, Cayenne | 157.6 s |
| TPC-DS | SF-100 | Single engine | Spice 2.0, DuckDB | 93.9 s |
| TPC-DS | SF-100 | Single engine | Spice 1.x, Cayenne | 4,196 s |
| TPC-DS | SF-100 | Single engine | Spice 1.x, DuckDB | 108.6 s |
| ClickBench | Fixed | Single engine | Spice Cayenne, hot run | 33.4 s |
| ClickBench | Fixed | Single engine | DuckDB, hot run | 26.3 s |
| ClickBench | Fixed | Single engine | chDB, hot run | 33.8 s |
| ClickBench | Fixed | Single engine | DataFusion on Parquet, hot run | 42.3 s |
| CH-benCHmark | SF-1000 | Single engine | Analytical throughput | 1,046 QPH |
| CH-benCHmark | SF-1000 | Single engine | Transactional throughput | 266,861 tpmC |
| CH-benCHmark | SF-1000 | Single engine | Bootstrap ingest rate, Spice 2.0 native | ~566K rows/s |
| CH-benCHmark | SF-1000 | Single engine | Bootstrap ingest rate, Spice 1.0 with Debezium | ~3.3K rows/s |
Memory figures accompany several of these runs. On TPC-H SF-100, peak memory with DuckDB fell from 70.7 GB on Spice 1.x to 40.7 GB on Spice 2.0. Cayenne holds 3.6x lower steady-state memory than DuckDB on TPC-H and ClickBench, and 2.9x lower on TPC-DS. On ClickBench, Cayenne matches DuckDB query performance with 21% lower peak memory and a 34% smaller on-disk footprint. On TPC-H SF-100, the published Ballista comparison shows 5-10x lower memory than Apache Spark.
The explorer cells carry a few more figures. The 26x TPC-DS Cayenne result includes one pathological join query that took 65 minutes on Spice 1.x and completes in 5.3 seconds on Spice 2.0. On TPC-H SF-100, local Cayenne acceleration (88 s) is 31.1x faster than one node querying federated S3 Parquet. On CH-benCHmark SF-1000, Spice bootstrapped the 300M-row order_line table in roughly 9 minutes, while PostgreSQL ran 6.09M transactions in roughly 10 minutes.
How these benchmarks are run
Query-set durations cover the full suite, five iterations per query, single stream, with the SQL results cache on, which is the product default. ClickBench is the exception: it reports the hot run, meaning the fastest of three runs per query summed across all 43 queries, with the results cache disabled to match the public leaderboard method.
Version comparisons are like-for-like. Spice 1.x is rebuilt from its release tag and Spice 2.0 runs a release-candidate build from latest-green trunk, against an identical harness, dataset, runner, and spec. Where an engine cannot run a query, the query is excluded from both sides rather than one: q8, q38, and q87 are unsupported on DuckDB in both versions and are dropped from the DuckDB TPC-DS rows.
Coverage and gaps
Three scaled benchmarks at four scale factors, plus ClickBench's one fixed dataset, across three modes is 39 distinct configurations, and Spice has published runs for 7 of them. The explorer shows the whole matrix so the gaps are visible rather than hidden:
- SF-1 and SF-1000 for TPC-H and TPC-DS have no published result. SF-100 is the scale Spice reports these two benchmarks at.
- SF-10 for TPC-H is published as a freshness measurement from Spicebench, not as a query-set duration.
- ClickBench defines no scale factors. It ships one fixed dataset of roughly 100M rows, so the scale control does not apply to it.
- CH-benCHmark is published on a single node only. The benchmark needs one logical database, not one physical node, so distributed and Ballista runs are valid configurations. The published run replicates a PostgreSQL source into one Spice node, and Spice has not published a distributed or Ballista run, so those cells show as not published.
- Every Spice figure comes from a self-hosted Spice.ai OSS run, measured on the hardware named in that result's footer. The ClickBench chart's DuckDB, chDB, and DataFusion rows come from the public ClickBench leaderboard.
Workloads are derived from the TPC-H, TPC-DS, CH-benCHmark, and ClickBench specifications. Results are not audited TPC results.
FAQs
Common questions about how Spice measures and reports these benchmarks.
Spice 2.0 with the Cayenne accelerator, at 88.3 seconds for the full query set. The same runtime with DuckDB takes 138.3 seconds, so Cayenne is 1.57x faster. This is the fastest configuration in the Spice 2.0 launch benchmark matrix.
Distributed mode runs one Spice node against federated object storage, with no local copy of the data. Ballista mode adds executors: a scheduler distributes the same query across a multi-node cluster. Spice builds its multi-node compute on Apache Ballista, so Ballista is the scale-out form of distributed execution.
2.9x on TPC-H SF-100 over federated S3 Parquet. One node completes the query set in 2,741 seconds. Three Ballista executors complete it in 935 seconds against the same source.
This page reports measured runs only. TPC-H and TPC-DS are published at SF-100, so SF-1 and SF-1000 are empty for both. A result from one scale factor is not carried to another, because query plans change with the size of the data.
No. ClickBench ships one fixed dataset of roughly 100M rows and 43 queries. The scale control is disabled when ClickBench is selected.
CH-benCHmark runs TPC-C transactions and TPC-H-style analytics at the same time, against the same data. TPC-H measures analytics on data at rest. CH-benCHmark measures whether analytical queries stay fast and correct while transactions change the data underneath them.
No. The workloads derive from the TPC-H, TPC-DS, CH-benCHmark, and ClickBench specifications. The runs are not audited, so the figures are not official TPC results and must not be compared to published TPC benchmark submissions.
Run these workloads on your own data
Get a guided walkthrough of how development teams use Spice to query, accelerate, and integrate AI for mission-critical workloads.
Get a demo (opens in a new tab)