Learn how to configure and run Spice in distributed mode to handle larger scale queries across multiple nodes.
:::info Preview
Multi-node distributed query execution based on Apache Ballista is available as a preview feature in Spice v1.9.0.
:::
Spice integrates Apache Ballista to schedule and coordinate distributed queries across multiple executor nodes. This integration enables distributed execution when running large queries over partitioned data lake formats such as Parquet, Delta Lake, or Iceberg.
A distributed Spice cluster consists of two components:
The scheduler holds the cluster-wide configuration for a Spicepod, while executors connect to the scheduler to receive work.
Cluster deployment typically starts with a scheduler instance, followed by one or more executors that register with the scheduler.
The scheduler is the only spiced process that needs to be configured (i.e. have a spicepod.yaml in the current dir). Override the Flight bind address when it must be reachable outside of localhost:
Executors need the scheduler's Flight URI to register and pull work. The executors do not require a spicepod.yaml to be present, it will fetch the configuration from the coordinator. Each executor automatically selects a free port if the default is unavailable:
Queries run against the scheduler endpoint. The EXPLAIN output confirms that distributed planning is active—Spice includes a distributed_plan section showing how stages are split across executors:
:::warning[Limitations]
:::
