:::note[Enterprise edition] Acceleration Snapshots are available in the Spice Enterprise edition. :::
Acceleration snapshots let Spice reuse a pre-built acceleration file on startup instead of waiting for a full refresh. When a dataset uses a file-mode acceleration engine (DuckDB, SQLite, Cayenne, or Turso) and the local file is missing (for example on first boot or when using ephemeral NVMe storage), Spice downloads the most recent snapshot from object storage and moves the dataset straight to a ready state.
acceleration.params (for example duckdb_file) exists.snapshots_trigger mode.Snapshots are organized with Hive-style partitioning so they are easy to retain and prune. For a dataset named my_dataset, Spice writes files such as:
The timestamp is recorded in UTC using ISO 8601 without punctuation.
:::warning Dedicated files only
Every accelerated dataset must write to its own file (for example, /nvme/my_dataset.db). Sharing a single file across multiple datasets is not supported.
:::
Snapshots are controlled with a top-level snapshots block in the Spicepod. The location can point to S3, Azure ADLS Gen2, Google Cloud Storage, or the local filesystem.
| Backend | URL scheme | Environment variables |
|---|---|---|
| Amazon S3 | s3:// | Standard AWS credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, etc.) |
| Azure ADLS Gen2 | abfss://, abfs:// | AZURE_STORAGE_ACCOUNT_NAME, AZURE_STORAGE_ACCOUNT_KEY, AZURE_CLIENT_ID/AZURE_TENANT_ID/AZURE_FEDERATED_TOKEN_FILE |
| Google Cloud Storage | gs:// | GOOGLE_APPLICATION_CREDENTIALS, Workload Identity |
| Local filesystem | Absolute or relative path | N/A |
When the location is an S3 bucket, the configuration accepts any S3 dataset parameters under params. Azure and GCS locations also accept their respective connector parameters under params for explicit credential overrides. When no explicit credentials are supplied, Spice reads standard environment variables for each cloud provider.
bootstrap_on_failure_behavior controls what Spice does when it cannot load the most recent snapshot.
retry – keep retrying the newest snapshot until it succeeds.fallback – try older snapshot files until one loads successfully.warn – log a warning and continue with an empty acceleration. (Default.)Each dataset opts into snapshotting through the acceleration.snapshots field. Four modes are available:
enabled – download snapshots on startup and write a new snapshot after each refresh.bootstrap_only – only download snapshots; never write new ones.create_only – write new snapshots after refreshes, but never download them on startup.disabled – disable snapshot usage for this dataset. (Default.)Complete configuration:
The snapshots_trigger setting controls when Spice creates new snapshots. The available triggers depend on the dataset's refresh mode.
Datasets using refresh_mode: full, refresh_mode: caching, or refresh_mode: append with a time_column support the following triggers:
| Trigger | Description |
|---|---|
refresh_complete | Create a snapshot after each data refresh completes. (Default.) |
time_interval | Create snapshots at a fixed time interval. |
Example with default trigger:
Example with time-based trigger:
Datasets using refresh_mode: changes, or refresh_mode: append without a time_column, support the following triggers:
| Trigger | Description |
|---|---|
time_interval | Create snapshots at a fixed time interval. (Default: 10m.) |
stream_batches | Create a snapshot after a specified number of batches are processed. |
Example with time-based trigger (default):
Example with batch-based trigger:
For DuckDB-based accelerations, enable snapshots_compaction to compact the database before uploading. This uses DuckDB's internal mechanism (COPY DATABASE) to reduce file size and improve read performance.
:::info Compaction is only available for the DuckDB acceleration engine. :::
When using Caching refresh mode with DuckDB-based acceleration, you can enable snapshots_reset_expiry_on_load to extend the data's expiry to now() + TTL each time a snapshot is loaded.
:::info Readiness with append refreshes
Append-mode accelerations that define a time_column wait to report ready until the first append refresh completes after snapshot bootstrap. This keeps the dataset out of rotation until the freshest data is available while still benefiting from the snapshot-assisted startup. See Fast Cold Starts for additional context.
:::
snapshots_compaction: enabled for DuckDB accelerations to reduce snapshot size and improve bootstrap performance.snapshots_trigger_threshold.For the full reference, see snapshots in the Spicepod specification and acceleration.snapshots.
For the production deployment pattern that uses snapshots to separate ingest from read workloads, see Read/Write Separation.
:::warning[Limitations]
:::