---

title: 'Snapshots' sidebar_label: 'Snapshots' description: 'Bootstrap file-mode accelerations from managed snapshots to eliminate cold starts.' sidebar_position: 3

:::note[Enterprise edition] Acceleration Snapshots are available in the Spice Enterprise edition (opens in a new tab). :::

Spicepod Example

Overview

Acceleration snapshots let Spice reuse a pre-built acceleration file on startup instead of waiting for a full refresh. When a dataset uses a file-mode acceleration engine (DuckDB, SQLite, Cayenne, or Turso) and the local file is missing (for example on first boot or when using ephemeral NVMe storage), Spice downloads the most recent snapshot from object storage and moves the dataset straight to a ready state.

How it works

• On startup, Spice checks whether the file supplied in acceleration.params (for example duckdb_file) exists.
• If the file is missing and snapshots are enabled, Spice looks under the configured snapshot location and downloads the newest snapshot for that dataset.
• If no snapshot is available, the acceleration boots empty and refreshes from the source.
• Spice creates new snapshots based on the configured snapshots_trigger mode.

Snapshots are organized with Hive-style partitioning so they are easy to retain and prune. For a dataset named my_dataset accelerated with DuckDB, Spice writes files such as:

The timestamp is recorded in UTC using ISO 8601 without punctuation. The file extension names the acceleration engine that wrote the snapshot: .duckdb, .sqlite, .cayenne, or .turso. The engine is also recorded in the snapshot metadata, and a snapshot whose engine differs from the dataset's current engine is rejected at bootstrap rather than restored.

:::warning Dedicated files only Every accelerated dataset must write to its own file (for example, /nvme/my_dataset.db). Sharing a single file across multiple datasets is not supported. :::

What each snapshot contains

Each snapshot is a complete copy of that dataset's acceleration file at the time Spice writes it. The object under the snapshot location is the whole accelerated dataset — DuckDB, SQLite, Cayenne, or Turso — ready for a reader to download and open. Spice writes that full file on every snapshot. It does not publish an incremental delta of the rows or objects that changed since the previous snapshot.

A bootstrap, and a refresh_mode: snapshot (opens in a new tab) reload, replaces the local acceleration file with that copy. Every upload moves the full file, and bucket or object replication of the snapshot prefix moves the full file again. Storage grows with the size of the accelerated dataset times how often snapshots are written, until a lifecycle rule expires older objects. DuckDB snapshots_compaction (opens in a new tab) can shrink each copy. The uploaded object is still a full file.

When the workload needs incremental replication of changed data between regions or storage tiers, use a path that publishes the changed objects:

• Cayenne cold / datalake tier. cayenne_datalake_location (opens in a new tab) graduates data onto standard object storage (a general-purpose S3 or S3-compatible bucket). Promotion carries unchanged cold files forward and rewrites the cold files a change can touch. The tier requires refresh_mode: changes or append and a primary key. It is a data tier, separate from snapshots.location.
• Iceberg batched writes with merge-on-read. Iceberg writes (opens in a new tab) append new data files, and deletes (opens in a new tab) are equality delete files that readers merge on scan. A catalog commit publishes those objects. Use this path when the dataset lives in Iceberg and readers apply delete files during scan.

Neither path bootstraps a file-mode accelerator. Keep snapshots for that.

Configure snapshot storage

Snapshots are controlled with a top-level snapshots block in the Spicepod. The location can point to S3, Azure ADLS Gen2, Google Cloud Storage, or the local filesystem.

Supported storage backends

Backend	URL scheme	Environment variables
Amazon S3	s3://	Standard AWS credentials (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, etc.)
Azure ADLS Gen2	abfss://, abfs://	AZURE_STORAGE_ACCOUNT_NAME, AZURE_STORAGE_ACCOUNT_KEY, AZURE_CLIENT_ID/AZURE_TENANT_ID/AZURE_FEDERATED_TOKEN_FILE
Google Cloud Storage	gs://	GOOGLE_APPLICATION_CREDENTIALS, Workload Identity
Local filesystem	file://	N/A

location must be a URI with a scheme — a bare filesystem path such as /nvme/snapshots is not a valid URI, so it fails to parse and snapshots are disabled with an error logged. Use file:///nvme/snapshots/ for a local folder.

When the location is an S3 bucket, the configuration accepts any S3 dataset parameters (opens in a new tab) under params. Azure and GCS locations also accept their respective connector parameters under params for explicit credential overrides. When no explicit credentials are supplied, Spice reads standard environment variables for each cloud provider.

Snapshot location and the Cayenne data tier

Keep location on standard object storage: Amazon S3 (s3://), Google Cloud Storage (gs://), or Azure ADLS Gen2 (abfss://, abfs://). A local file:// folder is valid on a single machine. Standard cloud storage is what bucket and object replication, and readers in another region, use.

location is independent of Cayenne's S3 Express One Zone data tier. cayenne_file_path and the cayenne_s3_* parameters (cayenne_s3_region, cayenne_s3_zone_ids, cayenne_s3_auth, and the related keys) store Cayenne Vortex files on an Express One Zone directory bucket. That bucket is single-zone storage for the accelerator's data files. It does not substitute for the snapshot bucket, and it does not provide the replication or multi-region reads a standard snapshot location does. See S3 Express One Zone storage (opens in a new tab).

The Cayenne cold tier (opens in a new tab) uses a general-purpose S3 or S3-compatible bucket, set by cayenne_datalake_location, not GCS or ADLS. That prefix is the cold data tier, separate from snapshots.location.

Failure behavior

bootstrap_on_failure_behavior controls what Spice does when it cannot load the most recent snapshot.

• retry – keep retrying the newest snapshot until it succeeds.
• fallback – try older snapshot files until one loads successfully.
• warn – log a warning and continue with an empty acceleration. (Default.)

Enable snapshots per dataset

Each dataset opts into snapshotting through the acceleration.snapshots field. Four modes are available:

• enabled – download snapshots on startup and write a new snapshot after each refresh.
• bootstrap_only – only download snapshots; never write new ones.
• create_only – write new snapshots after refreshes, but never download them on startup.
• disabled – disable snapshot usage for this dataset. (Default.)

Complete configuration:

Snapshot triggers

The snapshots_trigger setting controls when Spice creates new snapshots. The available triggers depend on the dataset's refresh mode.

Batch-based datasets

Datasets using refresh_mode: full, or refresh_mode: append with a time_column, support the following triggers:

Trigger	Description
refresh_complete	Create a snapshot after each data refresh completes. (Default.)
time_interval	Create snapshots at a fixed time interval.

Example with default trigger:

Example with time-based trigger:

Caching datasets

Datasets using refresh_mode: caching support a single trigger:

Trigger	Description
time_interval	Create snapshots at a fixed time interval. (Default, and the only supported trigger.)

The interval is set with snapshots_trigger_threshold and defaults to 10m. Setting snapshots_trigger to refresh_complete or stream_batches on a caching dataset is a configuration error.

Stream-based datasets

Datasets using refresh_mode: changes, or refresh_mode: append without a time_column, support the following triggers:

Trigger	Description
time_interval	Create snapshots at a fixed time interval. (Default: 10m.)
stream_batches	Create a snapshot after a specified number of batches are processed.

Example with time-based trigger (default):

Example with batch-based trigger:

Snapshot compaction

For DuckDB-based accelerations, enable snapshots_compaction to compact the database before uploading. This uses DuckDB's internal mechanism (COPY DATABASE) to reduce file size and improve read performance.

:::info Compaction is only available for the DuckDB acceleration engine. :::

Snapshot Resetting Expiry on Load

When using Caching refresh mode with DuckDB-based acceleration, you can enable snapshots_reset_expiry_on_load to extend the data's expiry to now() + TTL each time a snapshot is loaded.

Complete example

:::info Readiness with append refreshes Append-mode accelerations that define a time_column wait to report ready until the first append refresh completes after snapshot bootstrap. This keeps the dataset out of rotation until the freshest data is available while still benefiting from the snapshot-assisted startup. See Fast Cold Starts (opens in a new tab) for additional context. :::

Best practices

• Budget storage for a full file on every write. Each snapshot is a complete copy of the accelerated dataset. See What each snapshot contains (opens in a new tab) and Snapshot location and the Cayenne data tier (opens in a new tab).
• Pair with ephemeral storage: Deployments commonly place the acceleration file on fast ephemeral disks (such as NVMe instance storage) while relying on snapshots for persistence across restarts. Local NVMe is the recommended medium for accelerations — see Storage (opens in a new tab) for the tiers, the instance-store lifetime, and the capacity figures.
• Enable compaction for large datasets: Use snapshots_compaction: enabled for DuckDB accelerations to reduce snapshot size and improve bootstrap performance.
• Tune trigger thresholds for stream datasets: For high-throughput streaming datasets, balance snapshot frequency against I/O overhead by adjusting snapshots_trigger_threshold.
• Align retention policies: Apply an object storage lifecycle rule that mirrors the desired snapshot retention policy.
• Monitor bootstraps: Track warning logs emitted when Spice falls back to an empty acceleration so operators can respond quickly if snapshot loading fails.

For the full reference, see snapshots in the Spicepod specification (opens in a new tab) and acceleration.snapshots (opens in a new tab).

For the production deployment pattern that uses snapshots to separate ingest from read workloads, see Read/Write Separation (opens in a new tab).

:::warning[Limitations]

• Only datasets are supported for snapshots. Views are not supported.
• Partitioned Cayenne accelerations (engine: cayenne with partition_by) skip periodic and pre-recreate snapshots, with a warning naming the dataset.

:::

Cookbook

• A cookbook recipe to configure snapshots for file-mode accelerations so datasets avoid cold starts and recover quickly after restarts. Accelerated Snapshots (opens in a new tab)