title: 'Cayenne Data Accelerator' sidebar_label: 'Cayenne Data Accelerator' description: 'Cayenne Data Accelerator (Vortex) Documentation' sidebar_position: 1 tags:
:::info Alpha The Cayenne Data Accelerator is in Alpha. Features and configuration may change. Available in Spice v1.9.0-rc.1 and later. :::
Cayenne is a Spice data acceleration engine designed for high-performance, scalable query on large-scale datasets. Built on Vortex, a next-generation columnar file format, Cayenne combines columnar storage with in-process metadata management to provide fast query performance to scale to datasets beyond 1TB.
Cayenne uses Vortex as its storage format, providing significant performance advantages:
Vortex is a Linux Foundation (LF AI & Data) project under Apache-2.0 license with neutral governance.
While DuckDB excels for datasets up to approximately 1TB, Spice Cayenne with Vortex is designed to scale beyond these limits.
For detailed Vortex performance benchmarks, visit bench.vortex.dev.
To use Cayenne as the data accelerator, specify cayenne as the engine for acceleration. Cayenne supports mode: file, mode: file_create, and mode: file_update and stores data on disk.
Cayenne uses Vortex's advanced columnar format, which provides:
Consider the following limitations when using Cayenne acceleration:
mode: file, mode: file_create, and mode: file_update and does not support in-memory (mode: memory) acceleration.on_conflict Support: Cayenne does not yet support the on_conflict configuration for handling duplicate keys during data refresh.retention_sql: Data deletion and cleanup operations require configuring retention_sql to define retention policies. Manual DELETE statements can also be executed directly.:::warning[Alpha Software]
As an Alpha feature, Cayenne should be thoroughly tested in development environments before production deployment. Monitor release notes for updates, breaking changes, and new capabilities.
:::
Resource requirements for Cayenne depend on dataset size, query patterns, and metastore configuration.
Cayenne manages memory efficiently through columnar storage and selective caching. Allocate sufficient memory based on:
Cayenne stores data in a columnar format optimized for analytical queries. Ensure adequate disk space for:
Query performance scales with available CPU cores. Vortex's columnar format supports parallel decompression and scanning across multiple threads. Allocate sufficient CPU for:
Consider the following limitations when using Spice Cayenne acceleration:
mode: file and does not support in-memory (mode: memory) acceleration.Interval, Duration, Map, and FixedSizeBinary types require unsupported_type_action configuration.indexes configuration. Vortex's segment statistics and fast random access encodings provide equivalent or better performance for most point lookup workloads.:::warning ALPHA SOFTWARE As an Alpha feature, Spice Cayenne should be thoroughly tested in development environments before production deployment. Monitor release notes for updates, breaking changes, and new capabilities. :::
Complete example configuration using Cayenne:
Spice Documentation:
External References: