title: 'ScyllaDB Data Connector' sidebar_label: 'ScyllaDB Data Connector' description: 'ScyllaDB Data Connector Documentation' tags:

• data-connectors
• scylladb

The ScyllaDB Data Connector enables federated SQL queries on data stored in ScyllaDB (opens in a new tab) clusters using CQL (Cassandra Query Language).

:::note[Enterprise edition] The ScyllaDB Data Connector is available in the Spice Enterprise edition (opens in a new tab). It is feature-gated and not included in the default open source build; open source users can build it from source with the scylladb feature (make install-scylladb). :::

datasets:
  - from: scylladb:users
    name: users
    params:
      scylladb_host: localhost
      scylladb_port: 9042
      scylladb_keyspace: my_app

datasets:
  - from: scylladb:users
    name: users
    params:
      scylladb_host: localhost
      scylladb_port: 9042
      scylladb_keyspace: my_app

Configuration

from

The from field takes the form scylladb:{table_name} where table_name is the table identifier in the ScyllaDB keyspace to read from.

datasets:
  - from: scylladb:users
    name: users
    params:
      scylladb_keyspace: my_app
      ...

datasets:
  - from: scylladb:users
    name: users
    params:
      scylladb_keyspace: my_app
      ...

name

The dataset name. This will be used as the table name within Spice.

Example:

datasets:
  - from: scylladb:users
    name: app_users
    params: ...

datasets:
  - from: scylladb:users
    name: app_users
    params: ...

SELECT COUNT(*) FROM app_users;

SELECT COUNT(*) FROM app_users;

+----------+
| count(*) |
+----------+
| 6001215  |
+----------+

+----------+
| count(*) |
+----------+
| 6001215  |
+----------+

The dataset name cannot be a reserved keyword (opens in a new tab).

params

The ScyllaDB data connector can be configured by providing the following params. Use the secret replacement syntax (opens in a new tab) to load the secret from a secret store, e.g. ${secrets:scylladb_pass}.

Types

The table below shows the CQL data types supported, along with the type mapping to Apache Arrow types in Spice.

Decimal Handling

CQL decimal is an arbitrary-precision type, while Arrow Decimal128 has a maximum precision of 38 digits. The connector uses:

• Precision: 38 (maximum for Decimal128)
• Scale: 2 (suitable for monetary/financial data)

If a CQL decimal value cannot be represented exactly at this precision and scale — because rescaling to scale 2 would drop non-zero fractional digits, or the mantissa does not fit in 128 bits — the connector returns a query error rather than silently truncating, rounding, or returning NULL. Values that rescale losslessly are converted normally.

Date/Time Handling

• CQL date: Stored as days since epoch with a 2^31 offset. Converted to Arrow Date32 (signed days since 1970-01-01).
• CQL timestamp: Milliseconds since Unix epoch. Directly mapped to Arrow Timestamp(Millisecond).
• CQL time: Nanoseconds since midnight. Converted to Arrow Timestamp(Microsecond) with nanosecond truncation.

Query Execution

The connector pushes down partition key and clustering key filters to CQL where possible. Filters that CQL cannot express are evaluated locally by DataFusion after data retrieval. Joins, aggregations, and other SQL operations are always performed locally.

Filter Pushdown

The following filters are pushed down to ScyllaDB:

Clustering key filters are marked as Inexact, meaning DataFusion re-checks them after retrieval to ensure correctness.

CQL vs SQL

CQL lacks many SQL constructs, which is why most filter types cannot be pushed down:

Projection Pushdown

Projection pushdown is supported — only the columns referenced in the query are fetched from ScyllaDB.

Streaming Execution

Query results are streamed using the scylla driver's paging mechanism in batches of 8192 rows, minimizing memory usage for large result sets.

Performance Considerations

See the dedicated Performance (opens in a new tab) page — partition/clustering-key filters, enabling acceleration, datacenter locality, and connection-timeout tuning.

Examples

Basic Federated Query

datasets:
  - from: scylladb:users
    name: users
    params:
      scylladb_host: localhost
      scylladb_port: 9042
      scylladb_keyspace: my_app

datasets:
  - from: scylladb:users
    name: users
    params:
      scylladb_host: localhost
      scylladb_port: 9042
      scylladb_keyspace: my_app

With Authentication

datasets:
  - from: scylladb:orders
    name: orders
    params:
      scylladb_host: scylla-cluster.example.com
      scylladb_keyspace: ecommerce
      scylladb_user: app_user
      scylladb_pass: ${secrets:SCYLLA_PASSWORD}

datasets:
  - from: scylladb:orders
    name: orders
    params:
      scylladb_host: scylla-cluster.example.com
      scylladb_keyspace: ecommerce
      scylladb_user: app_user
      scylladb_pass: ${secrets:SCYLLA_PASSWORD}

Multi-Node Cluster

datasets:
  - from: scylladb:events
    name: events
    params:
      scylladb_hosts: node1.scylla.local,node2.scylla.local,node3.scylla.local
      scylladb_keyspace: analytics
      scylladb_datacenter: us-west-2

datasets:
  - from: scylladb:events
    name: events
    params:
      scylladb_hosts: node1.scylla.local,node2.scylla.local,node3.scylla.local
      scylladb_keyspace: analytics
      scylladb_datacenter: us-west-2

With Acceleration

datasets:
  - from: scylladb:products
    name: products
    params:
      scylladb_host: ${env:SCYLLADB_HOST}
      scylladb_keyspace: catalog
    acceleration:
      enabled: true
      engine: duckdb
      refresh_check_interval: 1h

datasets:
  - from: scylladb:products
    name: products
    params:
      scylladb_host: ${env:SCYLLADB_HOST}
      scylladb_keyspace: catalog
    acceleration:
      enabled: true
      engine: duckdb
      refresh_check_interval: 1h

Using Environment Variables

datasets:
  - from: scylladb:customer
    name: customer
    params:
      scylladb_host: ${env:SCYLLADB_HOST}
      scylladb_port: ${env:SCYLLADB_PORT}
      scylladb_keyspace: ${env:SCYLLADB_KEYSPACE}

datasets:
  - from: scylladb:customer
    name: customer
    params:
      scylladb_host: ${env:SCYLLADB_HOST}
      scylladb_port: ${env:SCYLLADB_PORT}
      scylladb_keyspace: ${env:SCYLLADB_KEYSPACE}

Limitations

CQL Limitations

The following SQL operations cannot be pushed down to ScyllaDB and are performed locally:

• JOINs: All joins are performed locally by DataFusion
• Aggregations: COUNT, SUM, AVG, etc. are computed locally
• Subqueries: Nested queries are not supported in CQL
• Window functions: RANK, ROW_NUMBER, etc. not supported
• Complex WHERE clauses: Only partition key equality and clustering key comparisons are pushed down; other filters are evaluated locally
• ORDER BY: Sorting is done locally

Connector Limitations

• Read-only: The connector does not support INSERT, UPDATE, or DELETE operations
• Decimal precision: Fixed at precision=38, scale=2; values that cannot be rescaled to this precision and scale without loss error rather than being silently truncated
• Collection types: Lists, sets, and maps are converted to JSON string representation
• Large tables: Without acceleration, large tables cause significant data transfer

Data Type Limitations

• varint: Arbitrary-precision integers are converted to strings
• duration: CQL durations are converted to string representation
• UDTs: User-defined types are converted to string representation
• Nested collections: Deeply nested collections become complex JSON strings

Secrets

Spice integrates with multiple secret stores to help manage sensitive data securely. For detailed information on supported secret stores, refer to the secret stores documentation (opens in a new tab). Additionally, learn how to use referenced secrets in component parameters by visiting the using referenced secrets guide (opens in a new tab).

Cookbook

• A cookbook recipe to configure ScyllaDB as a data connector in Spice. ScyllaDB Data Connector (opens in a new tab)

See Also

• ScyllaDB Documentation (opens in a new tab)
• ScyllaDB CQL Reference (opens in a new tab)
• Data Acceleration (opens in a new tab)