title: 'DuckLake Data Connector' sidebar_label: 'DuckLake Data Connector' description: 'DuckLake Data Connector Documentation' tags:
DuckLake is an open lakehouse format that stores metadata in a SQLite-compatible database (or PostgreSQL) and data in Parquet files. This connector enables querying individual DuckLake tables as datasets in Spice.
For automatic discovery of all schemas and tables in a DuckLake catalog, use the DuckLake Catalog Connector instead.
fromThe from field specifies the DuckLake table to connect to. Use ducklake:<table_path>, where table_path is the table name or a schema-qualified table name.
from | Description |
|---|---|
ducklake:my_table | Read from my_table in the default main schema |
ducklake:my_schema.my_table | Read from my_table in the my_schema schema |
nameThe dataset name. This will be used as the table name within Spice.
The dataset name cannot be a reserved keyword.
params| Parameter Name | Description |
|---|---|
ducklake_connection_string | Required. The DuckLake metadata location (e.g., s3://bucket/path/metadata.ducklake). |
ducklake_name | The name to attach the DuckLake catalog as in DuckDB. Default: ducklake. |
ducklake_open | Path to an existing DuckDB file for persistent storage. If not provided, an in-memory DuckDB instance is used. |
ducklake_aws_region | Optional. The AWS region for S3 storage. Default: us-east-1 when explicit credentials are provided. |
ducklake_aws_access_key_id | Optional. The AWS access key ID for S3 storage. Must be set together with ducklake_aws_secret_access_key. |
ducklake_aws_secret_access_key | Optional. The AWS secret access key for S3 storage. Must be set together with ducklake_aws_access_key_id. |
ducklake_aws_endpoint | Optional. Custom S3-compatible endpoint URL (e.g., for MinIO). |
ducklake_aws_allow_http |
| Backend | Example |
|---|---|
| Local file | /path/to/metadata.ducklake |
| AWS S3 | s3://bucket/path/metadata.ducklake |
| PostgreSQL | postgres:dbname=mydb host=localhost user=postgres password=secret |
When no explicit S3 credentials are configured, DuckDB falls back to its built-in credential chain provider:
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN)~/.aws/credentials)To provide explicit S3 credentials, use the ducklake_aws_* parameters:
For S3-compatible storage (e.g., MinIO), use ducklake_aws_endpoint:
This connector supports writing data to DuckLake tables using SQL INSERT INTO statements when access is set to read_write:
UPDATE and DELETE FROM are not supported. For DDL operations (CREATE TABLE, DROP TABLE), use the DuckLake Catalog Connector with access: read_write_create.
:::warning[Limitations]
ducklake_connection_string parameter is required — unlike the catalog connector, it cannot be omitted.INSERT INTO. UPDATE, DELETE FROM, and DDL (CREATE TABLE, DROP TABLE) are not supported on the data connector — use the DuckLake Catalog Connector for schema operations.:::
Optional. Set to true to allow HTTP (non-TLS) connections to S3. Default: false. |