title: 'Spice.ai Catalog Connector'
sidebar_label: 'Spice.ai'
description: 'Connect to the Spice.ai built-in catalog.'
sidebar_position: 3
pagination_prev: null
pagination_next: null
Query datasets hosted in the Spice.ai Cloud Platform (opens in a new tab). Discover available public datasets on Spicerack (opens in a new tab).
Configuration
Create a Spice.ai Cloud Platform (opens in a new tab) account and login with the CLI using spice login.
Example:
catalogs:
- from: spice.ai:demo-org/tpch # Load tables from the `demo-org` organization's `tpch` app
name: marketplace # Tables will be available in the "marketplace" catalog
params:
spiceai_api_key: ${secrets:SPICEAI_API_KEY} # Spice.ai API key
spiceai_region: us-east-1 # Required: Spice Cloud region
include:
- "tpch.part*" # include only the tables from the "tpch" schema and that start with "part"
- "tpch.supplier" # also include the "supplier" table
catalogs:
- from: spice.ai:demo-org/tpch # Load tables from the `demo-org` organization's `tpch` app
name: marketplace # Tables will be available in the "marketplace" catalog
params:
spiceai_api_key: ${secrets:SPICEAI_API_KEY} # Spice.ai API key
spiceai_region: us-east-1 # Required: Spice Cloud region
include:
- "tpch.part*" # include only the tables from the "tpch" schema and that start with "part"
- "tpch.supplier" # also include the "supplier" table
This configuration would load the following tables:
sql> show tables;
+---------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------+----------------+------------+
| marketplace | tpch | part | BASE TABLE |
| marketplace | tpch | partsupp | BASE TABLE |
| marketplace | tpch | supplier | BASE TABLE |
+---------------+--------------+----------------+------------+
sql> show tables;
+---------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------+----------------+------------+
| marketplace | tpch | part | BASE TABLE |
| marketplace | tpch | partsupp | BASE TABLE |
| marketplace | tpch | supplier | BASE TABLE |
+---------------+--------------+----------------+------------+
from
The from field specifies which organization and application to load tables from. The format is:
spice.ai/<organization>/<application>[/<catalog_name>]
spice.ai/<organization>/<application>[/<catalog_name>]
organization: The Spice.ai organization that owns the application.application: The specific application to load tables from.catalog_name (optional): A specific catalog within the application. Defaults to spice if not specified.
For example:
- Load the default catalog from the "tpch" application in the "demo-org" organization
catalogs:
- from: spice.ai/demo-org/tpch
name: marketplace
catalogs:
- from: spice.ai/demo-org/tpch
name: marketplace
This will create tables like:
SHOW TABLES;
+---------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------+----------------+------------+
| marketplace | tpch | region | BASE TABLE |
| marketplace | tpch | orders | BASE TABLE |
| marketplace | tpch | part | BASE TABLE |
| marketplace | tpch | supplier | BASE TABLE |
| marketplace | tpch | customer | BASE TABLE |
| marketplace | tpch | partsupp | BASE TABLE |
| marketplace | tpch | lineitem | BASE TABLE |
| marketplace | tpch | nation | BASE TABLE |
+---------------+--------------+----------------+------------+
SHOW TABLES;
+---------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name | table_type |
+---------------+--------------+----------------+------------+
| marketplace | tpch | region | BASE TABLE |
| marketplace | tpch | orders | BASE TABLE |
| marketplace | tpch | part | BASE TABLE |
| marketplace | tpch | supplier | BASE TABLE |
| marketplace | tpch | customer | BASE TABLE |
| marketplace | tpch | partsupp | BASE TABLE |
| marketplace | tpch | lineitem | BASE TABLE |
| marketplace | tpch | nation | BASE TABLE |
+---------------+--------------+----------------+------------+
- Load a specific catalog named "custom_catalog" from an application
catalogs:
- from: spice.ai/demo-org/tpch/custom_catalog
name: marketplace
catalogs:
- from: spice.ai/demo-org/tpch/custom_catalog
name: marketplace
If the remote application has tables in custom_catalog like:
SHOW TABLES;
+------------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name | table_type |
+------------------+--------------+----------------+------------+
| custom_catalog | ice_schema1 | table1 | BASE TABLE |
| custom_catalog | ice_schema1 | table2 | BASE TABLE |
| custom_catalog | ice_schema2 | table1 | BASE TABLE |
+------------------+--------------+----------------+------------+
SHOW TABLES;
+------------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name | table_type |
+------------------+--------------+----------------+------------+
| custom_catalog | ice_schema1 | table1 | BASE TABLE |
| custom_catalog | ice_schema1 | table2 | BASE TABLE |
| custom_catalog | ice_schema2 | table1 | BASE TABLE |
+------------------+--------------+----------------+------------+
They will be available locally as:
SHOW TABLES;
+------------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name | table_type |
+------------------+--------------+----------------+------------+
| marketplace | ice_schema1 | table1 | BASE TABLE |
| marketplace | ice_schema1 | table2 | BASE TABLE |
| marketplace | ice_schema2 | table1 | BASE TABLE |
+------------------+--------------+----------------+------------+
SHOW TABLES;
+------------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name | table_type |
+------------------+--------------+----------------+------------+
| marketplace | ice_schema1 | table1 | BASE TABLE |
| marketplace | ice_schema1 | table2 | BASE TABLE |
| marketplace | ice_schema2 | table1 | BASE TABLE |
+------------------+--------------+----------------+------------+
name
The name field defines what catalog name the tables will be available under in the local Spice instance. For example, with the following configuration:
from: spice.ai/demo-org/tpch
name: marketplace
from: spice.ai/demo-org/tpch
name: marketplace
Then tables that exist in the remote application as:
spice # Default catalog
|- schema1
|- table1
|- table2
spice # Default catalog
|- schema1
|- table1
|- table2
Will be available locally as:
marketplace
|- schema1
|- table1
|- table2
marketplace
|- schema1
|- table1
|- table2
Queries are run against the marketplace catalog, like SELECT * FROM marketplace.schema1.table1.
include
Use the include field to specify which tables to include from the catalog. The include field supports glob patterns to match multiple tables:
schema_name.* - Include all tables from a specific schema*.table_name - Include all tables with a specific name from any schemaschema_name.table_name - Include a specific table from a schemaschema_name.table_prefix* - Include all tables in a schema that start with a prefix
Multiple include patterns can be specified and are OR'ed together. For example:
include:
- "tpch.part*" # Include all tables from the "tpch" schema that start with "part"
- "tpch.supplier" # Include the "supplier" table from the "tpch" schema
include:
- "tpch.part*" # Include all tables from the "tpch" schema that start with "part"
- "tpch.supplier" # Include the "supplier" table from the "tpch" schema
exclude
Optional. Use the exclude field to omit tables that would otherwise be included. It is matched against the same table name as include, using the same glob syntax, and multiple exclude patterns are OR'ed together. exclude takes precedence over include: a table is registered only when it matches include (or no include is set) and matches no exclude pattern.
params
Authentication
| Parameter Name | Description |
|---|
spiceai_api_key | API key from the Spice.ai Cloud Platform. Takes precedence over spiceai_token. |
spiceai_token | Legacy alias for spiceai_api_key. Used only if spiceai_api_key is not set. |
Region
| Parameter Name | Description |
|---|
spiceai_region | The Spice Cloud region to connect to (e.g. us-east-1). Required unless spiceai_http_endpoint is set. To list available regions, run spice cloud regions. |
Endpoint Overrides
These parameters override the default endpoints derived from spiceai_region. They are typically only needed for development or custom deployments.
| Parameter Name | Description |
|---|
spiceai_http_endpoint | Custom HTTP endpoint for the Spice Cloud catalog API (Iceberg REST catalog compatible). If set, spiceai_region is not required and region validation is skipped. |
spiceai_endpoint | Custom Arrow Flight endpoint for query execution. Takes precedence over spiceai_flight_endpoint. |
spiceai_flight_endpoint | Legacy alias for spiceai_endpoint. Used only if spiceai_endpoint is not set. |
Cookbook
