spiceai/docs | Spice Cloud Platform

docs

spiceai/docs | Spice Cloud Platform

spiceai/docs/README.md

title: 'Spice.ai Catalog Connector' sidebar_label: 'Spice.ai' description: 'Connect to the Spice.ai built-in catalog.' sidebar_position: 3 pagination_prev: null pagination_next: null

Query datasets hosted in the Spice.ai Cloud Platform. Discover available public datasets on Spicerack.

Configuration

Create a Spice.ai Cloud Platform account and login with the CLI using spice login.

Example:

This configuration would load the following tables:

`from`

The from field specifies which organization and application to load tables from. The format is:

organization: The Spice.ai organization that owns the application
application: The specific application to load tables from
catalog_name: (optional): A specific catalog within the application. If not specified, the application's default catalog is used

For example:

Load the default catalog from the "tpch" application in the "demo-org" organization

This will create tables like:

Load a specific catalog named "custom_catalog" from an application

If the remote application has tables in custom_catalog like:

They will be available locally as:

`name`

The name field defines what catalog name the tables will be available under in the local Spice instance. For example, with the following configuration:

Then tables that exist in the remote application as:

Will be available locally as:

Queries are run against the marketplace catalog, like SELECT * FROM marketplace.schema1.table1.

`include`

Use the include field to specify which tables to include from the catalog. The include field supports glob patterns to match multiple tables:

schema_name.* - Include all tables from a specific schema
*.table_name - Include all tables with a specific name from any schema
schema_name.table_name - Include a specific table from a schema
schema_name.table_prefix* - Include all tables in a schema that start with a prefix

Multiple include patterns can be specified and are OR'ed together. For example:

`params`

The following parameters are supported for configuring the connection to the Spice Cloud catalog/tables:

Parameter Name	Definition
`spiceai_api_key`	Authorization API key from the Spice.ai Cloud Platform, used to login to the specified organization and app.

Cookbook

A cookbook recipe to configure Spice Cloud as a catalog connector in Spice. Spice Cloud Catalog Connector

catalogs:
  - from: spice.ai:demo-org/tpch # Load tables from the `demo-org` organization's `tpch` app
    name: marketplace # Tables will be available in the "marketplace" catalog
    params:
      spiceai_api_key: ${secrets:SPICEAI_API_KEY} # Spice.ai API key to login to the organization and app
    include:
      - "tpch.part*" # include only the tables from the "tpch" schema and that start with "part"
      - "tpch.supplier" # also include the "supplier" table

catalogs:
  - from: spice.ai:demo-org/tpch # Load tables from the `demo-org` organization's `tpch` app
    name: marketplace # Tables will be available in the "marketplace" catalog
    params:
      spiceai_api_key: ${secrets:SPICEAI_API_KEY} # Spice.ai API key to login to the organization and app
    include:
      - "tpch.part*" # include only the tables from the "tpch" schema and that start with "part"
      - "tpch.supplier" # also include the "supplier" table

sql> show tables;
+---------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name     | table_type |
+---------------+--------------+----------------+------------+
| marketplace   | tpch         | part           | BASE TABLE |
| marketplace   | tpch         | partsupp       | BASE TABLE |
| marketplace   | tpch         | supplier       | BASE TABLE |
+---------------+--------------+----------------+------------+

sql> show tables;
+---------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name     | table_type |
+---------------+--------------+----------------+------------+
| marketplace   | tpch         | part           | BASE TABLE |
| marketplace   | tpch         | partsupp       | BASE TABLE |
| marketplace   | tpch         | supplier       | BASE TABLE |
+---------------+--------------+----------------+------------+

spice.ai/[organization]/[application][/catalog_name]

spice.ai/[organization]/[application][/catalog_name]

catalogs:
  - from: spice.ai/demo-org/tpch
    name: marketplace

catalogs:
  - from: spice.ai/demo-org/tpch
    name: marketplace

SHOW TABLES;
+---------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name     | table_type |
+---------------+--------------+----------------+------------+
| marketplace   | tpch         | region         | BASE TABLE |
| marketplace   | tpch         | orders         | BASE TABLE |
| marketplace   | tpch         | part           | BASE TABLE |
| marketplace

SHOW TABLES;
+---------------+--------------+----------------+------------+
| table_catalog | table_schema | table_name     | table_type |
+---------------+--------------+----------------+------------+
| marketplace   | tpch         | region         | BASE TABLE |
| marketplace   | tpch         | orders         | BASE TABLE |
| marketplace   | tpch         | part           | BASE TABLE |
| marketplace

catalogs:
  - from: spice.ai/demo-org/tpch/custom_catalog
    name: marketplace

catalogs:
  - from: spice.ai/demo-org/tpch/custom_catalog
    name: marketplace

SHOW TABLES;
+------------------+--------------+----------------+------------+
| table_catalog    | table_schema | table_name     | table_type |
+------------------+--------------+----------------+------------+
| custom_catalog   | ice_schema1  | table1         | BASE TABLE |
| custom_catalog   | ice_schema1  | table2         | BASE TABLE |
| custom_catalog   | ice_schema2  | table1         | BASE TABLE |
+------------------+--------------+----------------+------------+

SHOW TABLES;
+------------------+--------------+----------------+------------+
| table_catalog    | table_schema | table_name     | table_type |
+------------------+--------------+----------------+------------+
| custom_catalog   | ice_schema1  | table1         | BASE TABLE |
| custom_catalog   | ice_schema1  | table2         | BASE TABLE |
| custom_catalog   | ice_schema2  | table1         | BASE TABLE |
+------------------+--------------+----------------+------------+

SHOW TABLES;
+------------------+--------------+----------------+------------+
| table_catalog    | table_schema | table_name     | table_type |
+------------------+--------------+----------------+------------+
| marketplace      | ice_schema1  | table1         | BASE TABLE |
| marketplace      | ice_schema1  | table2         | BASE TABLE |
| marketplace      | ice_schema2  | table1         | BASE TABLE |
+------------------+--------------+----------------+------------+

SHOW TABLES;
+------------------+--------------+----------------+------------+
| table_catalog    | table_schema | table_name     | table_type |
+------------------+--------------+----------------+------------+
| marketplace      | ice_schema1  | table1         | BASE TABLE |
| marketplace      | ice_schema1  | table2         | BASE TABLE |
| marketplace      | ice_schema2  | table1         | BASE TABLE |
+------------------+--------------+----------------+------------+

from: spice.ai/demo-org/tpch
name: marketplace

from: spice.ai/demo-org/tpch
name: marketplace

spice # Default catalog
  |- schema1
     |- table1
     |- table2

spice # Default catalog
  |- schema1
     |- table1
     |- table2

marketplace
  |- schema1
     |- table1
     |- table2

marketplace
  |- schema1
     |- table1
     |- table2

include:
  - "tpch.part*" # Include all tables from the "tpch" schema that start with "part"
  - "tpch.supplier" # Include the "supplier" table from the "tpch" schema

include:
  - "tpch.part*" # Include all tables from the "tpch" schema that start with "part"
  - "tpch.supplier" # Include the "supplier" table from the "tpch" schema