HelpLogin
© 2022-2025 Spice AI, Inc.
SQL Query ReferenceDocsFAQSupport
PrivacyTerms
Status
Home
Datasets
Models
trunk
Edit on GitHub
Fork
/docs/website/blog/releases/v0.16-alpha.md
spiceai/docs/README.md

date: 2024-07-22 title: 'Spice v0.16-alpha (July 22, 2024)' type: blog authors: [lukekim] categories: [release] tags: [secret store, odbc connector, beta milestone, java sdk, federated sql]

The v0.16-alpha release is the first candidate release for the beta milestone on a path to finalizing the v1.0 developer and user experience. Upgraders should be aware of several breaking changes designed to improve the Secrets configuration experience and to make authoring spicepod.yml files more consistent. See the [Breaking Changes](#Breaking Changes) section below for details. Additionally, the Spice Java SDK was released, providing Java developers a simple but powerful native experience to query Spice.

Highlights in v0.16-alpha

  • Secret Stores: More than one Secret Store can now be specified. For example, to configure Spice with both Environment Variable and AWS Secrets Manager Secret Stores, use the following secrets configuration in spicepod.yaml:

Secrets managed by configured Secret Stores can be referenced in component params using the syntax ${<store_name>:<key>}. E.g.

  • Java Client SDK: The Spice Java SDK has been released for JDK 17 or greater.

  • Federated SQL Query: Significant stability and reliability improvements have been made to federated SQL query support in most data connectors.

  • ODBC Data Connector: Providing a specific SQL dialect to query ODBC data sources is now supported using the sql_dialect param. For example, when querying Databricks using ODBC, the databricks dialect can be specified to ensure compatibility. Read the ODBC Data Connector documentation for more details.

Breaking Changes

  • Secret Stores: Secret Stores support has been overhauled including required changes to spicepod.yml schema. File based secrets stored in the ~/.spice/auth file are no longer supported. See Secret Stores Documentation for full reference.

To upgrade Secret Stores, rename any parameters ending in _key to remove the _key suffix and specify a secret inline via the secret replacement syntax (${<secret_store>:<key>}):

to:

And ensure the MY_PG_PASS environment variable is set.

  • Datasets: The default value of time_format has changed from unix_seconds to timestamp.

To upgrade:

  • HTTP Port: The default HTTP port has changed from port 3000 to port 8090 to avoid conflicting with frontend apps which typically use the 3000 range. If an SDK is used, upgrade it at the same time as the runtime.

To upgrade and continue using port 3000, run spiced with the --http command line argument:

  • HTTP Metrics Port: The default HTTP Metrics port has changed from port 9000 to 9090 to avoid conflicting with other metrics protocols which typically use port 9000.

To upgrade and continue using port 9000, run spiced with the metrics command line argument:

  • GraphQL Data Connector: json_path has been replaced with json_pointer to access nested data from the result of the GraphQL query. See the GraphQL Data Connector documentation for full details and RFC-6901 - JSON Pointer.

To upgrade, change:

To:

  • Data Connector Configuration: Consistent connector name prefixing has been applied to connector specific params parameters. Prefixed parameter names helps ensure parameters do not collide.

For example, the Databricks data connector specific params are now prefixed with databricks:

To upgrade:

Refer to the Data Connector documentation for parameter naming changes in this release.

Clickhouse Data Connector: The clickhouse_connection_timeout parameter has been renamed to connection_timeout as it applies to the client and is not Clickhouse configuration itself.

To upgrade, change:

To:

Contributors

  • @y-f-u
  • @phillipleblanc
  • @ewgenius
  • @github-actions
  • @sgrebnov
  • @lukekim
  • @digadeesh
  • @peasee
  • @Sevenannn

What's Changed

Dependencies

No major dependency updates.

Commits

  • bump helm chart versions to 0.15.2-alpha by @y-f-u in https://github.com/spiceai/spiceai/pull/1975
  • Remove unused Cargo.toml fields by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1981
  • Update version to 0.16.0-beta by @ewgenius in https://github.com/spiceai/spiceai/pull/1983
  • Update spicepod.schema.json by @github-actions in https://github.com/spiceai/spiceai/pull/1984
  • Enable sqlite acceleration testing in E2E by @sgrebnov in https://github.com/spiceai/spiceai/pull/1980
  • Revert "Revert "fix: validate time column and time format when constructing accelerated table refresh"" by @y-f-u in https://github.com/spiceai/spiceai/pull/1982
  • Add Datadog dashboard skeleton by @sgrebnov in https://github.com/spiceai/spiceai/pull/1971
  • Format Cargo.toml with taplo by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1988
  • Spice cli spice chat command, to interact with deployed spiced instance in spice.ai cloud by @ewgenius in https://github.com/spiceai/spiceai/pull/1990
  • Use platform api /v1/chat/completions with streaming in spice chat cli command by @ewgenius in https://github.com/spiceai/spiceai/pull/1998
  • update spiceai datafusion version to fix tpch queries by @y-f-u in https://github.com/spiceai/spiceai/pull/2001
  • Install a rustls default CryptoProvider by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2003
  • Roadmap update July, 2024 by @lukekim in https://github.com/spiceai/spiceai/pull/2002
  • Add local spice runtime support for spice chat command, add --model flag by @ewgenius in https://github.com/spiceai/spiceai/pull/2007
  • fix: GraphQL Data Connector - Change json path to json pointer by @digadeesh in https://github.com/spiceai/spiceai/pull/1930
  • Update ROADMAP.md to include MySQL data connector in Beta by @digadeesh in https://github.com/spiceai/spiceai/pull/2016
  • Load secrets from multiple secret stores & secrets UX refresh by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2011
  • upgrade spiceai datafusion to fix tpch simple query 3 by @y-f-u in https://github.com/spiceai/spiceai/pull/2021
  • feat: Autodetect ODBC dialect by @peasee in https://github.com/spiceai/spiceai/pull/1997
  • feat: Use CustomDialectBuilder for Databricks ODBC dialect by @peasee in https://github.com/spiceai/spiceai/pull/2020
  • Switch the secret replacement syntax to ${ <secret>:<key> } by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2026
  • fix spiceai connector lengthy error by @y-f-u in https://github.com/spiceai/spiceai/pull/2024
  • Log parameter key instead of value when injecting secret by @Sevenannn in https://github.com/spiceai/spiceai/pull/2031
  • Update benchmark yml to support postgres benchmark test by @Sevenannn in https://github.com/spiceai/spiceai/pull/2032
  • Separate data connector parameters into connector and runtime categories by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2028
  • Fix spice chat prompt and spinner by @ewgenius in https://github.com/spiceai/spiceai/pull/2029
  • Build spiced with odbc for release binaries by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2036
  • MySQL timestamp, int64 casting, date part extraction and intervals support by @sgrebnov in https://github.com/spiceai/spiceai/pull/2035
  • updating default http and metrics ports by @digadeesh in https://github.com/spiceai/spiceai/pull/2034
  • enable spark connect federated query by @y-f-u in https://github.com/spiceai/spiceai/pull/2041
  • fix: Use MySQL Interval for Databricks ODBC by @peasee in https://github.com/spiceai/spiceai/pull/2037
  • Re-enable test_quickstart_dremio E2E test by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2045
  • Fix ODBC build for release binaries by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2046
  • chore: Remove unused dependencies by @peasee in https://github.com/spiceai/spiceai/pull/2044
  • fix: Change version to alpha breaking by @peasee in https://github.com/spiceai/spiceai/pull/2051
  • Add connector prefix for dataset configure endpoint param by @sgrebnov in https://github.com/spiceai/spiceai/pull/2052
  • Fix unprefixed runtime parameters by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2050
  • Fix make install-with-models by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2054
  • Bump openssl from 0.10.64 to 0.10.66 by @dependabot in https://github.com/spiceai/spiceai/pull/2047
  • Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/2056
  • ignore empty constraints when creating accelerated table by @y-f-u in https://github.com/spiceai/spiceai/pull/2055

Full Changelog: https://github.com/spiceai/spiceai/compare/v0.15.2-alpha...v0.16.0-alpha

Resources

  • Getting started with Spice.ai
  • Documentation

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.

  • Twitter: @spice_ai
  • Discord: https://discord.gg/kZnTfneP5u
  • Telegram: Spice AI Discussion
  • Reddit: https://www.reddit.com/r/spiceai
  • Email: hey@spice.ai
secrets:
- from: env
name: env
- from: aws_secrets_manager:my_secret_name
name: aws_secret
secrets:
- from: env
name: env
- from: aws_secrets_manager:my_secret_name
name: aws_secret
datasets:
- from: postgres:my_table
name: my_table
params:
pg_host: localhost
pg_port: 5432
pg_pass: ${ env:MY_PG_PASS }
datasets:
- from: postgres:my_table
name: my_table
params:
pg_host: localhost
pg_port: 5432
pg_pass: ${ env:MY_PG_PASS }
datasets:
- from: postgres:my_table
name: my_table
params:
pg_host: localhost
pg_port: 5432
pg_pass_key: my_pg_pass
datasets:
- from: postgres:my_table
name: my_table
params:
pg_host: localhost
pg_port: 5432
pg_pass_key: my_pg_pass
datasets:
- from: postgres:my_table
name: my_table
params:
pg_host: localhost
pg_port: 5432
pg_pass: ${secrets:my_pg_pass}
datasets:
- from: postgres:my_table
name: my_table
params:
pg_host: localhost
pg_port: 5432
pg_pass: ${secrets:my_pg_pass}
datasets:
- from:
name: my_dataset
# Explicitly define format when not specified.
time_format: unix_seconds
datasets:
- from:
name: my_dataset
# Explicitly define format when not specified.
time_format: unix_seconds
json_path: my.json.path
json_path: my.json.path
json_pointer: /my/json/pointer
json_pointer: /my/json/pointer
datasets:
- from: databricks:spiceai.datasets.my_awesome_table # A reference to a table in the Databricks unity catalog
name: my_delta_lake_table
params:
mode: spark_connect
endpoint: dbc-a1b2345c-d6e7.cloud.databricks.com
token: MY_TOKEN
datasets:
- from: databricks:spiceai.datasets.my_awesome_table # A reference to a table in the Databricks unity catalog
name: my_delta_lake_table
params:
mode: spark_connect
endpoint: dbc-a1b2345c-d6e7.cloud.databricks.com
token: MY_TOKEN
datasets:
# Example for Spark Connect
- from: databricks:spiceai.datasets.my_awesome_table # A reference to a table in the Databricks unity catalog
name: my_delta_lake_table
params:
mode: spark_connect
databricks_endpoint: dbc-a1b2345c-d6e7.cloud.databricks.com # Now prefixed with databricks
databricks_token: ${secrets:my_token} # Now prefixed with databricks
datasets:
# Example for Spark Connect
- from: databricks:spiceai.datasets.my_awesome_table # A reference to a table in the Databricks unity catalog
name: my_delta_lake_table
params:
mode: spark_connect
databricks_endpoint: dbc-a1b2345c-d6e7.cloud.databricks.com # Now prefixed with databricks
databricks_token: ${secrets:my_token} # Now prefixed with databricks
clickhouse_connection_timeout: time
clickhouse_connection_timeout: time
connection_timeout: time
connection_timeout: time
# Using Dockerfile or spiced directly
spiced --http 127.0.0.1:3000
# Using Dockerfile or spiced directly
spiced --http 127.0.0.1:3000
# Using Dockerfile or spiced directly
spiced --metrics 127.0.0.1:9000
# Using Dockerfile or spiced directly
spiced --metrics 127.0.0.1:9000