date: 2024-07-29 title: 'Spice v0.17-beta (July 29, 2024)' type: blog authors: [phillipleblanc] categories: [release] tags: [beta release, data-connector, tls, sqlite, secrets replacement]

Announcing the first beta release of Spice.ai OSS! 🎉

The core Spice runtime has graduated from alpha to beta! Components, such as Data Connectors and Models, follow independent release milestones. Data Connectors graduating from alpha to beta include databricks, spiceai, postgres, s3, odbc, and mysql. From beta to 1.0, project will be to on improving performance and scaling to larger datasets.

This release also includes enhanced security with Transport Layer Security (TLS) secured APIs, a new spice install CLI command, and several performance and stability improvements.

Highlights in v0.17-beta

• Encryption in transit with TLS: The HTTP, gRPC, Metrics, and OpenTelemetry (OTEL) API endpoints can be secured with TLS by specifying a certificate and private key in PEM (opens in a new tab) format.

Enable TLS using the --tls-certificate-file and --tls-key-file command-line flags:

spice run -- --tls-certificate-file /path/to/cert.pem --tls-key-file /path/to/key.pem

spice run -- --tls-certificate-file /path/to/cert.pem --tls-key-file /path/to/key.pem

Or configure in the spicepod.yml:

runtime:
  tls:
    certificate_file: /path/to/cert.pem
    key_file: /path/to/key.pem

runtime:
  tls:
    certificate_file: /path/to/cert.pem
    key_file: /path/to/key.pem

Get started with TLS by following the TLS Sample (opens in a new tab). For more details see the TLS Documentation (opens in a new tab).

• spice install: Running the spice install CLI command will download and install the latest version of the runtime.

spice install

spice install

• Improved SQLite and DuckDB compatibility: The SQLite and DuckDB accelerators support more complex queries and additional data types.

• Pass through arguments from spice run to runtime: Arguments passed to spice run are now passed through to the runtime.

• Secrets replacement within connection strings: Secrets are now replaced within connection strings:

datasets:
  - from: mysql:my_table
    name: my_table
    params:
      mysql_connection_string: mysql://user:${secrets:mysql_pw}@localhost:3306/db

datasets:
  - from: mysql:my_table
    name: my_table
    params:
      mysql_connection_string: mysql://user:${secrets:mysql_pw}@localhost:3306/db

Breaking Changes

The odbc data connector is now optional and has been removed from the released binaries. To use the odbc data connector, use the official Spice Docker image or build the Spice runtime from source.

To build Spice from source with the odbc feature:

cargo build --release --features odbc

cargo build --release --features odbc

To use the official Spice Docker image from DockerHub (opens in a new tab):

# Pull the latest official Spice image
docker pull spiceai/spiceai:latest

# Pull the official v0.17-beta Spice image
docker pull spiceai/spiceai:0.17.0-beta

# Pull the latest official Spice image
docker pull spiceai/spiceai:latest

# Pull the official v0.17-beta Spice image
docker pull spiceai/spiceai:0.17.0-beta

Contributors

• @y-f-u
• @peasee
• @digadeesh
• @phillipleblanc
• @ewgenius
• @sgrebnov
• @Sevenannn
• @lukekim

What's Changed

Dependencies

• Upgraded delta-kernel-rs (opens in a new tab) to v0.2.0 (opens in a new tab).

Commits

• update helm chart versions for v0.16.0-alpha by @y-f-u in https://github.com/spiceai/spiceai/pull/2057 (opens in a new tab)
• Update spicepod.schema.json by @github-actions in https://github.com/spiceai/spiceai/pull/2060 (opens in a new tab)
• fix: Install unixodbc for E2E test release installation by @peasee in https://github.com/spiceai/spiceai/pull/2063 (opens in a new tab)
• update next release to 0.16.1-beta by @digadeesh in https://github.com/spiceai/spiceai/pull/2065 (opens in a new tab)
• update version to 0.17.0-beta by @digadeesh in https://github.com/spiceai/spiceai/pull/2068 (opens in a new tab)
• Update ROADMAP.md - removing delivered features and updating Beta timeline. by @digadeesh in https://github.com/spiceai/spiceai/pull/2066 (opens in a new tab)
• make bench works for more connectors by @y-f-u in https://github.com/spiceai/spiceai/pull/2042 (opens in a new tab)
• enable spark benchmark by @y-f-u in https://github.com/spiceai/spiceai/pull/2069 (opens in a new tab)
• Make the json_pointer param optional for the GraphQL connector by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2072 (opens in a new tab)
• Fix secrets init to not bail if a secret store can't load by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2073 (opens in a new tab)
• Update end_game.md by @ewgenius in https://github.com/spiceai/spiceai/pull/2059 (opens in a new tab)
• Fix time predicate with timezone info casting for Dremio by @sgrebnov in https://github.com/spiceai/spiceai/pull/2058 (opens in a new tab)
• Add benchmark tests for S3 data connector by @sgrebnov in https://github.com/spiceai/spiceai/pull/2049 (opens in a new tab)
• Add benchmark tests for MySQL data connector by @sgrebnov in https://github.com/spiceai/spiceai/pull/2048 (opens in a new tab)
• fix: Add Athena dialect for ODBC by @peasee in https://github.com/spiceai/spiceai/pull/2084 (opens in a new tab)
• Workflow to build MySQL image with TPCH benchmark data by @sgrebnov in https://github.com/spiceai/spiceai/pull/2070 (opens in a new tab)
• Fix secrets replacement within connection strings by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2086 (opens in a new tab)
• fix: Correctly prefix missing required parameters by @peasee in https://github.com/spiceai/spiceai/pull/2088 (opens in a new tab)
• Add Postgres Data Connector TPCH Benchmark Tests by @Sevenannn in https://github.com/spiceai/spiceai/pull/2009 (opens in a new tab)
• Add spice install CLI command by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2090 (opens in a new tab)
• Use MySQL service container for benchmark tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/2089 (opens in a new tab)
• Remove ODBC from default released binaries by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2092 (opens in a new tab)
• Add cfg flag to properly support build w / wo feature in benchmark tests by @Sevenannn in https://github.com/spiceai/spiceai/pull/2095 (opens in a new tab)
• Move Prometheus metrics server to runtime by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2093 (opens in a new tab)
• fix: Remove unixodbc from test release install by @peasee in https://github.com/spiceai/spiceai/pull/2103 (opens in a new tab)
• Upgrade delta_kernel to 0.2.0 by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2102 (opens in a new tab)
• Allow DuckDB to load extensions in Docker by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2104 (opens in a new tab)
• Spawn the metrics server in the background. by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2105 (opens in a new tab)
• fix: suffix delta kernel table location with slash if none by @y-f-u in https://github.com/spiceai/spiceai/pull/2107 (opens in a new tab)
• Bump object_store from 0.10.1 to 0.10.2 by @dependabot in https://github.com/spiceai/spiceai/pull/2094 (opens in a new tab)
• Decision Record: Default HTTP and GRPC ports for Spice.ai OSS by @digadeesh in https://github.com/spiceai/spiceai/pull/2091 (opens in a new tab)
• Enable TLS for metrics endpoint by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2108 (opens in a new tab)
• Use Postgres container for tpch bench by @Sevenannn in https://github.com/spiceai/spiceai/pull/2112 (opens in a new tab)
• Add workflow to build Postgres Docker image using tpch data by @Sevenannn in https://github.com/spiceai/spiceai/pull/2101 (opens in a new tab)
• Enable TLS for HTTP endpoint by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2109 (opens in a new tab)
• Enable TLS on the Flight GRPC endpoint by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2110 (opens in a new tab)
• add timeout parameters for object store client options by @y-f-u in https://github.com/spiceai/spiceai/pull/2114 (opens in a new tab)
• Enable TLS on the OpenTelemetry GRPC endpoint by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2111 (opens in a new tab)
• feat: Add ODBC Databricks Benches by @peasee in https://github.com/spiceai/spiceai/pull/2113 (opens in a new tab)
• Support configuring TLS in the spicepod by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2118 (opens in a new tab)
• add broken tpch simple queries by @y-f-u in https://github.com/spiceai/spiceai/pull/2116 (opens in a new tab)
• Add integration test for TLS by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2121 (opens in a new tab)
• Improve SQLite and DuckDB compatibility by @sgrebnov in https://github.com/spiceai/spiceai/pull/2122 (opens in a new tab)
• Pass through arguments from spice run and spice sql to runtime by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2123 (opens in a new tab)
• Handle TLS in the spice CLI when connecting to the runtime by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2124 (opens in a new tab)
• Handle connecting over TLS for spice sql by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2125 (opens in a new tab)
• Remove --tls flag by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2128 (opens in a new tab)
• fix: Handle SQLResult error instead of unwrapping by @peasee in https://github.com/spiceai/spiceai/pull/2127 (opens in a new tab)
• Add delta bench by @y-f-u in https://github.com/spiceai/spiceai/pull/2120 (opens in a new tab)
• feat: Add Athena ODBC benches by @peasee in https://github.com/spiceai/spiceai/pull/2129 (opens in a new tab)
• fix: Use odbc-api fork for decimal conversion fix by @peasee in https://github.com/spiceai/spiceai/pull/2133 (opens in a new tab)
• Update benchmarks job env for delta testing by @y-f-u in https://github.com/spiceai/spiceai/pull/2134 (opens in a new tab)
• Use forked dotenvy to disable variable substitution by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2135 (opens in a new tab)
• Remove unnecessary memory allocations in the query path by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2136 (opens in a new tab)
• upgrade spiceai df for tpch simple 6 and 7 by @y-f-u in https://github.com/spiceai/spiceai/pull/2137 (opens in a new tab)
• Avoid more unnecessary allocations in the query path by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2138 (opens in a new tab)

Full Changelog: https://github.com/spiceai/spiceai/compare/v0.16.0-alpha...v0.17-beta (opens in a new tab)

Resources

• Getting started with Spice.ai (opens in a new tab)
• Documentation (opens in a new tab)

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

• Twitter: @spice_ai (opens in a new tab)
• Slack: spiceai.org/slack (opens in a new tab)
• Telegram: Spice AI Discussion (opens in a new tab)
• Reddit: https://www.reddit.com/r/spiceai (opens in a new tab)
• Email: hey@spice.ai (opens in a new tab)