spiceai/docs

date: 2024-08-05 title: 'Spice v0.17.1-beta (August 5, 2024)' type: blog authors: [phillipleblanc] categories: [release] tags: [beta release, data-connector, tls, sqlite, secrets replacement]

The v0.17.1-beta minor release focuses on enhancing stability, performance, and usability. The Flight interface now supports the GetSchema API and s3, ftp, sftp, http, https, and databricks data connectors have added support for a client_timeout parameter.

Highlights in v0.17.1-beta

Flight API GetSchema: The GetSchema (opens in a new tab) API is now supported by the Flight interface. The schema of a dataset can be retrieved using GetSchema with the PATH or CMD FlightDescriptor types. The CMD FlightDescriptor type is used to get the schema of an arbitrary SQL query as the CMD bytes. The PATH FlightDescriptor type is used to retrieve the schema of a dataset.

Client Timeout: A client_timeout parameter has been added for Data Connectors: ftp, sftp, http, https, and databricks. When defined, the client timeout configures Spice to stop waiting for a response from the data source after the specified duration. The default timeout is 30 seconds.

datasets:
  - from: ftp://remote-ftp-server.com/path/to/folder/
    name: my_dataset
    params:
      file_format: csv
      # Example client timeout
      client_timeout: 30s
      ftp_user: my-ftp-user
      ftp_pass: ${secrets:my_ftp_password}

datasets:
  - from: ftp://remote-ftp-server.com/path/to/folder/
    name: my_dataset
    params:
      file_format: csv
      # Example client timeout
      client_timeout: 30s
      ftp_user: my-ftp-user
      ftp_pass: ${secrets:my_ftp_password}

Breaking Changes

TLS is now required to be explicitly enabled. Enable TLS on the command line using --tls-enabled true:

spice run -- --tls-enabled true --tls-certificate-file /path/to/cert.pem --tls-key-file /path/to/key.pem

spice run -- --tls-enabled true --tls-certificate-file /path/to/cert.pem --tls-key-file /path/to/key.pem

Or in the spicepod.yml with enabled: true:

runtime:
  tls:
    # TLS explicitly enabled
    enabled: true
    certificate_file: /path/to/cert.pem
    key_file: /path/to/key.pem

runtime:
  tls:
    # TLS explicitly enabled
    enabled: true
    certificate_file: /path/to/cert.pem
    key_file: /path/to/key.pem

Contributors

@Jeadie
@y-f-u
@phillipleblanc
@sgrebnov
@peasee
@Sevenannn

What's Changed

Dependencies

Rust: Upgraded from v1.79.0 to v1.80.0

Commits

Update README.md by @Jeadie in https://github.com/spiceai/spiceai/pull/2142 (opens in a new tab)
update helm chart to 0.17.0-beta by @y-f-u in https://github.com/spiceai/spiceai/pull/2144 (opens in a new tab)
Update spicepod.schema.json by @github-actions in https://github.com/spiceai/spiceai/pull/2143 (opens in a new tab)
Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/2141 (opens in a new tab)
Update Spice runtime to require explicit enablement for TLS by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2148 (opens in a new tab)
Update next version, ROADMAP, End Game template, move alpha release notes by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2145 (opens in a new tab)
Update EXTENSIBILITY to be correct, update README.md with Beta connectors by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2146 (opens in a new tab)
Add benchmark tests for duckdb acceleration by @sgrebnov in https://github.com/spiceai/spiceai/pull/2151 (opens in a new tab)
fix: Increase benchmark dataset setup timeout for Databricks by @peasee in https://github.com/spiceai/spiceai/pull/2149 (opens in a new tab)
Add LLMs to v1/models by @Jeadie in https://github.com/spiceai/spiceai/pull/2152 (opens in a new tab)
Dataset with acceleration enabled = false shouldn't go through accelerated dataset hot reload by @Sevenannn in https://github.com/spiceai/spiceai/pull/2155 (opens in a new tab)
Show single error string in Spice SQL REPL command line by @Sevenannn in https://github.com/spiceai/spiceai/pull/2150 (opens in a new tab)
Add CI to build makefile install targets by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2157 (opens in a new tab)
Make the FlightClient struct cheap to clone by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2162 (opens in a new tab)
Fix bugs with local Unity Catalog server by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2160 (opens in a new tab)
Benchmark: data connector tests should continue on query error (s3) by @sgrebnov in https://github.com/spiceai/spiceai/pull/2161 (opens in a new tab)
fix hanging spiced when odbc loading data and received a cancel signal by @y-f-u in https://github.com/spiceai/spiceai/pull/2156 (opens in a new tab)
Improve MySql schema extraction and add InList and ScalarFunction expr support by @sgrebnov in https://github.com/spiceai/spiceai/pull/2158 (opens in a new tab)
Fix issue with use of EmbeddingConnector by @Jeadie in https://github.com/spiceai/spiceai/pull/2165 (opens in a new tab)
add client timeout for all object store providers by @y-f-u in https://github.com/spiceai/spiceai/pull/2168 (opens in a new tab)
Benchmark: include sqlite acceleration and enable more tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/2172 (opens in a new tab)
feat: Use datafusion SQLite streaming updates by @peasee in https://github.com/spiceai/spiceai/pull/2171 (opens in a new tab)
Benchmark: include arrow acceleration and enable more tests (tpch_q22) by @sgrebnov in https://github.com/spiceai/spiceai/pull/2173 (opens in a new tab)
Localhost -> Sink; Fix Sink connector to not require schema via CREATE TABLE... and infer on first write by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2167 (opens in a new tab)
Fix misspelled acceleration engine name in benchmark tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/2175 (opens in a new tab)
update spark bench catalog by @y-f-u in https://github.com/spiceai/spiceai/pull/2178 (opens in a new tab)
Benchmark: Discard first measurement of sql query, disable result caching by @Sevenannn in https://github.com/spiceai/spiceai/pull/2179 (opens in a new tab)
clear message when invalid params configured for accelerator by @y-f-u in https://github.com/spiceai/spiceai/pull/2177 (opens in a new tab)
Implement the Flight GetSchema API by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2169 (opens in a new tab)
Support AppendStream for SpiceAI data connector by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2181 (opens in a new tab)
Support MySQL BINARY, VARBINARY, Postgres BYTEA and improve MySQL auth error message by @sgrebnov in https://github.com/spiceai/spiceai/pull/2184 (opens in a new tab)
Benchmark: use SF1 for MySQL TPC-H tests by @sgrebnov in https://github.com/spiceai/spiceai/pull/2183 (opens in a new tab)
fix windows build broken by adding tokio unix signal by @y-f-u in https://github.com/spiceai/spiceai/pull/2193 (opens in a new tab)
Adds TLS support for flightsubscriber/flightpublisher tools by @phillipleblanc in https://github.com/spiceai/spiceai/pull/2194 (opens in a new tab)
Update README output samples by @ewgenius in https://github.com/spiceai/spiceai/pull/2195 (opens in a new tab)
Update acknowledgements by @github-actions in https://github.com/spiceai/spiceai/pull/2197 (opens in a new tab)

Full Changelog: https://github.com/spiceai/spiceai/compare/v0.17.0-beta...v0.17.1-beta (opens in a new tab)

Resources

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.

Twitter: @spice_ai (opens in a new tab)
Slack: spiceai.org/slack (opens in a new tab)
Telegram: Spice AI Discussion (opens in a new tab)
Reddit: https://www.reddit.com/r/spiceai (opens in a new tab)
Email: hey@spice.ai (opens in a new tab)

spiceai/docs/README.md