The v0.15.1-alpha minor release focuses on enhancing stability, performance, and usability. Memory usage has been significantly improved for the postgres and duckdb acceleration engines which now use stream processing. A new Delta Lake Data Connector has been added, sharing a delta-kernel-rs (opens in a new tab) based implementation with the Databricks Data Connector supporting deletion vectors (opens in a new tab).
Improved memory usage for PostgreSQL and DuckDB acceleration engines: Large dataset acceleration with PostgreSQL (opens in a new tab) and DuckDB (opens in a new tab) engines has reduced memory consumption by streaming data directly to the accelerated table as it is read from the source.
Delta Lake Data Connector: A new Delta Lake Data Connector (opens in a new tab) has been added for using Delta Lake (opens in a new tab) outside of Databricks.
ODBC Data Connector Streaming: The ODBC Data Connector (opens in a new tab) now streams results, reducing memory usage, and improving performance.
GraphQL Object Unnesting: The GraphQL Data Connector (opens in a new tab) can automatically unnest objects from GraphQL queries using the unnest_depth parameter.
None.
The MySQL, PostgreSQL, SQLite and DuckDB DataFusion TableProviders developed by Spice AI have been donated to the datafusion-contrib/datafusion-table-providers (opens in a new tab) community repository.
From the v0.15.1-alpha release, a new dependency is taken on datafusion-contrib/datafusion-table-providers
Full Changelog: https://github.com/spiceai/spiceai/compare/v0.15.0-alpha...v0.15.1-alpha (opens in a new tab)
Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Slack or by email to get involved.
datafusion-table-providers crate by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1873 (opens in a new tab)delta-rs with delta-kernel-rs and add new delta data connector. by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1878 (opens in a new tab)delta tables by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1891 (opens in a new tab)delta to delta_lake by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1892 (opens in a new tab)