The v0.15.1-alpha minor release focuses on enhancing stability, performance, and usability. Memory usage has been significantly improved for the postgres and duckdb acceleration engines which now use stream processing. A new Delta Lake Data Connector has been added, sharing a delta-kernel-rs based implementation with the Databricks Data Connector supporting deletion vectors.
Improved memory usage for PostgreSQL and DuckDB acceleration engines: Large dataset acceleration with PostgreSQL and DuckDB engines has reduced memory consumption by streaming data directly to the accelerated table as it is read from the source.
Delta Lake Data Connector: A new Delta Lake Data Connector has been added for using Delta Lake outside of Databricks.
ODBC Data Connector Streaming: The ODBC Data Connector now streams results, reducing memory usage, and improving performance.
GraphQL Object Unnesting: The GraphQL Data Connector can automatically unnest objects from GraphQL queries using the unnest_depth parameter.
None.
The MySQL, PostgreSQL, SQLite and DuckDB DataFusion TableProviders developed by Spice AI have been donated to the datafusion-contrib/datafusion-table-providers community repository.
From the v0.15.1-alpha release, a new dependency is taken on datafusion-contrib/datafusion-table-providers
datafusion-table-providers crate by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1873delta-rs with delta-kernel-rs and add new delta data connector. by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1878delta tables by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1891delta to delta_lake by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1892Full Changelog: https://github.com/spiceai/spiceai/compare/v0.15.0-alpha...v0.15.1-alpha
Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.