HelpLogin
© 2022-2025 Spice AI, Inc.
SQL Query ReferenceDocsFAQSupport
PrivacyTerms
Status
Home
Datasets
Models
trunk
Edit on GitHub
Fork
/docs/website/blog/releases/v0.13.1-alpha.md
spiceai/docs/README.md

date: 2024-05-27 title: 'Spice v0.13.1-alpha (May 27, 2024)' type: blog authors: [lukekim] categories: [release] tags: [results caching, stability, operability, dataset schema, query history]

The v0.13.1-alpha release of Spice is a minor update focused on stability, quality, and operability. Query result caching provides protection against bursts of queries and schema support for datasets has been added logical grouping. An issue where Refresh SQL predicates were not pushed down underlying data sources has been resolved along with improved Acceleration Refresh logging.

Highlights in v0.13.1-alpha

  • Results Caching: Introduced query results caching to handle bursts of requests and support caching of non-accelerated results, such as refresh data returned on zero results. Results caching is enabled by default with a 1s item time-to-live (TTL). Learn more.

  • Query History Logging: Recent queries are now logged in the new spice.runtime.query_history dataset with a default retention of 24-hours. Query history is initially enabled for HTTP queries only (not Arrow Flight queries).

  • Dataset Schemas: Added support for dataset schemas, allowing logical grouping of datasets by separating the schema name from the table name with a .. E.g.

    In this example, queries against app.users will be federated to my_schema.my_table, and app.purchases will be federated to app2.purchases.

Contributors

@y-f-u @Jeadie @sgrebnov @ewgenius @phillipleblanc @lukekim @gloomweaver @Sevenannn

New in this release

  • Add more type support on mysql connector by @y-f-u in https://github.com/spiceai/spiceai/pull/1449
  • Add in-memory caching support for Arrow Flight queries by @sgrebnov in https://github.com/spiceai/spiceai/pull/1450
  • Fix the table reference to use the full table reference, not just the table by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1460
  • Make file_format parameter required for S3/FTP/SFTP connector by @ewgenius in https://github.com/spiceai/spiceai/pull/1455
  • Add more verbose logging when acceleration refresh update is finished by @y-f-u in https://github.com/spiceai/spiceai/pull/1453
  • Fix snowflake dataset path when using federation query by @y-f-u in https://github.com/spiceai/spiceai/pull/1474
  • Update cargo to use spiceai datafusion fork by @y-f-u in https://github.com/spiceai/spiceai/pull/1475
  • Enable in-memory results caching by default by @sgrebnov in https://github.com/spiceai/spiceai/pull/1473
  • Add basic integration test for MySQL federation by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1477
  • Update results_cache config names per final spec by @sgrebnov in https://github.com/spiceai/spiceai/pull/1487
  • Add DuckDB quickstart to E2E tests by @lukekim in https://github.com/spiceai/spiceai/pull/1461
  • Add X-Cache header for http queries by @sgrebnov in https://github.com/spiceai/spiceai/pull/1472
  • Add telemetry for in-memory caching by @sgrebnov in https://github.com/spiceai/spiceai/pull/1456
  • Pin Git dependencies to a specific commit hash by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1490
  • Detect file_format from dataset path by @ewgenius in https://github.com/spiceai/spiceai/pull/1489
  • Add file_format to helm chart sample dataset by @ewgenius in https://github.com/spiceai/spiceai/pull/1493
  • Improve duckdb data connector error messages by @Sevenannn in https://github.com/spiceai/spiceai/pull/1486
  • Add file_format prompt for s3 and ftp datasets in Dataset Configure CLI if no extension detected by @ewgenius in https://github.com/spiceai/spiceai/pull/1494
  • Add llms to the spicepod definition and use throughout by @Jeadie in https://github.com/spiceai/spiceai/pull/1447
  • Fix duckdb acceleration converting null into default values. by @y-f-u in https://github.com/spiceai/spiceai/pull/1500
  • Separate runtime Dataset from spicepod Dataset by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1503
  • Duckdb e2e test OSX support by @y-f-u in https://github.com/spiceai/spiceai/pull/1505
  • Use TableReference for dataset name by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1506
  • Tweak Results Cache naming and output by @lukekim in https://github.com/spiceai/spiceai/pull/1509
  • Fix refresh_sql not properly passing down filters by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1510
  • Allow datasets to specify a schema by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1507
  • Cache invalidation for accelerated tables by @sgrebnov in https://github.com/spiceai/spiceai/pull/1498
  • Improve spark data connector error messages by @Sevenannn in https://github.com/spiceai/spiceai/pull/1497
  • Parse postgres table schema from prepare statement to support empty tables by @ewgenius in https://github.com/spiceai/spiceai/pull/1445
  • Improve clarity of README and add FAQ by @lukekim in https://github.com/spiceai/spiceai/pull/1512
  • Use binary data transfer for ftp by @gloomweaver in https://github.com/spiceai/spiceai/pull/1517
  • Add support for time64 for SQL insertion statement by @y-f-u in https://github.com/spiceai/spiceai/pull/1519
  • Add Spice Extensions PoC by @ewgenius in https://github.com/spiceai/spiceai/pull/1476
  • Add results cache metrics, pod and quantile filters to Grafana dashboard by @sgrebnov in https://github.com/spiceai/spiceai/pull/1513
  • Add unit tests for results caching utils by @sgrebnov in https://github.com/spiceai/spiceai/pull/1514
  • Add E2E tests for results caching by @sgrebnov in https://github.com/spiceai/spiceai/pull/1515
  • Pass table_reference full string into spark_session table so it can query across schemas or catalogs by @y-f-u in https://github.com/spiceai/spiceai/pull/1521
  • Trace on debug level for tables in runtime schema by @ewgenius in https://github.com/spiceai/spiceai/pull/1524
  • Update SparkSessionBuilder::remote and update spark fork hash by @Sevenannn in https://github.com/spiceai/spiceai/pull/1495
  • Fix federation push-down for datasets with schemas by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1526
  • Store history of queries in 'spice.runtime.query_history' by @Jeadie in https://github.com/spiceai/spiceai/pull/1501
  • Disable cache for system queries by @sgrebnov in https://github.com/spiceai/spiceai/pull/1528
  • Register runtime tables with runtime schema by @phillipleblanc in https://github.com/spiceai/spiceai/pull/1532
  • Fix acknowledgments workflow to include all cargo features by @Jeadie in https://github.com/spiceai/spiceai/pull/1531

Full Changelog: https://github.com/spiceai/spiceai/compare/v0.13.0-alpha...v0.13.1-alpha

Resources

  • Getting started with Spice.ai
  • Documentation

Community

Spice.ai started with the vision to make AI easy for developers. We are building Spice.ai in the open and with the community. Reach out on Discord or by email to get involved.

  • Twitter: @spice_ai
  • Discord: https://discord.gg/kZnTfneP5u
  • Telegram: Spice AI Discussion
  • Reddit: https://www.reddit.com/r/spiceai
  • Email: hey@spice.ai
datasets:
- from: mysql:app1.identities
name: app.users

- from: postgres:app2.purchases
name: app.purchases
datasets:
- from: mysql:app1.identities
name: app.users

- from: postgres:app2.purchases
name: app.purchases