Spice AI Announces Contribution of TableProviders for PostgreSQL, MySQL, DuckDB, and SQLite to the Apache DataFusion Project
Spice AI has contributed new TableProviders for PostgreSQL, MySQL, DuckDB, and SQLite to the Apache DataFusion project. This addition reflects our commitment to building together in the data and AI ecosystem and supporting the open-source community.
What is Apache DataFusion?
Apache DataFusion is a high-performance query engine built on Apache Arrow. It allows you to execute SQL queries quickly and efficiently on data stored in various formats. By using the in-memory columnar format of Apache Arrow, DataFusion speeds up data processing and works natively with other Arrow-based tools.
About the Spice OSS Project
Spice OSS is an open-source project from Spice AI that provides developers with a unified SQL query interface to locally materialize, accelerate, and query datasets from any database, data warehouse, or data lake. Spice OSS incorporates Apache DataFusion as its SQL query engine.
Our goal with Spice OSS is to make data and AI-driven development more accessible. By contributing to projects like DataFusion and Arrow, we can help making accessing and using data better for everyone building in the space.
New TableProviders for DataFusion
We've initially added TableProviders for PostgreSQL, MySQL, DuckDB, and SQLite to DataFusion. This expands the range of data sources you can query using DataFusion. And we plan to add more in the future.
- PostgreSQL: A robust and extensible open-source relational database.
- MySQL: A reliable and user-friendly open-source relational database.
- DuckDB: An in-process SQL OLAP database for analytical queries.
- SQLite: A lightweight, disk-based database commonly used in embedded systems.
These new TableProviders make DataFusion even more versatile, allowing you to work with your existing databases and data lakes more easily.
Our Commitment to Data and AI
At Spice AI, we believe in the power of open-source and the potential of data and AI. By contributing to Apache DataFusion, we're helping to advance data processing technology and make powerful tools available to developers everywhere.
To learn more about Spice AI and our open-source projects, check out our GitHub repository. You can also explore Apache DataFusion at datafusion.apache.org.
Stay tuned for more updates from Spice AI as we continue to contribute to the data and AI ecosystem.
Get the latest insights
New releases, tutorials, platform updates, and more.
Explore more lorem ipsum sit amet consectetur
Vorem ipsum dolor sit amet, consectetur adipiscing elit. Nunc vulputate libero et velit interdum, ac aliquet odio mattis.
Real-Time Hybrid Search Using RRF: A Hands-On Guide with Spice
Surfacing relevant answers to searches across datasets has historically meant navigating significant tradeoffs. Keyword (or lexical) search is fast, cheap, and commoditized, but limited by the constraints of exact matching. Vector (or semantic) search captures nuance and intent, but can be slower, harder to debug, and expensive to run at scale. Combining both usually entails standing up multiple engines […]
Spice Cloud v1.8.0: Iceberg Write Support, Acceleration Snapshots & More
Spice Cloud & Spice.ai Enterprise 1.8.0 are live! v1.8.0 includes Iceberg write support, acceleration snapshots, partitioned S3 Vector indexes, a new AI SQL function for LLM integration, and an updated Spice.js SDK. V1.8.0 also introduces developer experience upgrades, including a redesigned Spice Cloud dashboard with tabbed navigation: Spice Cloud customers will automatically upgrade to v1.8.0 on deployment, while […]
Spice Cloud v1.7.0: DataFusion v49, Full-Text Search Updates & More
Spice Cloud & Spice.ai Enterprise 1.7.0 are now live, bringing performance upgrades with DataFusion v49, real-time full-text search indexing, EmbeddingGemma support, and improvements across search, embeddings, and API integrations. Spice Cloud customers will automatically upgrade to v1.7.0 on deployment, while Spice.ai Enterprise customers can consume the Enterprise v1.7.0 image from the Spice AWS Marketplace listing. What’s New in v1.7.0 […]
See Spice in action
Get a guided walkthrough of how development teams use Spice to query, accelerate, and integrate AI for mission-critical workloads.
Get a demo
