Spice AI has contributed new TableProviders for PostgreSQL, MySQL, DuckDB, and SQLite to the Apache DataFusion project. This addition reflects our commitment to building together in the data and AI ecosystem and supporting the open-source community.
What is Apache DataFusion?
Apache DataFusion is a high-performance query engine built on Apache Arrow. It allows you to execute SQL queries quickly and efficiently on data stored in various formats. By using the in-memory columnar format of Apache Arrow, DataFusion speeds up data processing and works natively with other Arrow-based tools.
About the Spice OSS Project
Spice OSS is an open-source project from Spice AI that provides developers with a unified SQL query interface to locally materialize, accelerate, and query datasets from any database, data warehouse, or data lake. Spice OSS incorporates Apache DataFusion as its SQL query engine.
Our goal with Spice OSS is to make data and AI-driven development more accessible. By contributing to projects like DataFusion and Arrow, we can help making accessing and using data better for everyone building in the space.
New TableProviders for DataFusion
We've initially added TableProviders for PostgreSQL, MySQL, DuckDB, and SQLite to DataFusion. This expands the range of data sources you can query using DataFusion. And we plan to add more in the future.
- PostgreSQL: A robust and extensible open-source relational database.
- MySQL: A reliable and user-friendly open-source relational database.
- DuckDB: An in-process SQL OLAP database for analytical queries.
- SQLite: A lightweight, disk-based database commonly used in embedded systems.
These new TableProviders make DataFusion even more versatile, allowing you to work with your existing databases and data lakes more easily.
Our Commitment to Data and AI
At Spice AI, we believe in the power of open-source and the potential of data and AI. By contributing to Apache DataFusion, we're helping to advance data processing technology and make powerful tools available to developers everywhere.
To learn more about Spice AI and our open-source projects, check out our GitHub repository. You can also explore Apache DataFusion at datafusion.apache.org.
Stay tuned for more updates from Spice AI as we continue to contribute to the data and AI ecosystem.
Interested in working with Spice AI or looking to learn a little more about the work we do? We are always looking for our next big challenge. Book an introductory call via our Calendly. Take a deeper look at our enterprise offerings by visiting Spice.ai.
