🎓 Learn how it works with the Amazon S3 Vectors with Spice (opens in a new tab) engineering blog post.
Data sourced by Data Connectors, or views built atop them with vector embedding columns can be indexed and efficiently searched using a vector engine.
A vector engine will store all vector embeddings associated with columns in a dataset/view, provide efficient vector search operations and avoid unnecessary recomputation of embeddings.
A vector engine is configured by setting the vectors configuration. E.g.
For the complete reference specification see datasets (opens in a new tab).
Supported Vector engines:
| Name | Description |
|---|---|
s3_vectors (opens in a new tab) | AWS S3 vectors |
elasticsearch (opens in a new tab) | Elasticsearch (Spice.ai Enterprise) |
duckdb (opens in a new tab) | DuckDB VSS (HNSW) extension |
:::warning[Limitations]
A dataset or view must be accelerated (i.e. datasets[].acceleration.enabled: true, see docs (opens in a new tab)) for a vector engine to be provided the appropriate data to ingest.
:::
import DocCardList from '@theme/DocCardList';