title: 'Embedding Models' sidebar_label: 'Embeddings' description: '' image: /img/og/embeddings.png sidebar_position: 6 pagination_prev: null pagination_next: null tags:
Embedding models convert raw text into numerical representations that can be used by machine learning models. Spice supports running embedding models locally or using remote services such as OpenAI or la Plateforme.
Embeddings are used for vector-based and similarity search, like document retrieval. For chat-based large language models, refer to Model Providers.
Spice supports various model sources and formats to provide embedding components:
| Name | Description | Status | ML Format(s) | LLM Format(s)* |
|---|---|---|---|---|
file | Local filesystem | Release Candidate | ONNX | GGUF, GGML, SafeTensor |
huggingface | Models hosted on HuggingFace | Release Candidate | ONNX | GGUF, GGML, SafeTensor |
openai | OpenAI (or compatible) LLM endpoint | Release Candidate | - | OpenAI-compatible HTTP endpoint |
azure | Azure OpenAI | Alpha | - | OpenAI-compatible HTTP endpoint |
databricks | Models deployed to Databricks Mosaic AI | Alpha | - | OpenAI-compatible HTTP endpoint |
bedrock | Models deployed on AWS Bedrock | Alpha | - | OpenAI-compatible HTTP endpoint |
Spice provides three distinct methods for handling embedding columns in datasets:
Embedding models are defined in the spicepod.yaml file as top-level components.
Example configuration in spicepod.yaml:
Embedding models can be used either by:
Embedding models can be configured to create vector embeddings for specific columns in a dataset. Define embeddings under columns in the spicepod.yaml file, under the datasets section.
Example configuration in spicepod.yaml:
Refer to the embeddings and datasets Spicepod reference for more details on configuring embeddings for datasets.
JIT embeddings are computed during query execution. This is useful when pre-computing embeddings is infeasible (e.g. if the dataset is large, infrequently queried, has heavy prefiltering). To add an embedding column, specify it within the dataset's column.
To improve query performance, column embeddings can be precomputed, and stored in any data accelerator. The only change required for this it to set up the data accelerator. For example, just add
to the dataset configuration. All other data accelerator configurations are optional, but can be applied as per their respective documentation.
Full example:
Datasets that already have embedding columns can utilize the same functionalities (e.g. vector search) as those augmented with Spice-generated embeddings. They should follow the same schema as Spice-generated embeddings (or be altered with a view.
A sales table with an address column that has an embedding.
The same table if it was chunked:
Passthrough embedding columns must still be defined in the spicepod.yaml file. The spice instance must also have access to the same embedding model used to generate the embeddings.
To ensure compatibility, these table columns must adhere to the following constraints:
Underlying Column Presence:
string Arrow data type .Embeddings Column Naming Convention:
<column_name>_embedding. For example, a customer_reviews table with a review column must have a review_embedding column.Embeddings Column Data Type:
FixedSizeList[Float32 or Float64, N], where N is the dimension (size) of the embedding vector. FixedSizeList is used for efficient storage and processing of fixed-size vectors.List[FixedSizeList[Float32 or Float64, N]].Offset Column for Chunked Data:
<column_name>_offsets with the following Arrow data type:
Following these guidelines ensures that the dataset with pre-existing embeddings is fully compatible with embedding functionalities provided by Spice.
Spice also supports chunking of content before embedding, which is useful for large text columns such as those found in Document Tables. Chunking ensures that only the most relevant portions of text are returned during search queries. Chunking is configured as part of the embedding configuration.
The body column will be divided into chunks of approximately 512 tokens, while maintaining structural and semantic integrity (e.g. not splitting sentences). See the API reference for full details.
Like a primary key, the row_id field specifies which column(s) uniquely identifies each row. This is useful for embedding datasets that don't have a primary key by default. This is important for chunked embedding datasets, so that operations (e.g. v1/search), can be able to map multiple chunked vectors to a single dataset row. The row_id can be set in the columns[*].embeddings[*].row_id.
import DocCardList from '@theme/DocCardList';
List[FixedSizeList[Int32, 2]], where each element is a pair of integers [start, end] representing the start and end indices of the chunk in the underlying text column. This offset column maps each chunk in the embeddings back to the corresponding segment in the underlying text column.[[0, 100], [101, 200]] indicates two chunks covering indices 0–100 and 101–200, respectively.