title: 'HuggingFace Text Embedding Models' sidebar_label: 'HuggingFace' sidebar_position: 3

To use an embedding model from HuggingFace with Spice, specify the huggingface path in the from field of your configuration. The model and its related files will be automatically downloaded, loaded, and served locally by Spice.

The following parameters are specific to HuggingFace models:

Here is an example configuration in spicepod.yaml:

embeddings:
  - from: huggingface:huggingface.co/sentence-transformers/all-MiniLM-L6-v2
    name: all_minilm_l6_v2

embeddings:
  - from: huggingface:huggingface.co/sentence-transformers/all-MiniLM-L6-v2
    name: all_minilm_l6_v2

Supported models include:

• All models tagged as text-embeddings-inference (opens in a new tab) on Huggingface
• Any Huggingface repository with the correct files to be loaded as a local embedding model (opens in a new tab).

Model Weights

Spice downloads the repository's safetensors weights. When a repository publishes none, it falls back to pytorch_model.bin and logs a warning at startup:

safetensors weights not found; falling back to `pytorch_model.bin`. Model loading is significantly slower.

safetensors weights not found; falling back to `pytorch_model.bin`. Model loading is significantly slower.

A repository that publishes neither format fails to load. Prefer a repository with safetensors weights where one is available.

With the same semantics as language models (opens in a new tab), spice can run private HuggingFace embedding models:

embeddings:
  - from: huggingface:huggingface.co/secret-company/awesome-embedding-model
    name: top_secret
    params:
      hf_token: ${ secrets:HF_TOKEN }

embeddings:
  - from: huggingface:huggingface.co/secret-company/awesome-embedding-model
    name: top_secret
    params:
      hf_token: ${ secrets:HF_TOKEN }