Cookbook recipes to get up and running with Spice.ai quickly 🚀
No Spice release includes the llama.cpp GGUF engine this recipe needs yet: v2.4.2 and earlier releases refuse this model's deepseek4 GGUF architecture. Until a release includes the engine, follow the build steps under Prerequisites (opens in a new tab).
DeepSeek-V4 is DeepSeek's 2026 generation of open-weight mixture-of-experts models under the MIT license: DeepSeek-V4-Pro (1.6T parameters, 49B active) and DeepSeek-V4-Flash (284B, 13B active), previewed in April 2026, with DeepSeek-V4-Flash-0731 (opens in a new tab) as the official Flash release on July 31. Both are 1M-context reasoning models with low, high and max reasoning levels.
No DeepSeek-V4 build fits a 36 GB MacBook Pro. The smallest GGUF of DeepSeek-V4-Flash is 82.5 GB at 1-bit, and DeepSeek publishes no smaller current-generation model (its last small models were the January 2025 R1 distillations). This recipe therefore targets a 128 GB MacBook Pro (M3 Max or M4 Max), the smallest Mac that runs the model, and serves DeepSeek-V4-Flash-0731 with Spice's built-in llama.cpp engine and Metal. An abliterated variant (opens in a new tab) is in abliterated/ (opens in a new tab).
| Model | Parameters | Released | Smallest GGUF | Fits |
|---|---|---|---|---|
| DeepSeek-V4-Flash-0731 | 284B MoE, 13B active | July 2026 | 82.5 GB | A 128 GB Mac (this recipe) |
| DeepSeek-V4-Pro-0813 | 1.6T MoE, 49B active | August 2026 | 400 GB-class | No Mac |
| DeepSeek-V4.1-Flash-1017 | 552B + 196B Engram | October 2026 | 233 GB | No MacBook Pro; no recipe yet |
recommendedMaxWorkingSetSize), which holds the default 90.9 GB file with room for the KV cache.spice version must report a runtime with the +models.metal suffix and the llama.cpp GGUF engine. No release includes the engine yet, and v2.4.2 and earlier releases refuse this model. Run spice upgrade once a release that includes it ships; until then, use a trunk build (see Using a trunk build (opens in a new tab)).~/.cache/huggingface/hub. The download takes about two hours at 12 MB/s.huggingface.co. No Hugging Face account is needed.| Unified memory | File prefix (UD-<quant>/DeepSeek-V4-Flash-0731-UD-<quant>-0000N-of-0000M.gguf) | Shards | Size |
|---|---|---|---|
| 128 GB | UD-IQ1_S | 3 | 82.5 GB |
| 128 GB | UD-IQ1_M | 3 | 86.9 GB |
| 128 GB | UD-IQ2_M (the default in spicepod.yaml) | 3 | 90.9 GB |
| 128 GB | UD-Q2_K_XL | 3 | 96.8 GB |
| 128 GB, little headroom | UD-IQ3_XXS | 4 | 104.2 GB |
| 192 GB and up | UD-Q3_K_XL (128.2 GB), UD-IQ4_XS (136.7 GB), UD-Q4_K_XL (155.1 GB) | 4 to 5 |
All files are in unsloth/DeepSeek-V4-Flash-0731-GGUF (opens in a new tab); the UD- prefix marks Unsloth's dynamic quantizations, which keep the layers that matter most at higher precision. List every shard of the chosen build in files. The repository's dspark-*.gguf files are DeepSeek's DSpark speculative-decoding head for other engines; Spice reads only draft heads named mtp-*.gguf, so leave them out.
spicepod.yaml in this directory defines one model:
from names the Hugging Face repository and files the shards of the GGUF. A repository revision can be pinned with from: huggingface:huggingface.co/<repo>:<commit>. This recipe does not enable tools: DeepSeek's chat template trips a runtime bug when Spice primes the conversation with its list_datasets call (spiceai/spiceai#15578 (opens in a new tab)).
The first start downloads the three shards (90.9 GB); later starts load them from cache. Loading 91 GB of weights takes about a minute and a half:
These lines are from a run of this spicepod, as written, on an NVIDIA DGX Spark (128 GB of unified memory, the same memory class as a 128 GB MacBook Pro) with the same llama.cpp engine built for CUDA and a 130 MB/s connection; on a Mac the runtime reports +models.metal. Wait for the deployed, ready for inferencing line before chatting: /v1/models lists the model while it is still downloading. With the model loaded, 91 GB of the machine's 121 GB were in use.
In a second terminal, from the same directory:
Spice serves the model at http://localhost:8090/v1/chat/completions, so OpenAI SDKs and clients work unchanged with the model name deepseek-v4-flash.
With Unsloth's chat template, DeepSeek-V4-Flash answers directly unless a request asks it to think. "Reply with only the integer 1729." came back in 2 tokens with nothing set and with reasoning_effort: none; low and medium each gave an 88-token answer with a 318-character trace, and high a 13-token answer with a short one. The trace arrives in reasoning_content, separate from content. The field takes the OpenAI levels only (none, minimal, low, medium, high, xhigh). DeepSeek's own "max" is not one of them: the request is refused with HTTP 422 and reasoning_effort: unknown variant max``, before it reaches the model. To set a default for every request, add hf_reasoning_effort: <level> to the model's params. DeepSeek recommends temperature 1.0 and top-p 1.0 (0.95 for agentic use). On the DGX Spark the model decoded about 21 tokens per second; a 128 GB MacBook Pro has comparable memory bandwidth. The engine uses an 8,192-token context, and a conversation that outgrows it is cut off with an error that says so.
abliterated/ (opens in a new tab) is the same recipe with huihui-ai/Huihui-DeepSeek-V4-Flash-0731-abliterated (opens in a new tab), a version of DeepSeek-V4-Flash-0731 with its refusal behavior removed by abliteration, as a single 86.7 GB 2-bit GGUF. Read its README before using it: an abliterated model has none of the base model's safety training.
| Symptom | Cause and fix |
|---|---|
Architecture 'deepseek4' in <file> is not supported by this Spice release | The runtime predates the llama.cpp GGUF engine. Upgrade Spice, or run a trunk build (see below). |
GGUF architecture deepseek4 (DeepSeek-V4) is recognized but not implemented | A v2.4.2 or earlier runtime, which has no llama.cpp GGUF engine. The runtime then never reports ready (spiceai/spiceai#15744 (opens in a new tab)). Run a trunk build (see below). |
Adding tools: auto makes every request fail with cannot convert value into pairs | This model's chat template iterates tool-call arguments, and the list_datasets call Spice primes the conversation with has empty arguments (spiceai/spiceai#15578 (opens in a new tab)). Leave tools off until it is fixed. |
| The model never finishes loading, or macOS becomes unresponsive | The build does not fit the GPU's share of memory. Choose a smaller build from the Memory (opens in a new tab) table, or close other apps. |
A dspark-*.gguf file listed in files fails the load | It is not a shard of the model. Remove it. |
the prompt is N tokens, and the model's context holds 8192 |
Until a release includes the llama.cpp GGUF engine, download the latest trunk build of the runtime and point the CLI at it. The CLI's spice run, spice chat and spice sql work unchanged. The download needs the GitHub CLI (opens in a new tab) signed in to a GitHub account (brew install gh, then gh auth login), because Actions artifacts are only served to authenticated users.
Stop Spice with Ctrl+C. The model stays in the Hugging Face cache for the next start; remove it with:
| The conversation outgrew the context. Start a new chat. |