Cookbook recipes to get up and running with Spice.ai quickly 🚀
No Spice release includes the llama.cpp GGUF engine this recipe needs yet: v2.4.2 and earlier releases refuse the deepseek2 GGUF architecture of GLM-4.7-Flash and the glm5-next architecture of GLM-5.3-Flash. Until a release includes the engine, follow the build steps under Prerequisites (opens in a new tab).
Z.ai publishes its GLM models as open weights under the MIT license. The current flagships are GLM-5.3 (opens in a new tab) (744B parameters, 40B active) and GLM-5.3-Flash (opens in a new tab) (320B, 18B active), both released in August 2026. Neither fits a standard MacBook Pro: the smallest GLM-5.3-Flash build is 93 GB, a 128 GB Mac at the least. The GLM that runs on a 36 GB MacBook Pro is GLM-4.7-Flash (opens in a new tab), Z.ai's January 2026 30B mixture-of-experts reasoning model with about 3.6B active parameters and a 200K context.
This recipe serves GLM-4.7-Flash on an Apple silicon Mac with Spice's built-in llama.cpp engine and Metal, and shows how to run GLM-5.3-Flash on a 128 GB MacBook Pro by swapping the model block. Abliterated variants of both are in abliterated/ (opens in a new tab).
GLM-5.2 (June 2026) and GLM-5.3 are the 744B-class flagships: their smallest GGUFs are 217 GB and 223 GB, for a 512 GB Mac Studio rather than a MacBook Pro (a 256 GB Mac's GPU budget is about 192 GiB under the rule in Memory (opens in a new tab), below these files before the KV cache). Their glm-dsa GGUF architecture is one Spice serves, so the same spicepod shape works there, with the shards of unsloth/GLM-5.2-GGUF (opens in a new tab) or unsloth/GLM-5.3-GGUF (opens in a new tab) listed in files.
| Model | Parameters | Released | Fits a 36 GB MacBook Pro | GGUF repository |
|---|---|---|---|---|
| GLM-4.7-Flash | 30B MoE, ~3.6B active | January 2026 | Yes, from 32 GB; 24 GB at 3-bit (this recipe) | unsloth/GLM-4.7-Flash-GGUF (opens in a new tab) |
| GLM-5.3-Flash | 320B MoE, 18B active | August 2026 | No: 93 GB at 1-bit; a 128 GB Mac | unsloth/GLM-5.3-Flash-GGUF (opens in a new tab) |
| GLM-5.3 | 744B MoE, 40B active | August 2026 | No: 223 GB at 1-bit | unsloth/GLM-5.3-GGUF |
| GLM-5.2 | 753B MoE, 8 of 256 experts active | June 2026 | No: 217 GB at 1-bit | unsloth/GLM-5.2-GGUF (opens in a new tab) |
spice version must report a runtime with the +models.metal suffix and the llama.cpp GGUF engine. No release includes the engine yet, and v2.4.2 and earlier releases refuse this model. Run spice upgrade once a release that includes it ships; until then, use a trunk build (see Using a trunk build (opens in a new tab)).~/.cache/huggingface/hub.huggingface.co. No Hugging Face account is needed.Spice loads the whole GGUF into GPU memory. macOS gives the GPU most of unified memory but not all of it: Metal reports the budget as recommendedMaxWorkingSetSize, which has been two-thirds of unified memory on Macs below 64 GB and three-quarters from 64 GB up (10.7 GiB on a 16 GB Mac, 16 GiB on 24 GB, 24 GiB on 36 GB, 48 GiB on 64 GB); macOS 27 on a 128 GB M3 Max reports more, 107 GiB (84%). The rows below are sized by the smaller figure and leave at least 2 GB under it for the KV cache (under 1 GB at the engine's default 8,192-token context) and the compute buffers. A file larger than the budget fails to load or pages, so when in doubt take the row below yours. A 16 GB Mac is under the minimum for this model: its 10.7 GiB budget leaves no room for even the 2-bit files (10.5 GB and up).
| Unified memory | Model | File | Size |
|---|---|---|---|
| 24 GB | GLM-4.7-Flash | GLM-4.7-Flash-UD-Q3_K_XL.gguf | 13.8 GB |
| 32 GB | GLM-4.7-Flash | GLM-4.7-Flash-UD-Q4_K_XL.gguf (the default in spicepod.yaml) | 17.5 GB |
| 36 GB | GLM-4.7-Flash | GLM-4.7-Flash-UD-Q5_K_XL.gguf (or the default) | 21.7 GB |
| 48 GB to 64 GB | GLM-4.7-Flash | GLM-4.7-Flash-Q8_0.gguf (near-lossless; UD-Q6_K_XL, 26.2 GB, leaves more room) | 31.8 GB |
| 96 GB and up | GLM-4.7-Flash | BF16/GLM-4.7-Flash-BF16-00001-of-00002.gguf and -00002-of-00002.gguf (full precision; over a 64 GB Mac's 48 GiB budget) | 59.9 GB |
| 128 GB | GLM-5.3-Flash | UD-IQ2_XXS/GLM-5.3-Flash-UD-IQ2_XXS-00001-of-00004.gguf … -00004-of-00004.gguf (needs the 107 GiB budget of macOS 27; see GLM-5.3-Flash on a 128 GB MacBook Pro (opens in a new tab)) | 101.8 GB |
The GLM-4.7-Flash files are in unsloth/GLM-4.7-Flash-GGUF (opens in a new tab); the UD- files are Unsloth's dynamic quantizations, which keep the layers that matter most at higher precision. To switch, change files[0].path, listing every shard of a split file.
spicepod.yaml in this directory defines one model:
from names the Hugging Face repository and files the GGUF in it. A repository revision can be pinned with from: huggingface:huggingface.co/<repo>:<commit>. This recipe does not enable tools: GLM's chat template trips a runtime bug when Spice primes the conversation with its list_datasets call (spiceai/spiceai#15578 (opens in a new tab)), and GLM's tool-call format is not one Spice's parser reads.
The first start downloads the model (17.5 GB); later starts load it from cache in a few seconds.
The elided lines report settings the runtime derives for this host. The download ran on a slow, shared link; the log reports progress every 30 seconds. Wait for the deployed, ready for inferencing line before chatting: /v1/models lists the model while it is still downloading.
In a second terminal, from the same directory:
Spice serves the model at http://localhost:8090/v1/chat/completions, so OpenAI SDKs and clients work unchanged with the model name glm-4.7-flash.
GLM-4.7-Flash is a reasoning model: it thinks before it answers and returns the trace in reasoning_content, separate from content. Asked to "Reply with only the integer 1729", it spent 122 tokens (a 477-character trace) before answering; with reasoning_effort: none on the request it answered in 2 tokens. To make direct answers the default for every request, add hf_reasoning_effort: none to the model's params. On an M3 Max the model decodes about 75 tokens per second: 3.6B active parameters make it one of the fastest models of its quality on a laptop. The engine uses an 8,192-token context, and a conversation that outgrows it is cut off with an error that says so.
GLM-5.3-Flash is Z.ai's 320B multimodal mixture of experts with 18B active parameters and a 1M-token context, in the glm5-next GGUF architecture. Its 2-bit build is 101.8 GB in four shards and fits the 107 GiB GPU budget that macOS 27 reports on a 128 GB M3 Max, with room for the KV cache; an older macOS that budgets 96 GiB (103 GB) needs the UD-IQ1_M or UD-IQ1_S build listed below. The GGUF carries GLM's own multi-token-prediction (NextN) layer, and Spice drafts with it automatically: on an NVIDIA DGX Spark, drafting lifted decode speed from 17.5 to 27.1 tokens per second (spiceai/spiceai#7864 (opens in a new tab)).
Replace the models entry with:
UD-IQ1_M (97.6 GB, three shards) and UD-IQ1_S (93.1 GB, three shards) builds trade fidelity for headroom; UD-Q2_K_XL (108.7 GB, four shards) is the largest that fits, with little room to spare. Unsloth measured top-1 agreement with the full model at 71% for UD-IQ1_S, 76% for UD-IQ2_XXS and 78% for UD-Q2_K_XL.reasoning_effort low or high, and reasons at its maximum when nothing is set. hf_reasoning_effort: low makes short reasoning the default, which suits a laptop. A request for none or medium is refused with HTTP 400 and a message that names the levels the model takes.This section was run on an NVIDIA DGX Spark (128 GB of unified memory, the same memory class as a 128 GB MacBook Pro) with the same llama.cpp engine built for CUDA, the four shards checked against the Hub's sha256 and named by file: path. The model loaded in 88 to 97 seconds across three starts with 102 GB resident; reasoning_effort none and medium were refused with the messages in Troubleshooting (opens in a new tab); "Reply with only the integer 1729." took 13 tokens with a 37-character trace at the low default and at high; and decode ran at 22 tokens per second over 300 tokens (two runs: 22.3 and 21.9), with 0.4 s to the first token. Its answer to the recipe's question:
abliterated/ (opens in a new tab) is the same recipe with huihui-ai/Huihui-GLM-4.7-Flash-abliterated (opens in a new tab), a version of GLM-4.7-Flash with its refusal behavior removed by abliteration, in the importance-matrix GGUF quantization by mradermacher. Its README also names the abliterated GLM-5.3-Flash builds for a 128 GB Mac. Read it before using either: an abliterated model has none of the base model's safety training.
| Symptom | Cause and fix |
|---|---|
Architecture 'deepseek2' in <file> is not supported by this Spice release (or glm5-next) | The runtime predates the llama.cpp GGUF engine. Upgrade Spice, or run a trunk build (see below). |
Unknown GGUF architecture 'deepseek2' (or glm5-next) | A v2.4.2 or earlier runtime, which has no llama.cpp GGUF engine. The runtime then never reports ready (spiceai/spiceai#15744 (opens in a new tab)). Run a trunk build (see below). |
Model 'glm-5.3-flash' always reasons, so reasoning_effort: none can't turn reasoning off | GLM-5.3-Flash's template has no off switch. Use low for the least reasoning. |
Adding tools: auto makes every request fail with cannot convert value into pairs | This model's chat template iterates tool-call arguments, and the list_datasets call Spice primes the conversation with has empty arguments (spiceai/spiceai#15578 (opens in a new tab)). Leave tools off until it is fixed. |
| The model never finishes loading, or macOS becomes unresponsive | The file does not fit the GPU's share of memory. Choose a smaller file from the Memory (opens in a new tab) table. |
the prompt is N tokens, and the model's context holds 8192 | The conversation outgrew the context. Start a new chat. |
Until a release includes the llama.cpp GGUF engine, download the latest trunk build of the runtime and point the CLI at it. The CLI's spice run, spice chat and spice sql work unchanged. The download needs the GitHub CLI (opens in a new tab) signed in to a GitHub account (brew install gh, then gh auth login), because Actions artifacts are only served to authenticated users.
Stop Spice with Ctrl+C. The model stays in the Hugging Face cache for the next start; remove it with: