docs: publish v2.2 as latest (#2111)
2 months ago
docs: publish v2.2 as latest (#2111)
2 months ago
docs: quote placeholders in shell examples so the blocks paste safely
An unquoted <placeholder> inside a shell-tagged fence is redirection from
a file of that name, so a copied command aborts (exit 1), fails to parse
(exit 2), or — where it sits mid-block — is silently skipped while the
block still reports exit 0. Quote the placeholder, which keeps it visible
as a placeholder and passes it through verbatim.
Usage synopses are left alone: where a fence carries [...] optional-argument
notation, or a placeholder fills a command slot, the <> is grammar for the
command's shape rather than a command to paste.
Fixes #2196
a month ago
docs: publish v2.2 as latest (#2111)
2 months ago
docs(models): paged attention is requested automatically, not unconditionally enabled
The local-model paged-attention note collapsed three branches into one
absolute sentence. `MistralLlama::paged_attention_config` returns
`None` when the device is CPU or when the build is not
`all(feature = "cuda", target_family = "unix")`, and the engine
downgrades to dense attention again for architectures with no paged
kernel (the Multi-head Latent Attention GGUFs) — so a CUDA+Unix build
is necessary, not sufficient.
From v2.2.0 (spiceai/spiceai#11990) the note is also wrong to say no
configuration is needed: `FileModelParams::paged_attention` accepts
`auto` (default) or `disabled`, and `disabled` forces dense
attention. It is a `from: file:` parameter only —
`HuggingFaceModelParams` has no such field and `from_hf` calls
`paged_attention_config` unconditionally.
Scoped accordingly: the branch distinction lands in v2.0.x-v2.3.x and
vNext; the escape hatch only in v2.2.x, v2.3.x and vNext.
a month ago
docs: publish v2.2 as latest (#2111)
2 months ago
docs(huggingface): the v2.2.x model token is huggingface_token, and bare token never worked
a month ago
docs: publish v2.2 as latest (#2111)
2 months ago