docs: keyed Iceberg appends on other engines read by time, partitioned Cayenne snapshots, Cayenne in the Cluster-Sidecar example (#2442)
10 days ago
docs: per-dataset snapshot settings and delegation behavior in the cluster-sidecar guide; reranker, schema evolution, and refresh processed metrics (#2472)
* docs: per-dataset snapshot settings and delegation behavior in the cluster-sidecar guide; reranker, schema evolution, and refresh processed metrics
* docs: say that the zero-results fallback skips the sidecar's refresh_sql filter
---------
Co-authored-by: Emma Spice <emma@EmmasMacStudio.localdomain>
5 days ago
Add cluster-sidecar hybrid architecture (#1387)
* Refactor deployment architecture documentation to use MDX components for better modularity and maintainability
- Updated microservice, sharded, sidecar, tiered, cluster, hosted, and hybrid architecture documentation to import content from MDX partials.
- Removed inline content in favor of centralized MDX files for each architecture type.
- Added new hybrid architecture documentation to support sidecar caching backed by a centralized cluster.
* Update image formatting in deployment architecture documentation for consistency
8 months ago
Add cluster-sidecar hybrid architecture (#1387)
* Refactor deployment architecture documentation to use MDX components for better modularity and maintainability
- Updated microservice, sharded, sidecar, tiered, cluster, hosted, and hybrid architecture documentation to import content from MDX partials.
- Removed inline content in favor of centralized MDX files for each architecture type.
- Added new hybrid architecture documentation to support sidecar caching backed by a centralized cluster.
* Update image formatting in deployment architecture documentation for consistency
8 months ago
docs: cluster-sidecar transport security, Spice Cloud params, append time_column, snapshot egress (#2420)
* docs: fix cluster-sidecar transport security claim and example spicepods
The cluster-sidecar page said sidecar-to-cluster traffic uses mTLS by
default. The Spice.ai connector uses TLS for https:// and grpc+tls://
endpoints, accepts http:// without TLS, and presents a client
certificate only when the spiceai_tls_client_* parameters are set.
The sidecar example read from Spice Cloud without spiceai_api_key or
spiceai_region, which the connector requires for Cloud endpoints. The
cluster example used refresh_mode: append on an S3 dataset with no
time_column or primary_key, which the accelerated-table builder rejects.
The network-surface guidance now names the snapshot bucket the sidecar
example bootstraps from.
* docs: keep the v1.11 hybrid page on its original text
The cluster-sidecar corrections cite spiceai_tls_client_* and spiceai_region, which v1.11.6's Spice.ai connector does not have, so the v1.11-only _hybrid partial keeps the pre-correction content instead of importing the shared partial.
---------
Co-authored-by: Emma Spice <emma@EmmasMacStudio.localdomain>
Co-authored-by: Luke Kim <80174+lukekim@users.noreply.github.com>
15 days ago
docs: Sync the cookbook page and recipe links with spiceai/cookbook (#2318)
* Sync the cookbook page with the current spiceai/cookbook recipes
List all 122 current recipes (up from 79) and leave out evals, llm-judge, and websearch, which the cookbook marks deprecated in v2.0+.
- Correct entries that pointed at the wrong recipe or described an old version: Results Caching linked the Caching Accelerator recipe, Nvidia NIM on Kubernetes linked the EC2 recipe, Hashed Partitioning now uses Cayenne, and Federated SQL Query no longer uses Dremio.
- Add the missing recipes, including Cayenne, async queries, native PostgreSQL/MySQL/MongoDB CDC, DynamoDB Streams, HTTP, Redshift, ScyllaDB, SMB, the catalog connectors, full-text search, distributed query, and Spice as an MCP server.
- Add Change Data Capture and Advanced Topics sections. Local Dataset Replication was tagged for no rendered section, so it never appeared.
- Show guides in Sample Applications and Guides instead of Security.
- Align Core Features with the cookbook's core scenarios and feature Cayenne as the data accelerator.
- Label recipes that need Spice.ai Enterprise.
- Remove forbidden words (enhance, leverage, utilize, robust) from section intros.
* Fix broken and stale cookbook recipe references across doc versions
Every version links to the recipes on cookbook trunk, so these corrections apply to each version that carries them.
- Query Federation: replace the walkthrough with the current Federated SQL Query recipe, which joins NYC taxi trips in S3 with taxi zones in PostgreSQL. The old walkthrough logged in to a demo Dremio, ran 'spice add spiceai/fed-demo' (that Spicepod no longer exists on Spicerack), and ran 'cd demo' twice. The new steps and output come from a run of the recipe on Spice v2.3.2.
- FAQ: the RAG recipe link (cookbook/tree/trunk/rag) never existed; link the RAG use-case page instead.
- Use cases: fix duckdb/accelerator/README and federation/README links that were missing the .md extension.
- PostgreSQL connector: link the connector recipe instead of the accelerator recipe, and fix 'configure Supabase a data connector'.
- ODBC connector: the recipe now connects to MySQL with the MariaDB driver, not Postgres.
- Glue catalog connector: its recipe configures the Glue Catalog Connector, not the Data Connector.
- Release notes: fix three recipe links that pointed at moved or malformed paths.
* Link docs pages to their cookbook recipes
49 cookbook recipes had no link from any docs page. Add a Cookbook section, or a bullet in an existing one, on the matching page in vNext and 2.3.x. Covered pages include CDC (PostgreSQL, MySQL, Aurora MySQL, MongoDB), the catalog connectors (DuckLake, MySQL, SQL Server, Unity Catalog, Databricks, Iceberg Hadoop, PostgreSQL CDC), acceleration (partitioning, snapshots, indexes, constraints, views, caching, cron refresh, dual-dataset registration), search, distributed query and async queries, MCP, LLM memory, text-to-SQL, the ai() function, models (OpenAI, DeepSeek, filesystem, xAI, NVIDIA NIM on Kubernetes), API keys, ADBC, JDBC, the SDKs, and Cloud Connect. The sidecar and microservice architecture partials, which every version shares, link their recipes too.
Related updates:
- The docs landing page lists Cayenne among the accelerators and links the Cayenne recipe.
- The Cayenne Transactions intro also covers tables configured for durable write-back, matching the section's own requirements.
- The FAQ says the cookbook has over 120 recipes, up from over 65.
docs: Comprehensive partitioning, partition transforms, and pruning rules
Rewrite the partitioning page with full coverage of supported
partition transforms (bucket, truncate, date_part, date_trunc,
modulo), composite partitioning, pruning matrix per filter shape,
engine-specific behavior (Arrow / DuckDB files-mode / DuckDB tables-mode
/ Cayenne), validation rules, and worked examples.
Verified against the Rust source: bucket return type matches the
num_buckets literal (Int8..Int64 / UInt8..UInt64) — not always Int64
as previously documented in scalar_functions.md. Truncate accepts
Utf8 and Binary in addition to numeric types. date_part filter
pruning is not yet implemented; equality on the partition expression
still prunes correctly.
Also:
- Cross-link from data-acceleration index to the new partitioning page
- Distinguish deployment-level sharding from acceleration partitioning
in the sharded architecture page
- Correct bucket return type and truncate input type list in scalar_functions.md
6 months ago
docs: Sync the cookbook page and recipe links with spiceai/cookbook (#2318)
* Sync the cookbook page with the current spiceai/cookbook recipes
List all 122 current recipes (up from 79) and leave out evals, llm-judge, and websearch, which the cookbook marks deprecated in v2.0+.
- Correct entries that pointed at the wrong recipe or described an old version: Results Caching linked the Caching Accelerator recipe, Nvidia NIM on Kubernetes linked the EC2 recipe, Hashed Partitioning now uses Cayenne, and Federated SQL Query no longer uses Dremio.
- Add the missing recipes, including Cayenne, async queries, native PostgreSQL/MySQL/MongoDB CDC, DynamoDB Streams, HTTP, Redshift, ScyllaDB, SMB, the catalog connectors, full-text search, distributed query, and Spice as an MCP server.
- Add Change Data Capture and Advanced Topics sections. Local Dataset Replication was tagged for no rendered section, so it never appeared.
- Show guides in Sample Applications and Guides instead of Security.
- Align Core Features with the cookbook's core scenarios and feature Cayenne as the data accelerator.
- Label recipes that need Spice.ai Enterprise.
- Remove forbidden words (enhance, leverage, utilize, robust) from section intros.
* Fix broken and stale cookbook recipe references across doc versions
Every version links to the recipes on cookbook trunk, so these corrections apply to each version that carries them.
- Query Federation: replace the walkthrough with the current Federated SQL Query recipe, which joins NYC taxi trips in S3 with taxi zones in PostgreSQL. The old walkthrough logged in to a demo Dremio, ran 'spice add spiceai/fed-demo' (that Spicepod no longer exists on Spicerack), and ran 'cd demo' twice. The new steps and output come from a run of the recipe on Spice v2.3.2.
- FAQ: the RAG recipe link (cookbook/tree/trunk/rag) never existed; link the RAG use-case page instead.
- Use cases: fix duckdb/accelerator/README and federation/README links that were missing the .md extension.
- PostgreSQL connector: link the connector recipe instead of the accelerator recipe, and fix 'configure Supabase a data connector'.
- ODBC connector: the recipe now connects to MySQL with the MariaDB driver, not Postgres.
- Glue catalog connector: its recipe configures the Glue Catalog Connector, not the Data Connector.
- Release notes: fix three recipe links that pointed at moved or malformed paths.
* Link docs pages to their cookbook recipes
49 cookbook recipes had no link from any docs page. Add a Cookbook section, or a bullet in an existing one, on the matching page in vNext and 2.3.x. Covered pages include CDC (PostgreSQL, MySQL, Aurora MySQL, MongoDB), the catalog connectors (DuckLake, MySQL, SQL Server, Unity Catalog, Databricks, Iceberg Hadoop, PostgreSQL CDC), acceleration (partitioning, snapshots, indexes, constraints, views, caching, cron refresh, dual-dataset registration), search, distributed query and async queries, MCP, LLM memory, text-to-SQL, the ai() function, models (OpenAI, DeepSeek, filesystem, xAI, NVIDIA NIM on Kubernetes), API keys, ADBC, JDBC, the SDKs, and Cloud Connect. The sidecar and microservice architecture partials, which every version shares, link their recipes too.
Related updates:
- The docs landing page lists Cayenne among the accelerators and links the Cayenne recipe.
- The Cayenne Transactions intro also covers tables configured for durable write-back, matching the section's own requirements.
- The FAQ says the cookbook has over 120 recipes, up from over 65.
Add cluster-sidecar hybrid architecture (#1387)
* Refactor deployment architecture documentation to use MDX components for better modularity and maintainability
- Updated microservice, sharded, sidecar, tiered, cluster, hosted, and hybrid architecture documentation to import content from MDX partials.
- Removed inline content in favor of centralized MDX files for each architecture type.
- Added new hybrid architecture documentation to support sidecar caching backed by a centralized cluster.
* Update image formatting in deployment architecture documentation for consistency
8 months ago