Workers in the Spice runtime represent configurable units of compute that help coordinate and manage interactions between models and tools. Each worker is defined as a component in the spicepod.yaml file, specifying its behavior and interaction logic.
Workers are configured in the workers section of the spicepod.yaml file. Each worker definition includes a name, description, and a list of models or tools it encapsulates.
Example spicepod.yaml configuration:
Workers currently help implement:
Workers can be invoked using the same API endpoints as individual models. For example, to call a worker named fallback using the OpenAI-compatible HTTP API:
The vision for workers includes support for dynamic serverless compute, enabling execution of user-defined functions within the Spice runtime. This direction aims to help developers define custom logic and orchestration patterns directly in the worker configuration, supporting more advanced workflows and automation. Further details and implementation timelines will be provided in future updates. For ongoing progress, refer to the project repository and documentation.
For a complete specification of worker configuration, routing rules, and available options, refer to the Spicepod Workers Reference.
