The GraphQL (opens in a new tab) Data Connector enables federated SQL queries on any GraphQL endpoint by specifying graphql as the selector in the from value for the dataset.
:::warning[Limitations]
LIMIT, is not currently supported. Using a LIMIT will reduce the amount of data requested from the GraphQL server.:::
fromThe from field takes the form of graphql:your-graphql-endpoint.
nameThe dataset name. This will be used as the table name within Spice. The dataset name cannot be a reserved keyword (opens in a new tab).
paramsThe GraphQL data connector can be configured by providing the following params. Use the secret replacement syntax (opens in a new tab) to load the password from a secret store, e.g. ${secrets:my_graphql_auth_token}.
| Parameter Name | Description | Required | Default |
|---|---|---|---|
graphql_query | The GraphQL query to execute. See examples (opens in a new tab) for a sample GraphQL query. | Yes | - |
json_pointer | The JSON pointer (opens in a new tab) into the response body. When graphql_query is paginated (opens in a new tab), the json_pointer can be inferred. | No | - |
unnest_depth | Depth level to automatically unnest objects to. Disabled if unspecified or 0. Maximum value is 50. | No | 0 |
graphql_auth_header | A custom header name to use for authentication instead of the default Authorization: Bearer header. When set, the value of graphql_auth_token is sent as the value of this header. Useful for APIs that require authentication via a custom header (e.g. ). |
Example using the GitHub GraphQL API and Bearer Auth. The following will use json_pointer to retrieve all of the nodes in starredRepositories:
Some APIs require authentication via a custom header instead of the standard Authorization: Bearer header. Use the graphql_auth_header parameter to specify a custom header name:
graphql_auth_header | graphql_auth_token | graphql_auth_user/graphql_auth_pass | Result |
|---|---|---|---|
| set | set | - | Custom header: X-Custom: <token> |
| not set | set | - | Default: Authorization: Bearer <token> |
| not set | not set | set | HTTP Basic Auth |
| not set | not set | not set | No auth |
The GraphQL connector supports shared HTTP rate control to limit concurrency and request rate per upstream origin. These parameters can be set per-dataset (in params) or globally (in runtime.params). Dataset-level settings override the global defaults.
| Parameter Name | Description |
|---|---|
max_concurrent_requests | Maximum number of concurrent HTTP requests to the same upstream origin. Overrides runtime.params.http_max_concurrent_requests. If both are unset, concurrency limiting is disabled. |
requests_per_second_limit | Maximum number of HTTP requests per second to the same upstream origin. Overrides runtime.params.http_requests_per_second_limit. If both are unset, no per-second rate limit is applied. |
requests_per_minute_limit | Maximum number of HTTP requests per minute to the same upstream origin. Overrides runtime.params.http_requests_per_minute_limit. If both are unset, no per-minute rate limit is applied. |
rate_control_jitter_min | Minimum random delay added before HTTP requests when rate control is active. Accepts durations such as 5ms or 0ms. Defaults to 5ms when a request-rate limit is configured. |
rate_control_jitter_max | Maximum random delay added before HTTP requests when rate control is active. Accepts durations such as 10ms or 0ms. Defaults to when a request-rate limit is configured. |
Multiple datasets targeting the same GraphQL endpoint share the same rate controller.
The GraphQL Data Connector supports automatic pagination of the response for queries using cursor pagination (opens in a new tab).
The graphql_query must include the pageInfo field as per spec (opens in a new tab). The connector will parse the graphql_query, and when pageInfo is present, will retrieve data until pagination completes.
The query must have the correct pagination arguments in the associated paginated field.
Forward Pagination:
Backward Pagination:
Tips for working with JSON data. For more information see Datafusion Docs (opens in a new tab).
You can access the fields of the object using the square bracket notation. Arrays are indexed from 1.
Example for the stargazers query from pagination section (opens in a new tab):
You can use Datafusion unnest function to pipe values from array into rows.
We'll be using countries GraphQL api (opens in a new tab) as an example.
Example query:
You can also use the unnest_depth parameter to control automatic unnesting of objects from GraphQL responses.
This examples uses the GitHub stargazers endpoint:
If unnest_depth is set to 0, or unspecified, object unnesting is disabled. When enabled, unnesting automatically moves nested fields to the parent level.
Without unnesting, stargazers data looks like this in a query:
With unnesting, these properties are automatically placed into their own columns:
By default, the Spice Runtime will error when a duplicate column is detected during unnesting.
For example, this example spicepod.yml query would fail due to name fields:
This example would fail with a runtime error:
Avoid this error by using aliases in the query (opens in a new tab) where possible. In the example above, a duplicate error was introduced from emergency_contact { name }.
The example below uses a GraphQL alias to rename emergency_contact.name as emergencyContactName.
X-Shopify-Access-Token| No |
| - |
graphql_auth_token | The authentication token to use to connect to the GraphQL server. Uses bearer authentication by default, or sent via the custom header specified by graphql_auth_header. | No | - |
graphql_auth_user | The username to use for basic auth. E.g. graphql_auth_user: my_user | No | - |
graphql_auth_pass | The password to use for basic auth. E.g. graphql_auth_pass: ${secrets:my_graphql_auth_pass} | No | - |
10ms