The GraphQL (opens in a new tab) Data Connector enables federated SQL queries on any GraphQL endpoint by specifying graphql as the selector in the from value for the dataset.
:::warning[Limitations]
LIMIT, is not currently supported. Using a LIMIT will reduce the amount of data requested from the GraphQL server.:::
fromThe from field takes the form of graphql:your-graphql-endpoint.
nameThe dataset name. This will be used as the table name within Spice. The dataset name cannot be a reserved keyword (opens in a new tab).
paramsThe GraphQL data connector can be configured by providing the following params. Use the secret replacement syntax (opens in a new tab) to load the password from a secret store, e.g. ${secrets:my_graphql_auth_token}.
| Parameter Name | Description |
|---|---|
unnest_depth | Depth level to automatically unnest objects to. By default, disabled if unspecified or 0. |
graphql_auth_token | The authentication token to use to connect to the GraphQL server. Uses bearer authentication. |
graphql_auth_user | The username to use for basic auth. E.g. graphql_auth_user: my_user |
graphql_auth_pass | The password to use for basic auth. E.g. graphql_auth_pass: ${secrets:my_graphql_auth_pass} |
graphql_query | The GraphQL query to execute. See examples (opens in a new tab) for a sample GraphQL query |
json_pointer | The JSON pointer (opens in a new tab) into the response body. When graphql_query is paginated (opens in a new tab), the json_pointer can be inferred. |
Example using the GitHub GraphQL API and Bearer Auth. The following will use json_pointer to retrieve all of the nodes in starredRepositories:
The GraphQL Data Connector supports automatic pagination of the response for queries using cursor pagination (opens in a new tab).
The graphql_query must include the pageInfo field as per spec (opens in a new tab). The connector will parse the graphql_query, and when pageInfo is present, will retrieve data until pagination completes.
The query must have the correct pagination arguments in the associated paginated field.
Forward Pagination:
Backward Pagination:
Tips for working with JSON data. For more information see Datafusion Docs (opens in a new tab).
You can access the fields of the object using the square bracket notation. Arrays are indexed from 1.
Example for the stargazers query from pagination section (opens in a new tab):
You can use Datafusion unnest function to pipe values from array into rows.
We'll be using countries GraphQL api (opens in a new tab) as an example.
Example query:
You can also use the unnest_depth parameter to control automatic unnesting of objects from GraphQL responses.
This examples uses the GitHub stargazers endpoint:
If unnest_depth is set to 0, or unspecified, object unnesting is disabled. When enabled, unnesting automatically moves nested fields to the parent level.
Without unnesting, stargazers data looks like this in a query:
With unnesting, these properties are automatically placed into their own columns:
By default, the Spice Runtime will error when a duplicate column is detected during unnesting.
For example, this example spicepod.yml query would fail due to name fields:
This example would fail with a runtime error:
Avoid this error by using aliases in the query (opens in a new tab) where possible. In the example above, a duplicate error was introduced from emergency_contact { name }.
The example below uses a GraphQL alias to rename emergency_contact.name as emergencyContactName.