FTP (File Transfer Protocol) and SFTP (SSH File Transfer Protocol) are network protocols used for transferring files between a client and server, with FTP being less secure and SFTP providing encrypted file transfer over SSH.
The FTP/SFTP Data Connector enables federated/accelerated SQL query across supported file formats (opens in a new tab) stored in FTP/SFTP servers.
fromThe from field takes one of two forms: ftp://<host>/<path> or sftp://<host>/<path> where <host> is the host to connect to and <path> is the path to the file or directory to read from.
If a folder is provided, all child files will be loaded.
nameThe dataset name. This will be used as the table name within Spice.
Example:
The dataset name cannot be a reserved keyword (opens in a new tab).
params| Parameter Name | Description |
|---|---|
file_format | Specifies the data file format. Required if the format cannot be inferred by from the from path. See Object Store File Formats (opens in a new tab). |
ftp_port | Optional, specifies the port of the FTP server. Default is 21. E.g. ftp_port: 21 |
ftp_user | Required. The username for the FTP server. E.g. ftp_user: my-ftp-user |
ftp_pass | Required. The password for the FTP server. Use the secret replacement syntax (opens in a new tab) to load the password from a secret store, e.g. ${secrets:my_ftp_pass}. |
client_timeout | Optional. Specifies timeout for FTP connection. E.g. client_timeout: 30s. When not set, no timeout will be configured for FTP client. |
hive_partitioning_enabled | Optional. Enable partitioning using hive-style partitioning from the folder structure. Defaults to |
| Parameter Name | Description |
|---|---|
file_format | Specifies the data file format. Required if the format cannot be inferred by from the from path. See Object Store File Formats (opens in a new tab). |
sftp_port | Optional, specifies the port of the SFTP server. Default is 22. E.g. sftp_port: 22 |
sftp_user | Required. The username for the SFTP server. E.g. sftp_user: my-sftp-user |
sftp_pass | Required. The password for the SFTP server. Use the secret replacement syntax (opens in a new tab) to load the password from a secret store, e.g. ${secrets:my_sftp_pass}. |
client_timeout | Optional. Specifies timeout for SFTP connection. E.g. client_timeout: 30s. When not set, no timeout will be configured for SFTP client. |
hive_partitioning_enabled | Optional. Enable partitioning using hive-style partitioning from the folder structure. Defaults to |
Spice integrates with multiple secret stores to help manage sensitive data securely. For detailed information on supported secret stores, refer to the secret stores documentation (opens in a new tab). Additionally, learn how to use referenced secrets in component parameters by visiting the using referenced secrets guide (opens in a new tab).
Refer to the FTP cookbook recipe (opens in a new tab) to see an example of the FTP connector in use.
falsefalse