How to connect and sync Snowflake tables to Airfold
Airfold makes it easy to connect with data that you have stored in Snowflake. It also enables you to automate data updates using scheduled cron jobs.
This guide walks through an example of configuring a Snowflake connector in your workspace.
Navigate to Sources
and click on Create new Source
. Alternatively, click on the +
next to Sources
in the left-hand tool bar.
In the popup window, click on Snowflake
The next window in the pop up will ask you for some information from your Snowflake account.
Account
The Account
field requires the following format:
<your_account_id>.<your-snowflake-region>.<your_cloud_provider>
(e.g., lh10221.us-central1.gcp)
Run the following query in a Snowflake SQL worksheet to generate the correct account string:
Copy the result of this query and paste into the Airfold UI where needed. If this query does not return the expected value, you can manually construct the string using the following:
For example, if the above query returned:
Then the correct value for Account
would be `gu81828.us-east-2.aws’
Username
This is the username that you use to log in to your Snowflake account. You can obtain this by running the following query:
Password
The password that you use to log in to your Snowflake Account. If unknown, contact your Snowflake administrator.
Database
Name of the database that you wish to access.
Optional Settings:
Warehouse
Name of your warehouse, defaults to COMPUTE_WH
Schema
The schema name of the data you wish to ingest into Airfold, defaults to PUBLIC
Role
Name of your role
Click on “Test Connection”, and you should see this:
Select the table you wish to add from your database. A Snowflake connection can only have one table. If you wish to join the data from multiple tables together, you will need to create a Source
for each table and then join the data using a downstream Pipe
.
This section will require you to run administrative queries in your Snowflake account that will create a Role
and a User
and then grant permissions for those new resources to the database that you are syncing data from.
Run these queries one by one in your Snowflake account. Once each query has successfully run, click the Mark as Done
checkbox and then Next
.
Set up automated data updates with a cron expression:
Airfold will automatically infer a schema from Snowflake, but you have the opportunity to make any changes. Remember - once a source is created, the table settings cannot be altered. If you wish to alter the table at a later stage, all the data will have to be dropped which could cause issues in your downstream, production APIs. Source settings, such as the Engine
type and the Primary Key
, are critical for optimal query performance - Take care at this stage to ensure your Source is optimally configured.
Click Confirm Schema to finalize the connection. Your Snowflake table will now sync into Airfold based on your cron schedule, and the Source will appear in your workspace.
How to connect and sync Snowflake tables to Airfold
Airfold makes it easy to connect with data that you have stored in Snowflake. It also enables you to automate data updates using scheduled cron jobs.
This guide walks through an example of configuring a Snowflake connector in your workspace.
Navigate to Sources
and click on Create new Source
. Alternatively, click on the +
next to Sources
in the left-hand tool bar.
In the popup window, click on Snowflake
The next window in the pop up will ask you for some information from your Snowflake account.
Account
The Account
field requires the following format:
<your_account_id>.<your-snowflake-region>.<your_cloud_provider>
(e.g., lh10221.us-central1.gcp)
Run the following query in a Snowflake SQL worksheet to generate the correct account string:
Copy the result of this query and paste into the Airfold UI where needed. If this query does not return the expected value, you can manually construct the string using the following:
For example, if the above query returned:
Then the correct value for Account
would be `gu81828.us-east-2.aws’
Username
This is the username that you use to log in to your Snowflake account. You can obtain this by running the following query:
Password
The password that you use to log in to your Snowflake Account. If unknown, contact your Snowflake administrator.
Database
Name of the database that you wish to access.
Optional Settings:
Warehouse
Name of your warehouse, defaults to COMPUTE_WH
Schema
The schema name of the data you wish to ingest into Airfold, defaults to PUBLIC
Role
Name of your role
Click on “Test Connection”, and you should see this:
Select the table you wish to add from your database. A Snowflake connection can only have one table. If you wish to join the data from multiple tables together, you will need to create a Source
for each table and then join the data using a downstream Pipe
.
This section will require you to run administrative queries in your Snowflake account that will create a Role
and a User
and then grant permissions for those new resources to the database that you are syncing data from.
Run these queries one by one in your Snowflake account. Once each query has successfully run, click the Mark as Done
checkbox and then Next
.
Set up automated data updates with a cron expression:
Airfold will automatically infer a schema from Snowflake, but you have the opportunity to make any changes. Remember - once a source is created, the table settings cannot be altered. If you wish to alter the table at a later stage, all the data will have to be dropped which could cause issues in your downstream, production APIs. Source settings, such as the Engine
type and the Primary Key
, are critical for optimal query performance - Take care at this stage to ensure your Source is optimally configured.
Click Confirm Schema to finalize the connection. Your Snowflake table will now sync into Airfold based on your cron schedule, and the Source will appear in your workspace.