Manage your pipelines using Git and YAML
By using Git repositories and YAML files, you can apply familiar engineering workflows like pull requests, branching, and CI/CD to your projects.
Airfold workspaces are fully defined in YAML files, which enables you to leverage version control and CI/CD similar to tools like dbt, Terraform, and other infrastructure as code systems.
Your files represent data pipelines as code with Airfold.
Restore Workspace History
Each commit represents the exact state of your Airfold workspace at a given point in time.
Easily navigate to and restore any previous version of the workspace to:
Roll Back Issues
If a recent change causes unexpected errors, simply roll back to the last known working commit.
This is helpful for avoiding downtime in production by reverting to a stable state with minimal effort.
Enable Collaboration
Git enables multiple team members to work on the same workspace simultaneously without conflicts:
save
on a query edit can accidentally overwrite each other’s changesSeamless Sharing
Within your organization:
You can also open-source your Airfold workspaces just as you would with any Git repository, allowing the broader community to reuse and adapt your workspaces.
Branch for Production and Staging
Maintain separate branches for staging and production environments:
A typical Airfold project is organized into the following directory structure:
Pipes: Define the sequence of SQL transformations in your data pipeline
Sources: Define data sources
Endpoints: (Optional) Define API endpoints for serving pipeline results.
From your project directory, use Git to initialize version control:
Commit existing workspace files to the repository:
Push your repository:
Create a branch for staging:
Use this staging
branch to test new changes:
Once changes are validated, merge into production:
Use pull requests for any workspace updates:
Separate branches for staging and production:
main
for production-ready workspace configurationsstaging
for test configurationsManage your pipelines using Git and YAML
By using Git repositories and YAML files, you can apply familiar engineering workflows like pull requests, branching, and CI/CD to your projects.
Airfold workspaces are fully defined in YAML files, which enables you to leverage version control and CI/CD similar to tools like dbt, Terraform, and other infrastructure as code systems.
Your files represent data pipelines as code with Airfold.
Restore Workspace History
Each commit represents the exact state of your Airfold workspace at a given point in time.
Easily navigate to and restore any previous version of the workspace to:
Roll Back Issues
If a recent change causes unexpected errors, simply roll back to the last known working commit.
This is helpful for avoiding downtime in production by reverting to a stable state with minimal effort.
Enable Collaboration
Git enables multiple team members to work on the same workspace simultaneously without conflicts:
save
on a query edit can accidentally overwrite each other’s changesSeamless Sharing
Within your organization:
You can also open-source your Airfold workspaces just as you would with any Git repository, allowing the broader community to reuse and adapt your workspaces.
Branch for Production and Staging
Maintain separate branches for staging and production environments:
A typical Airfold project is organized into the following directory structure:
Pipes: Define the sequence of SQL transformations in your data pipeline
Sources: Define data sources
Endpoints: (Optional) Define API endpoints for serving pipeline results.
From your project directory, use Git to initialize version control:
Commit existing workspace files to the repository:
Push your repository:
Create a branch for staging:
Use this staging
branch to test new changes:
Once changes are validated, merge into production:
Use pull requests for any workspace updates:
Separate branches for staging and production:
main
for production-ready workspace configurationsstaging
for test configurations