Airfold workspaces are fully defined in YAML files, which enables you to leverage version control and CI/CD similar to tools like dbt, Terraform, and other infrastructure as code systems. Your files represent data pipelines as code with Airfold.

​
How CI/CD Works

CI/CD (Continuous Integration and Continuous Deployment) automates the process of testing, validating, and deploying changes to your projects. In a typical CI/CD workflow:
  • Continuous Integration (CI) ensures that changes, such as updates to pipelines or configurations, are tested and validated automatically whenever new code is pushed or a pull request is opened
  • Continuous Deployment (CD) automates the deployment of validated changes to staging or production environments

​
Why Use CI/CD?

Implementing CI/CD for Airfold workspaces is highly recommended for production systems as it enables you to:
  • Automate the process of testing, validating, and deploying changes to Airfold pipelines
  • Track changes by leveraging version control with your CI/CD pipelines
  • Maintain consistency across various environments (e.g., staging and production)

​
GitHub Actions Example

GitHub Actions is one way to integrate Airfold with CI/CD pipelines. Let’s go through an example of setting up a deployment workflow for Airlang using GitHub Actions:
deploy.yaml
name: Airfold deploy

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
  workflow_dispatch:

permissions:
  id-token: write
  contents: write
  pull-requests: write

jobs:
  airfold:
    name: Plan / Apply
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
          submodules: recursive
          token: ${{ secrets.DEPLOY_TOKEN }}

      - name: Set up Python 3.11
        uses: actions/setup-python@v4
        with:
            python-version: 3.11

      - name: Setup Airfold CLI
        uses: airfold/setup-airfold@v1
        with:
          api-token: ${{ secrets.AIRFOLD_API_KEY }}

      - name: Airfold diff
        id: diff
        if: github.event_name == 'pull_request'
        run: |
          af diff airfold
        continue-on-error: true

      - name: Airfold plan
        id: plan
        if: github.event_name == 'pull_request'
        run: |
          af push airfold --dry-run
        continue-on-error: true

      - name: Airfold Status
        if: steps.plan.outcome == 'failure'
        run: exit 1

      - name: Airfold push and apply
        if: (github.ref == 'refs/heads/main' && github.event_name == 'push') || github.event_name == 'workflow_dispatch'
        run: af push airfold

​
on

push: Runs the workflow whenever changes are pushed to the main branch pull_request: Executes the workflow when a pull request is opened or updated for the main branch workflow_dispatch: Allows the workflow to be triggered manually from the Actions tab in GitHub

​
permissions

Grants the workflow specific permissions:
  • id-token: Required for OIDC authentication
  • contents: Allows the workflow to read/write repository contents
  • pull-requests: Enables the workflow to comment on pull requests

​
jobs

jobs: Defines a single job named Plan / Apply runs-on: ubuntu-latest: Specifies the environment where the job will run 1. Checkout Repository 
uses: actions/checkout@v3
with:
  fetch-depth: 0
  submodules: recursive
  token: ${{ secrets.DEPLOY_TOKEN }}
  • Clones the repository, fetching all commit history and submodules
  • Authenticates using a secret DEPLOY_TOKEN
2. Set Up Python 3.11 
uses: actions/setup-python@v4
with:
    python-version: 3.11
  • Installs and configures Python 3.11 for the workflow
3. Setup Airfold CLI 
uses: airfold/setup-airfold@v1
with:
  api-token: ${{ secrets.AIRFOLD_API_KEY }}
  • Installs the Airfold CLI and configures it using the API key
4. Airfold diff 
id: diff
if: github.event_name == 'pull_request'
run: |
  af diff airfold
continue-on-error: true
  • Runs af diff to compare the current and previous states of the Airfold configuration
  • Executes only for pull requests
  • continue-on-error: true: Prevents failures in this step from stopping the workflow
5. Airfold plan 
id: plan
if: github.event_name == 'pull_request'
run: |
  af push airfold --dry-run
continue-on-error: true
  • Performs a dry run of af push to simulate changes
6. Airfold Status 
if: steps.plan.outcome == 'failure'
run: exit 1
  • Stops the workflow if the plan step fails, ensuring only valid changes proceed
7. Airfold push and apply 
if: (github.ref == 'refs/heads/main' && github.event_name == 'push') || github.event_name == 'workflow_dispatch'
run: af push airfold
Pushes changes to Airfold workspace when:
  • Code is pushed to main
  • The workflow is triggered manually through workflow_dispatch

​
GitHub Actions

GitHub Actions enable you to define workflows as code using YAML configuration files in your repository. High-level overview of a GitHub Actions workflow:
  1. Define a Workflow
  2. Specify Triggers
  3. Define Jobs
See more on the GitHub Actions Documentation.