Testing GitHub Actions

At Doist, we're switching away from Azure Pipelines towards GitHub Actions as our only CI provider.

When improving any of our workflows or when switching an existing task from a Pipeline to an Action we're faced with the problem of asserting whether the CI is working as expected. We could use the "YOLO" approach: merge our changes into main and iterate on them there until they work. But that's rarely a nice or professional approach and causes too much noise and frustration for everybody involved.

When looking around for solutions to this problem, most people recommend using nektos/act. Act can run GitHub Actions locally and allows you to test whether your workflow runs as you expect it to. Unfortunately, in our most recent use case we couldn't use act as we needed to access repository secrets which were not available locally.

Solution 1: Change trigger to the PR

Instead of succumbing to the YOLO workflow, we use another ingenious approach: while developing the Action we change the trigger to run on the pull request:

name: GitHub Action Workflow

on: pull_request

Instead of running only when there's a new tag or a new push to a certain branch, we tell the Action to run on every PR. This way, we can push changes to our PR until we ensured everything works correctly. After that, we change the trigger back to the actual value, clean up the PR and ask for a final review before merging it with quiet conscience.

Solution 2: Add manual action trigger

When the code of your action works correctly (verified through solution 1) you might want to check that it behaves correctly under different circumstances and with different parameters. To do this, you can add a ˋworkflow_dispatchˋ to your triggers. Doing so will create a button in the GitHub UI which you can use to run your action manually and even supply it with various input parameters. Check the official announcement blog post

# simple case, no inputs
on: workflow_disptach

# with inputs
on: workflow_dispatch
    inputs:
        logLevel:
            description: "Log Level"
            required: true
            default: "warning"