Switching from Jenkins to GitHub as a CI/CD system

In April 2020, GitHub announced its own CI/CD platform, and thanks to the enterprise contract an alternative to CircleCI became available to us pretty much overnight. Since we already use the Github Package Registry for Docker images and NPM packages, we definitely wanted to try out the new actions. Our first impression, however, was rather sobering – the actions seemed unfinished and alien. In addition, a 1:1 migration was impossible. The concept of manual gates that we use for deployments at CircleCI does not exist there. So far, we have found it reassuring that pipelines do not update software on live systems, even following successful checks, without our intervention – some changes need to be very closely monitored. A manual gate only allows us to deploy code if we have the capacity for monitoring and a rollback.
Since we’ve put a lot of work into our CircleCI workflows over the last two years, there was no interest in having to restart from scratch.
After an initial proof of concept, our opinion changed for the better and we decided to dare to make the move.
We have migrated a total of 64 CircleCI pipelines to GitHub over the last few weeks and can now proudly give GitHub Actions a warm welcome!

Before we launched the move we identified the similarities between our pipelines.
All our pipelines:

• go through a build step (in our team, specifically: npm ci and npm run build)
• run unit/interactions/E2E tests (npm run test)
• audit dependencies (npm audit)
• check software licences, as we are not allowed to use all licences
• provide documentation in a central S3 bucket.

Besides this we also have three types of pipelines:

1. pipelines that build within the AWS infrastructure (terraform apply) and then roll out software
2. pipelines that provide software as an NPM package or Docker image
3. pipelines that require both.

Pipelines with an impact on the customer also report their deployment to a central monitoring function.
Besides this, we also have the following rules:

• feature branches require a gate to be rolled out to a develop system
• only the main branch can launch a live deployment behind a gate.

We were able to create appropriate GitHub workflows for all requirements. Only the gates could not be depicted. Instead, we use the ‘Released’ trigger for a deployment, which can be activated via the GitHub GUI or the GitHub API. We use prereleases for deployments that are only to be rolled out up to the develop system.

Another advantage is that JavaScript is a First Class Citizen at GitHub Actions. This is very convenient for our team, which feels most comfortable using Node.js.

Pipelines, especially if they are very similar, are unfortunately vulnerable to code duplications. For CircleCI we use Orbs and YAML anchors to make sure we stay DRY – ‘Don’t Repeat Yourself’.
For example, a deployment to our develop system is different from a live deployment in one place precisely – the target environment (live instead of develop). All other steps are completely identical. In the YAML files for CircleCI we were able to create a single source of truth via Anchors and Aliases, so we only have to make changes in one place.
Because we have pipelines that deploy within up to 6 different AWS accounts, we were able to cut out much of our identical code and keep a clear overview of the workflow. Orbs helped us reuse code across pipelines.
Regrettably there are no anchors on GitHub Actions and this feature is sorely missed by the community. GitHub is aware of this pain point; so far, however, there has only been one announcement that they’re working on it.
But since we didn't want to start duplicating code at all, we wrote an application called Gitty that updates workflows in all repositories via the GitHub API.
In each repo we have a config that Gitty reads via GitHub API. YAML workflows are generated from the config and checked back in by commit. Our initial intensive effort quickly paid off.

Example:
When it was recently announced that set-env would soon be deactivated, we needed to adjust code in just one place to update all repositories (read more).

Pipelines outside AWS need credentials to create resources within AWS. At CircleCI we had already distributed the AWS Credentials via Lambda and API and rotated them regularly. This is also possible with GitHub. But beware! With the GitHub API, the abuse detection mechanism strikes quickly. An unlimited Promise.all() against all repos triggers a very snappy 403.

Unfortunately, a CI system can always crash. This has rarely been the case with CircleCI in recent years, but when it has actually occurred it has always been at the wrong time. We have therefore designed all pipelines so that they can also be executed locally.
That's why we use an AWS role in the pipeline that can also be ‘assumed’ by our local AWS users. For instance, a deployment no longer fails because of a missing IAM policy. Fortunately, there are just a few core requirements for a development computer: Node.js, terraform, git, GITHUB_TOKEN, and AWS Credentials are enough to build, test and deploy all services locally. We definitely want to maintain this independence in all situations. While CI systems handle a lot of the hard work, they should never be the only systems available to roll out code in an emergency.

We are now very happy with GitHub Actions and have hardly had to make any further adjustments over the last few weeks. Jobs are running fast and stable. There are no waiting times for us at the moment, although things may soon get tight here too.
Changing the CI system is always a good opportunity for a good spring clean, and we have taken full advantage of this. The close dovetailing with GitHub has proven to be a great advantage. We are really happy to have dared to change and do not want to do without this solution in future!