How to configure CI/CD in GitHub Actions for Node.js and Docker.

I'm currently developing my Master's Degree final project, which is a monorepo with a Node.js backend (Nest.js framework) and a React frontend (Next.js framework) Just for you to know, those are the frameworks I use for my full stacks developments, including this site. It's a complex software architecture, as it involves the two frameworks and a database. So to speed up my development, and to allow my professors to easily access my updates (I mean, download the current version of the project and run it), I decided to set up a CI/CD flow in GitHub Actions.

in this article, I will explain how I did it, and how you can do it too. It will automatically run linters, styles, tests, linters, and build and publish the Docker images to GitHub Registry. If you prefer to upload your images to Docker Hub, just a few easy changes are needed, that I also explain here.

A first look to the files

In my monorepo project root, I have a .github/workflows folder, which contains the GitHub Actions workflow files. I have two files, one for the pull requests flows and one for my production branch main. Those are called pull_request.yml and main.yml respectively. Let's take a look at the pull_request.yml file:

Pull request workflow

# pull_request.yml
name: CI

on:
  pull_request:
    branches:
      - main
    types: [synchronize, opened, reopened, ready_for_review]

jobs:
  skip-draft:
    name: Skip draft
    runs-on: ubuntu-latest
    timeout-minutes: 1
    steps:
      - name: fail on draft
        if: github.event.pull_request.draft == true
        run: echo "is a draft PR, failing..." && exit 1
      - name: success
        run: echo "is not a draft PR"

  lint:
    needs: skip-draft
    name: Linter
    runs-on: ubuntu-latest
    container: node:16
    timeout-minutes: 10
    steps:
      - name: Check out repository code
        uses: actions/checkout@v3
      - uses: actions/cache@v3
        with:
          path: '**/node_modules'
          key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}
      - name: Install dependencies
        run: yarn install --frozen-lockfile
      - name: Typecheck
        run: yarn typecheck
      - name: Eslint
        run: yarn lint:ci
      - name: Prettier
        run: yarn format:ci

  test-backend:
    needs: skip-draft
    name: Test backend
    runs-on: ubuntu-latest
    container: node:16
    timeout-minutes: 15
    services:
      database:
        image: postgres:14.0-alpine
        env:
          POSTGRES_USER: test
          POSTGRES_PASSWORD: test
          POSTGRES_DB: test
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
    steps:
      - name: Check out repository code
        uses: actions/checkout@v3
      - uses: actions/cache@v2.1.7
        with:
          path: '**/node_modules'
          key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}
      - name: Install dependencies
        run: yarn install --frozen-lockfile
      - name: Tests
        run: |
          yarn migrate:up
          yarn test:back:ci
        env:
          DB_HOST: database
          DB_USER: test
          DB_PASSWORD: test
          DB_DATABASE: test

  test-frontend:
    needs: skip-draft
    name: Test frontend
    runs-on: ubuntu-latest
    container: node:16
    timeout-minutes: 15
    steps:
      - name: Check out repository code
        uses: actions/checkout@v3
      - uses: actions/cache@v2.1.7
        with:
          path: '**/node_modules'
          key: ${{ runner.os }}-modules-${{ hashFiles('**/yarn.lock') }}
      - name: Install dependencies
        run: yarn install --frozen-lockfile
      - name: Tests
        run: yarn test:front:ci

  docker-build:
    needs: skip-draft
    name: Build docker container
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - name: Checkout code
        uses: actions/checkout@v3
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v1
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1
      - name: Build docker image
        uses: docker/build-push-action@v2.10.0
        with:
          context: .
          file: ./docker/Dockerfile
          push: false

As you can see at the start of the file, this Actions flow is configured to run only on pull requests against main branch.

The first step is skip-draft, which will return 1 and stop the workflow if the pull request is a draft. This is useful because I don't want to run the tests and linters if the pull request is a draft and therefore consume my GitHub Actions minutes. Note: I think GH Actions workflows no longer start if the pull request is a draft, but I got this job configured from an older project.

If the pull request is not a draft, the next four steps start simultaneously (instead of the following waiting to the previous to finalize) toz save developing time.

The first step, lint, will check the types and run linters and style checkers. I use tsc, eslint and prettier, but you could use whatever you want. Just configure them as a npm script in package.json. Additionally, with the actions/cache action I cache the node_modules folder, so the dependencies are not installed every time in the following jobs. I also use the --frozen-lockfile yarn flag to ensure that the dependencies are the same as the ones in the yarn.lock file, and thus the same that I'm using for development. Otherwise, it will throw.

Steps test-backend and test-frontend will run the tests for every app with jest. test-frontend job is straight forward, so I'll focus on explaining test-backend. I'm developing using a PostgreSQL as database, and there are some end-to-end tests that requires a database, as I'm also testing the repositories of my hexagonal architecture. So, I need to start a PostgreSQL container in the same job. I set up the database container with the proper env, wait for it to start up, and then execute database migrations before the tests. I also set the env variables for the backend to connect to the database.

Finally, the last step is docker-build, which will build the Docker image. As this is a pull request flow, and therefore could be changes, I don't want the built image to push to anywhere. Just to check that the Dockerfile is valid and the image builds without errors. So then, I load my custom Docker file (which is in <project-root>/docker/Dockerfile) and set push to false.

Main branch workflow

The production branch workflow is very similar to the pull request one, but it will only run when a commit is pushed to main branch, and will also push the built image to GitHub Registry. The changes from pull-request.yml are:

# main.yml
name: CI

on:
  push:
    branches:
      - main

jobs:
  lint:
    # ...

  test-backend:
    # ...

  test-frontend:
    # ...

  docker-build-and-push:
    needs: ['lint', 'test-backend', 'test-frontend']
    name: Build Docker container and push to registry
    runs-on: ubuntu-latest
    timeout-minutes: 10
    permissions:
      contents: read
      packages: write
    steps:
      - name: Checkout repository code
        uses: actions/checkout@v3

      - name: Log in to the Container registry
        uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: ghcr.io/${{ github.repository }}

      - name: Build and push Docker image
        uses: docker/build-push-action@v3
        with:
          context: .
          file: ./docker/Dockerfile
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}

This time, the flow will only run when a commit is pushed to main branch. Although lint and tests are supposed to have run and success in PR workflow, I'll check them again just to ensure that there are not strange fails after merge or branch sync. And also, to ensure hot-fixes directly against main are not breaking the code. In fact, the docker-build-and-push job won't start until the previous three jobs are finished successfully, to avoid uploading broken versions.

The built docker image is uploaded to the GitHub registry and can be accessed from ghcr.io/<gh_username>/<repo_name>. It is automatically tagged thanks to the docker/metadata-action action, which will tag the image with the branch name, the commit SHA, and the date. This is very useful to know which image is running in production, and also to rollback to a previous version if needed.

Change GitHub Registry for Docker Hub

If you want to use Docker Hub instead of GitHub Registry, you can do it by changing the docker/login-action and docker/metadata-action like this:

docker-build-and-push:
  needs: ['lint', 'test-backend', 'test-frontend']
  name: Build Docker container and push to registry
  runs-on: ubuntu-latest
  timeout-minutes: 10
  permissions:
    contents: read
    packages: write
  steps:
    - name: Checkout repository code
      uses: actions/checkout@v3

    - name: Log in to the Container registry
      uses: docker/login-action@v2
      with:
        # TODO: set secrets in GitHub repository
        username: ${{ secrets.DOCKER_USERNAME }}
        password: ${{ secrets.DOCKER_PASSWORD }}

    - name: Extract metadata (tags, labels) for Docker
      id: meta
      uses: docker/metadata-action@v4
      with:
        # TODO: set your real Docker Hub namespace
        images: <docker-hub-namespace>/${{ github.repository }}

    - name: Build and push Docker image
      uses: docker/build-push-action@v3
      with:
        context: .
        file: ./docker/Dockerfile
        push: true
        tags: ${{ steps.meta.outputs.tags }}
        labels: ${{ steps.meta.outputs.labels }}

Remember to check that your Docker Hub account credentials are configured into your repository secrets.

NPM scripts for CI

As you could see, I tagged most of my npm scripts with the :ci suffix, which means that they are configured to run in a CI environment. I consider it a good practice and a easy way of configure CI specific script/configurations, and to ensure that changes to the development scripts won't break CI ones by mistake. Here is the relevant part of my package.json:

{
  "scripts": {
    "typecheck": "tsc",
    "format:ci": "prettier --check \"src/**/*.{ts,tsx}\" \"**/*.json\"",
    "lint:ci": "eslint --fix \"{src,apps,libs,test}/**/*.{ts,tsx}\"",
    "test:back:ci": "jest -c src/backend/test/jest.config.js",
    "test:front:ci": "jest -c src/frontend/test/jest.config.js",
    "migrate:up": "ts-node src/backend/api/infrastructure/database/kysely/kysely-migrator.script.ts"
  }
}

Protecting the main branch

To get a real CI/CD workflow, it's a good practice to protect the production branch, which is main in my case. GitHub easily allows to configure protected branches, so pull-requests cannot be merged until the pull-request.yml workflow is finished successfully. This way, we can ensure that the code is tested and linted before merging it to production.

This is especially useful if you have a team of developers working on the same repository, and/or if you configure automerge once all the checks are passed. So you can review the changes and mark them for merge once the checks are finished and forget. If everything goes well, the code will be automatically merged to production. Otherwise, you will be by default notified by a GitHub email and you can fix the issues.

What are you waiting for? Start automatizing!