Running test stages for react-native repos or NodeJS repos can be really painful if yarn install is taking too long to complete. Even though GitLab suggests you to use a cache configuration to cache the vendor folder so you won't have to wait for yarn install to complete everytime, there seems to be an issue with how long the caching actually takes to complete.

Journey

I've hit into this issue in the KopiRun pipelines and I set out to solve this. We're a small team and we don't particularly add or update packages that often, most of the time I'm the only one doing so. We don't want our machines to always be uploading caches if the cache is still relevant.

GitLab only has a pull-push, pull, push policy for the cache. They are missing a push if outdated as mentioned here.

Someone found out that if you remove the folders to be cached, the runners won't update the cache and therefore won't waste anytime to push the cache files. They provided a sample .gitlab-ci.yml configuration but that didn't really fit my needs so this led to me writing my own cache stage to not push if the cache is still relevant (Based off a hash of the yarn.lock file).

Solve

The following configuration has helped us save about 5 to 15 minutes depending on the project due to how it was previously configured.

stages:
  - cache
  - test

cache_job:
  stage: cache
  tags:
    - docker
  cache:
    key:
      files:
        - yarn.lock
    paths:
      - node_modules/
      - .yarn/
      - yarn.lock.sha256sum
    policy: pull-push
    when: on_failure
  script:
    - CI_JOB_SKIP_EXIT_CODE=0
    - yarn install --frozen-lockfile --prefer-offline --cache-folder .yarn
    -  # do whatever
    - |
      YARN_LOCK_SHA256_HASH=$( sha256sum yarn.lock | awk '{ print $1 }')
      echo "Current sha256 hash is $YARN_LOCK_SHA256_HASH"
      echo "Checking sha256 hash of yarn.lock"

      if ! sha256sum -c yarn.lock.sha256sum; then
        echo "yarn.lock checksum does not match cache"
        sha256sum yarn.lock > yarn.lock.sha256sum
        CI_JOB_SKIP_EXIT_CODE=218
      else
        echo "Cache is the same as before, won't update cache"
      fi
    - exit $CI_JOB_SKIP_EXIT_CODE
  allow_failure:
    exit_codes: 218

test_stage:
  stage: test
  tags:
    - docker
  cache:
    - key:
        files:
          - yarn.lock
      paths:
        - node_modules/
        - .yarn/
      policy: pull
  script:
    - echo "Test Stage"

Change Log

• 2023-05-03 - Renamed the job to cache_job
• 2022-09-10 - Updated example to support both SaaS and Self-Managed instances