Running scheduled cross-browser end-to-end tests on Github CI

Running cross-browser tests locally

Before running an automated CI task, we have to ensure that our existing end-to-end tests run locally. If you update Cypress to its latest version (4.0.2 as I'm writing these words) and have Chrome, Firefox, and Microsoft Edge installed on your machine, you might notice a little dropdown on the top right of the Cypress window when running cypress open, giving you a list of browser to run your tests on:

We can now run these tests and check if everything passes before going to the next steps.

Github workflow

Adding automated tasks to run on every Pull Request with Github workflows is very simple. However, here what we want to do is slightly different. We probably already have automated tasks that run our cypress tests for every PR and running them on every supported browser on top of that might be a bit overkill and might increase CI costs. Thus the following Github workflows will only work weekly, which is an option that Github allows:

1
name: Weekly cross-browser testing
2

3
on:
4
  schedule:
5
    # * is a special character in YAML so you have to quote this string
6
    - cron: '0 12 * * 1'
7

8
jobs:
9
  cross-browser:
10
    runs-on: ubuntu-latest
11
    container:
12
      image: cypress/browsers:node12.16.1-chrome80-ff73
13
      options: --user 1001
14
    steps:
15
      - name: Checkout Commit
16
        uses: actions/checkout@v1
17
      - name: Firefox Browser Tests
18
        uses: cypress-io/github-action@v1
19
        with:
20
          browser: firefox
21
          start: yarn start
22
          wait-on: 'http://localhost:8000'
23
          wait-on-timeout: 300
24
          config: defaultCommandTimeout=100000,pageLoadTimeout=100000,watchForFileChanges=false,video=false
25
      - name: Chrome Browser Tests
26
        uses: cypress-io/github-action@v1
27
        with:
28
          browser: chrome
29
          start: yarn start
30
          wait-on: 'http://localhost:8000'
31
          wait-on-timeout: 300
32
          config: defaultCommandTimeout=100000,pageLoadTimeout=100000,watchForFileChanges=false,video=false

In the code snippet above, we can see that instead of running this action "on push", we run them "on schedule" with a cron syntax setting the run every Monday at 12 PM. I used Crontab Guru as suggested in the Github docs to generate the proper cron schedule expression. When the scheduled time is reached, this Github workflow will get triggered with the following steps:

• Checking out the latest commit (this task will run on the master branch) (L16)
• Start the project (L21 and L 29)
• Run the end-to-end tests on Firefox
• Run the end-to-end tests on Chrome

For the last two steps, we use the cypress-io/github-action@v1 which allows running Cypress in Github workflows with little to almost no configuration required besides setting the browser on which we want to run the tests (see L20 and L28). However, we have to specify that we want to use the cypress/browsers (see L11-12) Docker image for this workflow, as in its default setting, it seems that Github CI doesn't include Firefox.

Now that this new automated weekly task is implemented we can ensure our projects run as expected and guarantee the best user experience regardless of which browser is being used 🎉!

Before adding such scheduled task on your own project, remember that, as I'm writing these words, Firefox support on Cypress is still currently in beta, so this post only showcased something I'd consider "experimental" at this point. We'll have to be a bit more patient before running this in production. If you want to see this workflow in action, I use it right now on some of my open-source repositories, click here to see an example.

More resources about running Cypress on Github CI:

• https://www.cypress.io/blog/2019/11/20/drastically-simplify-your-testing-with-cypress-github-action/
• https://github.com/cypress-io/github-action