Running scheduled cross-browser end-to-end tests on Github CI

March 10, 2020 / 4 min read

Last Updated: March 10, 2020

This is going to be a short post, the first one for 2020! Better late than never right?

You might have noticed in some of my previous posts that I like testing and love to use Cypress as a way to run my end-to-end tests. I was excited when I learned that they added support for Firefox which is one of the main features that was missing to use it in a professional setting. I also always wanted to make sure the personal projects I work on run well on every browser automatically on a weekly basis, including this blog. This is why I created this small Github action workflow to do just that and I wanted to share it with you all as it might come handy to some people.

Running cross-browser tests locally

Before running an automated CI task, we have to ensure that our existing end-to-end tests run locally. If you update Cypress to its latest version (4.0.2 as I'm writing these words) and have Chrome, Firefox, and Microsoft Edge installed on your machine, you might notice a little dropdown on the top right of the Cypress window when running cypress open, giving you a list of browser to run your tests on:

Screenshot showcasing the Cypress GUI with the browser selection dropdown expanded.

We can now run these tests and check if everything passes before going to the next steps.

Github workflow

Adding automated tasks to run on every Pull Request with Github workflows is very simple. However, here what we want to do is slightly different. We probably already have automated tasks that run our cypress tests for every PR and running them on every supported browser on top of that might be a bit overkill and might increase CI costs. Thus the following Github workflows will only work weekly, which is an option that Github allows:

Weekly cross-browser testing Github workflow

name: Weekly cross-browser testing
# * is a special character in YAML so you have to quote this string
- cron: '0 12 * * 1'
runs-on: ubuntu-latest
image: cypress/browsers:node12.16.1-chrome80-ff73
options: --user 1001
- name: Checkout Commit
uses: actions/checkout@v1
- name: Firefox Browser Tests
uses: cypress-io/github-action@v1
browser: firefox
start: yarn start
wait-on: 'http://localhost:8000'
wait-on-timeout: 300
config: defaultCommandTimeout=100000,pageLoadTimeout=100000,watchForFileChanges=false,video=false
- name: Chrome Browser Tests
uses: cypress-io/github-action@v1
browser: chrome
start: yarn start
wait-on: 'http://localhost:8000'
wait-on-timeout: 300
config: defaultCommandTimeout=100000,pageLoadTimeout=100000,watchForFileChanges=false,video=false

In the code snippet above, we can see that instead of running this action "on push", we run them "on schedule" with a cron syntax setting the run every Monday at 12 PM. I used Crontab Guru as suggested in the Github docs to generate the proper cron schedule expression. When the scheduled time is reached, this Github workflow will get triggered with the following steps:

  • ArrowAn icon representing an arrow
    Checking out the latest commit (this task will run on the master branch) (L16)
  • ArrowAn icon representing an arrow
    Start the project (L21 and L 29)
  • ArrowAn icon representing an arrow
    Run the end-to-end tests on Firefox
  • ArrowAn icon representing an arrow
    Run the end-to-end tests on Chrome

For the last two steps, we use the cypress-io/github-action@v1 which allows running Cypress in Github workflows with little to almost no configuration required besides setting the browser on which we want to run the tests (see L20 and L28). However, we have to specify that we want to use the cypress/browsers (see L11-12) Docker image for this workflow, as in its default setting, it seems that Github CI doesn't include Firefox.

Screenshot showcasing a successful Github CI run including our scheduled cross-browser tests on Chrome and Firefox

Now that this new automated weekly task is implemented we can ensure our projects run as expected and guarantee the best user experience regardless of which browser is being used 🎉!

Before adding such scheduled task on your own project, remember that, as I'm writing these words, Firefox support on Cypress is still currently in beta, so this post only showcased something I'd consider "experimental" at this point. We'll have to be a bit more patient before running this in production. If you want to see this workflow in action, I use it right now on some of my open-source repositories, click here to see an example.

More resources about running Cypress on Github CI:

Liked this article? Share it with a friend on Twitter or support me to take on more ambitious projects to write about. Have a question, feedback or simply wish to contact me privately? Shoot me a DM and I'll do my best to get back to you.

Have a wonderful day.

– Maxime

Github actions + Cypress + cross-browser testing = 🚀