Running scheduled cross-browser end-to-end tests on Github CI

Mar 10 2020

/ 2 min read /

0 Likes β€’

0 Replies β€’

0 Reposts

This is going to be a short post, the first one for 2020! Better late than never right?

You might have noticed in some of my previous posts that I like testing and love to use Cypress as a way to run my end-to-end tests. I was excited when I learned that they added support for Firefox which is one of the main features that was missing to use it in a professional setting. I also always wanted to make sure the personal projects I work on run well on every browser automatically on a weekly basis, including this blog. This is why I created this small Github action workflow to do just that and I wanted to share it with you all as it might come handy to some people.


Running cross-browser tests locally

Before running an automated CI task, we have to ensure that our existing end-to-end tests run locally. If you update Cypress to its latest version (4.0.2 as I'm writing these words) and have Chrome, Firefox, and Microsoft Edge installed on your machine, you might notice a little dropdown on the top right of the Cypress window when running cypress open, giving you a list of browser to run your tests on:

Screenshot showcasing the Cypress GUI with the browser selection dropdown expanded.
Screenshot showcasing the Cypress GUI with the browser selection dropdown expanded.

We can now run these tests and check if everything passes before going to the next steps.

Github workflow

Adding automated tasks to run on every Pull Request with Github workflows is very simple. However, here what we want to do is slightly different. We probably already have automated tasks that run our cypress tests for every PR and running them on every supported browser on top of that might be a bit overkill and might increase CI costs. Thus the following Github workflows will only work weekly, which is an option that Github allows:

Weekly cross-browser testing Github workflow

1
name: Weekly cross-browser testing
2
3
on:
4
schedule:
5
# * is a special character in YAML so you have to quote this string
6
- cron: '0 12 * * 1'
7
8
jobs:
9
cross-browser:
10
runs-on: ubuntu-latest
11
container:
12
image: cypress/browsers:node12.16.1-chrome80-ff73
13
options: --user 1001
14
steps:
15
- name: Checkout Commit
16
uses: actions/checkout@v1
17
- name: Firefox Browser Tests
18
uses: cypress-io/github-action@v1
19
with:
20
browser: firefox
21
start: yarn start
22
wait-on: 'http://localhost:8000'
23
wait-on-timeout: 300
24
config: defaultCommandTimeout=100000,pageLoadTimeout=100000,watchForFileChanges=false,video=false
25
- name: Chrome Browser Tests
26
uses: cypress-io/github-action@v1
27
with:
28
browser: chrome
29
start: yarn start
30
wait-on: 'http://localhost:8000'
31
wait-on-timeout: 300
32
config: defaultCommandTimeout=100000,pageLoadTimeout=100000,watchForFileChanges=false,video=false

In the code snippet above, we can see that instead of running this action "on push", we run them "on schedule" with a cron syntax setting the run every Monday at 12 PM. I used Crontab Guru as suggested in the Github docs to generate the proper cron schedule expression. When the scheduled time is reached, this Github workflow will get triggered with the following steps:

  • Checking out the latest commit (this task will run on the master branch) (L16)
  • Start the project (L21 and L 29)
  • Run the end-to-end tests on Firefox
  • Run the end-to-end tests on Chrome

For the last two steps, we use the cypress-io/github-action@v1 which allows running Cypress in Github workflows with little to almost no configuration required besides setting the browser on which we want to run the tests (see L20 and L28). However, we have to specify that we want to use the cypress/browsers (see L11-12) Docker image for this workflow, as in its default setting, it seems that Github CI doesn't include Firefox.

Screenshot showcasing a successful Github CI run including our scheduled cross-browser tests on Chrome and Firefox
Screenshot showcasing a successful Github CI run including our scheduled cross-browser tests on Chrome and Firefox

Now that this new automated weekly task is implemented we can ensure our projects run as expected and guarantee the best user experience regardless of which browser is being used πŸŽ‰!


Before adding such scheduled task on your own project, remember that, as I'm writing these words, Firefox support on Cypress is still currently in beta, so this post only showcased something I'd consider "experimental" at this point. We'll have to be a bit more patient before running this in production. If you want to see this workflow in action, I use it right now on some of my open-source repositories, click here to see an example.

More resources about running Cypress on Github CI:

Fetching Replies...

Do you have any questions, comments or simply wish to contact me privately? Don’t hesitate to shoot me a DM on Twitter.


Have a wonderful day.
Maxime


Β© 2020 Maxime Heckel β€”β€” Made in SF. Polished in NY.