Running scheduled cross-browser end-to-end tests on Github CI
March 10, 2020 / 4 min readLast Updated: March 10, 2020
This is going to be a short post, the first one for 2020! Better late than never right?
You might have noticed in some of my previous posts that I like testing and love to use Cypress as a way to run my end-to-end tests. I was excited when I learned that they added support for Firefox which is one of the main features that was missing to use it in a professional setting. I also always wanted to make sure the personal projects I work on run well on every browser automatically on a weekly basis, including this blog. This is why I created this small Github action workflow to do just that and I wanted to share it with you all as it might come handy to some people.
Running cross-browser tests locally
Before running an automated CI task, we have to ensure that our existing end-to-end tests run locally. If you update Cypress to its latest version (
4.0.2 as I'm writing these words) and have Chrome, Firefox, and Microsoft Edge installed on your machine, you might notice a little dropdown on the top right of the Cypress window when running
cypress open, giving you a list of browser to run your tests on:
We can now run these tests and check if everything passes before going to the next steps.
Adding automated tasks to run on every Pull Request with Github workflows is very simple. However, here what we want to do is slightly different. We probably already have automated tasks that run our cypress tests for every PR and running them on every supported browser on top of that might be a bit overkill and might increase CI costs. Thus the following Github workflows will only work weekly, which is an option that Github allows:
Weekly cross-browser testing Github workflow
1name: Weekly cross-browser testing23on:4schedule:5# * is a special character in YAML so you have to quote this string6- cron: '0 12 * * 1'78jobs:9cross-browser:10runs-on: ubuntu-latest11container:12image: cypress/browsers:node12.16.1-chrome80-ff7313options: --user 100114steps:15- name: Checkout Commit16uses: actions/checkout@v117- name: Firefox Browser Tests18uses: cypress-io/github-action@v119with:20browser: firefox21start: yarn start22wait-on: 'http://localhost:8000'23wait-on-timeout: 30024config: defaultCommandTimeout=100000,pageLoadTimeout=100000,watchForFileChanges=false,video=false25- name: Chrome Browser Tests26uses: cypress-io/github-action@v127with:28browser: chrome29start: yarn start30wait-on: 'http://localhost:8000'31wait-on-timeout: 30032config: defaultCommandTimeout=100000,pageLoadTimeout=100000,watchForFileChanges=false,video=false
In the code snippet above, we can see that instead of running this action "on push", we run them "on schedule" with a cron syntax setting the run every Monday at 12 PM. I used Crontab Guru as suggested in the Github docs to generate the proper cron schedule expression. When the scheduled time is reached, this Github workflow will get triggered with the following steps:
- Checking out the latest commit (this task will run on the master branch) (L16)
- Start the project (L21 and L 29)
- Run the end-to-end tests on Firefox
- Run the end-to-end tests on Chrome
For the last two steps, we use the
cypress-io/github-action@v1 which allows running Cypress in Github workflows with little to almost no configuration required besides setting the browser on which we want to run the tests (see L20 and L28). However, we have to specify that we want to use the
cypress/browsers (see L11-12) Docker image for this workflow, as in its default setting, it seems that Github CI doesn't include Firefox.
Now that this new automated weekly task is implemented we can ensure our projects run as expected and guarantee the best user experience regardless of which browser is being used 🎉!
Before adding such scheduled task on your own project, remember that, as I'm writing these words, Firefox support on Cypress is still currently in beta, so this post only showcased something I'd consider "experimental" at this point. We'll have to be a bit more patient before running this in production. If you want to see this workflow in action, I use it right now on some of my open-source repositories, click here to see an example.
More resources about running Cypress on Github CI:
Github actions + Cypress + cross-browser testing = 🚀