Il y a 4 ans -
Temps de lecture 21 minutes
End-to-end web testing with TestCafe
A complete guide to getting started
E2E tests are easy with TestCafe
What you’ll learn
Getting Started
What is TestCafe, how to install it and create your first test.
Using TestCafe
Write a real test using a sample project: this will show you how to query elements, observe page state, interact with elements, use assertions. We’ll also learn how to organize our files with the page object pattern and using hooks.
Running Tests
Browsers support, headless mode, device emulation, concurrency.
Continuous Integration
Using CircleCI to run tests and see results.
Features
Some features you should know exist: intercepting HTTP requests, screenshots and videos, reporters, TestCafe studio.
Initially this article was a presentation, you can download the slides support here. Sources are available on Github.
Getting Started
What is TestCafe
A node.js tool to automate end-to-end web testing. You can write tests in JS or TypeScript, run them and view results. The tool is free and open source. The v1.0.1 is out since February 2019.
Requirements and installation
Of course be sure Node and NPM are installed (also yarn if you prefer). No webdriver is needed, just one command:
yarn add --dev testcafe
Test Code Structure
fixture `Getting Started` .page `https://blog.xebia.fr`; test('My first test', async t => { // Test code });
Here is a basic test file. It contains a fixture: a category of tests. A fixture is defined by a name and a page function that targets the start page of our group of tests. TestCafe uses a proxy server, it injects scripts in the page then we can inspect the elements.
Each test receives a test controller as a parameter. The test controller is an object that allows us to access the test api. We’ll use it for actions and assertions.
We run the test with the following command:
testcafe chrome test.js
Then the test is ran in a chrome browser (you can choose another browser):
Running tests in: - Chrome 72.0.3626 / Mac OS X 10.13.6 Getting Started ✓ My first test 1 passed (2s)
The project we are going to test
A React app created with create-react-app and material-ui:
Home page with an enter button
Posts page with a list of links
Article page with content of a post
Add page to post a new article
The site is deployed on xke-introduction-testcafe.surge.sh
The tests we want
First scenario:
As a user, when I’m on the home page, I can enter the app and links to articles. If I click on a link, I can see the article page.
Second scenario:
As a user, from the home page, I can access the add page with a form in order to post a new article. I can post my article.
We will enable the live mode to watch for changes with the L option:
testcafe chrome e2e/**/* -L
Let’s write our first scenario skeleton
// For now we run our project locally fixture `Navigation`.page `http://localhost:3000`; test('Access to an article from the home page', async t => { // Test code });
e2e/index.js
The result in our terminal:
Live mode is enabled. TestCafe now watches source files and reruns the tests once the changes are saved. You can use the following keys in the terminal: 'Ctrl+S' - stops the test run; 'Ctrl+R' - restarts the test run; 'Ctrl+W' - enables/disables watching files; 'Ctrl+C' - quits live mode and closes the browsers. Watching the following files: /Users/proustibat/workspace/xebia/xke-introduction-testcafe/e2e/index.js Running tests in: - Chrome 72.0.3626 / Mac OS X 10.13.6 Navigation ✓ Access to a specific article from the home page 1 passed (0s)
Querying elements with Selector
Selector is a function that identifies a webpage element in the test. The selector API provides methods and properties to select elements on the page and get their state.
TestCafe uses standard CSS selectors to locate elements. It’s like when you use document.querySelector
in JS (learn more by reading the documentation).
In our test, we import Selector
then we need to select the start button on our home page:
import { Selector } from 'testcafe'; // import Selector fixture `Navigation`.page `http://localhost:3000`; test('Access to an article from the home page', async t => { const startBtn = Selector('a').nth(1); // define startBtn });
Actions
Test API provides a set of actions that enable you to interact with the webpage.
Actions are implemented as methods in the test controller object and can be chained.
Multiple actions are available: Click, Right Click, Double Click, Drag Element, Hover, Take Screenshot, Navigate, Press Key, Select Text, Type Text, Upload, Resize Window (read the documentation to learn more).
import { Selector } from 'testcafe'; fixture `Navigation`.page `http://localhost:3000`; test('Access to a specific article from the home page', async t => { const startBtn = Selector('a').nth(1); await t.click(startBtn); // click the start button });
Observing page state
TestCafe allows you to observe the page state via:
– Selector used to get direct access to DOM elements
– ClientFunction used to obtain arbitrary data from the client side.
For example in our test, once we click on the button, we should be on the posts page, so we can access the title page and log its innerText as follows:
import { Selector } from 'testcafe'; fixture `Navigation`.page `http://localhost:3000`; test('Access to an article from the home page', async t => { const startBtn = await Selector('a').nth(1); await t.click(startBtn); // Note these methods and property getters are asynchronous const title = await Selector('h4'); const titleText = await title.innerText; console.log(titleText); // => Posts Page });
Assertions
We now need assertions to check data or behaviors. The test controller provides an expect
function that should check the result of performed actions.
The following assertion methods are available: Deep Equal, Not Deep Equal, Ok, Not Ok, Contains, Not Contains, Type of, Not Type of, Greater than, Greater than or Equal to, Less than, Less than or Equal to, Within, Not Within, Match, Not Match.
In our test, we want to check that the title is “Posts Page”:
import { Selector } from 'testcafe'; fixture `Navigation`.page `http://localhost:3000`; test('Access to an article from the home page', async t => { const startBtn = await Selector('a').nth(1); await t.click(startBtn); const title = await Selector('h4'); const titleText = await title.innerText; // The assertion await t.expect(titleText).eql('Posts Page'); });
Finishing our tests
Now we learned the 4 main concepts of TestCafe: selectors, page state, actions and assertions. So we can complete our first test:
import { Selector } from 'testcafe'; fixture `Navigation`.page `http://localhost:3000`; test('Access to an article from the home page', async t => { // WHEN (enter click on home page) const startBtn = await Selector('a').nth(1); await t.click(startBtn); // THEN (the title is the right and we have 100 links on the page) await t.expect(await Selector('h4').innerText).eql('Posts Page'); const links = await Selector('ul').child('a'); await t.expect(await links.count).eql(100); // WHEN (click on first article link) const firstLink = await links.nth(0).child('div'); const firstLinkText = await firstLink.innerText; // save title value await t.click(firstLink); // THEN (article page) const titleArticleEl = await Selector('h4'); const titleArticleText = await titleArticleEl.innerText; await t.expect(titleArticleText).eql(firstLinkText); });
What about our second test
Remember for our second test we want to start from the home page, enter the site by clicking the start button (like the first part of our first test). Then we want to verify the users navigate to the form page when they click on the add button. Finally they should be able to fill the form and submit it.
So the beginning of our test is the same as the first, and we need to do what is written in comments:
test('Access to the form and posting an article, coming from home', async t => { // WHEN (enter click on home page) const startBtn = await Selector('a').nth(1); await t.click(startBtn); // THEN (posts page elements verification) await t.expect(await Selector('h4').innerText).eql('Posts Page'); const links = await Selector('ul').child('a'); await t.expect(await links.count).eql(100); // TODO // Check if a "plus button" exists // Click on the button // Check if we've navigated to the form page // Enter a title and a content // Submit the form // Check the success notification });
Let’s begin with the well known part!
This is not new now, we are able to get elements, have them undergo some actions then check some other elements exist on the page:
// Check if a "plus button" exists const addBtn = await Selector('button[aria-label=Add]'); await t.expect(addBtn.exists).ok(); // Click on the button await t.click(addBtn); // Check if we've navigated to the form page const formEl = await Selector('form'); await t.expect(formEl.exists).ok();
To fill the form, we use the typeText
method of the TestController. Submitting the form is not new, we just need to select the button then click it:
// Enter a title const inputTitle = await formEl.find('#field-title'); await t.typeText(inputTitle, 'How TestCafe is awesome!'); // Enter a content const textAreaField = await formEl.find('#field-content'); await t.typeText(textAreaField, 'Bla Bli Blou'); // Submit the form const submitBtn = await formEl.find('button'); await t.click(submitBtn); // Check the success notification const toastEl = await Selector('.Toastify').child('div'); await t.expect(toastEl.exists).ok();
The page object pattern
Now we need to talk about the organisation of our code. Writing our tests is repetitive: we go on a page, we check elements are present, we perform actions on it, then we check some elements are present, and so on.
During development process, markups, ids and classes can often change so we need to change Selectors everywhere it’s needed.
Fortunately, we can use the page object pattern:
- Page Model is a test automation pattern that allows you to create an abstraction of the tested page and use it in test code to refer to page elements.
- Keep page representation in the Page Model, while tests remain focused on the behavior.
- Improve maintainability.
Read TestCafe documentation
Now we will refactor our tests by creating page models.
Creating home model
A model is a basic class. We define Selector in the constructor:
import { Selector } from 'testcafe'; export default class HomePage { constructor () { this.startBtn = Selector('a').nth(1); } }
e2e/models/home.js
Now we can use our model in our tests by importing and instantiating it then accessing its selectors:
import { Selector } from 'testcafe'; import HomePage from './models/home'; // import the model const homePage = new HomePage(); // instantiation fixture `Navigation`.page `http://localhost:3000`; test.only('Access to an article from the home page', async t => { // const startBtn = await Selector('a').nth(1); // removed // await t.click(startBtn); // removed await t.click(homePage.startBtn); ... });
Creating a posts page model
Following the same principle, we create a model for the posts page that contains selectors for its title, its links and its add button:
import { Selector } from 'testcafe'; export default class PostsPage { constructor () { this.title = Selector('h4'); this.links = Selector('ul').child('a'); this.addBtn = Selector('button[aria-label=Add]'); } }
e2e/models/posts.js
Now we can use the model in our tests. Replace title and links selectors:
// THEN (posts page elements verification) await t.expect(await Selector('h4').innerText).eql('Posts Page'); const links = await Selector('ul').child('a'); await t.expect(await links.count).eql(100);
With:
// THEN (posts page elements verification) await t.expect(await postsPage.title.innerText).eql('Posts Page'); const links = await postsPage.links; await t.expect(await links.count).eql(100);
Replace add button selector:
// Check if a "plus button" exists const addBtn = await Selector('button[aria-label=Add]'); await t.expect(addBtn.exists).ok(); // Click the button await t.click(addBtn);
With:
// Check if a "plus button" exists await t.expect(await postsPage.addBtn.exists).ok(); // Click on the button await t.click(await postsPage.addBtn);
Creating article model
import { Selector } from 'testcafe'; export default class ArticlePage { constructor () { this.title = Selector('h4'); } }
e2e/models/article.js
In our tests, replace:
// THEN (article page) const titleArticleEl = await Selector('h4'); const titleArticleText = await titleArticleEl.innerText;
With:
// THEN (article page) const titleArticleText = await articlePage.title.innerText;
Creating add page model
import { Selector } from 'testcafe'; export default class AddPage { constructor () { this.form = Selector('form'); this.inputTitle = this.form.find('#field-title'); this.textAreaField = this.form.find('#field-content'); this.submitBtn = this.form.find('button'); } }
e2e/models/add.js
Use it in our tests by replacing:
// Check if we've navigated to the form page const formEl = await Selector('form'); await t.expect(formEl.exists).ok(); // Enter a title const inputTitle = await formEl.find('#field-title'); await t.typeText(inputTitle, 'How TestCafe is awesome!'); // Enter a content const textAreaField = await formEl.find('#field-content'); await t.typeText(textAreaField, 'Bla Bli Blou'); // Submit the form const submitBtn = await formEl.find('button'); await t.click(submitBtn);
with:
// Check if we've navigated to the form page const formEl = await addPage.form; await t.expect(formEl.exists).ok(); // Get the selectors from the addPage model const inputTitle = await addPage.inputTitle; const textAreaField = await addPage.textAreaField; const submitBtn = await addPage.submitBtn; // Chained methods to enter a title, a content and submit form await t .typeText(inputTitle, 'How TestCafe is awesome!') .typeText(textAreaField, 'Bla Bli Blou') .click(submitBtn);
Note that we also chained actions. This is now more readable.
We now have 4 page models that represent the pages of our application. Each page contains its own Selectors.
Now we are going to do the same thing for actions.
Adding actions to the model pages
TestCafe allows to use Test Controller outside of test code by importing it:
import { Selector, t } from ‘testcafe’;
Add this to our add page model:
async submitForm (title, content) { await t .typeText(this.inputTitle, title) .typeText(this.textAreaField, content) .click(this.submitBtn); }
Then use actions in our test:
/* REPLACE THIS: // Get the selectors from the addPage model const inputTitle = await addPage.inputTitle; const textAreaField = await addPage.textAreaField; const submitBtn = await addPage.submitBtn; // Chained methods to enter a title, a content and submit form await t .typeText(inputTitle, 'How TestCafe is awesome!') .typeText(textAreaField, 'Bla Bli Blou') .click(submitBtn); */ // WITH: // Enter a title, a content and submit form await addPage.submitForm( 'How TestCafe is awesome!', 'Bla Bli Blou' );
Complete the add page model with:
async isPageDisplayed () { await t.expect(await this.form.exists).ok(); await t.expect(await this.inputTitle.exists).ok(); await t.expect(await this.textAreaField.exists).ok(); await t.expect(await this.submitBtn.exists).ok(); }
Complete posts page model with:
async isPageDisplayed () { await t.expect(await this.title.innerText).eql('Posts Page'); await t.expect(await this.links.count).eql(100); await t.expect(await this.addBtn.exists).ok(); } async clickFirstLink () { const link = await this.links.nth(0).child('div'); const linkText = await link.innerText; await t.click(link); return linkText; // we'll need to save it }
Create a toast model as follows:
import { Selector, t } from 'testcafe'; export default class ToastPage { constructor () { this.toastEl = Selector('.Toastify').child('div'); } async isToastDisplayed () { await t.expect(await this.toastEl.exists).ok(); } }
e2e/models/toast.js
Our tests become even more simple
The first test was:
test('Access to an article from the home page', async t => { // WHEN (enter click on home page) const startBtn = await Selector('a').nth(1); await t.click(startBtn); // THEN (posts page elements verification) await t.expect(await Selector('h4').innerText).eql('Posts Page'); const links = await Selector('ul').child('a'); await t.expect(await links.count).eql(100); // WHEN (click on first article link) const firstLink = await links.nth(0).child('div'); const firstLinkText = await firstLink.innerText; await t.click(firstLink); // THEN (article page) const titleArticleEl = await Selector('h4'); const titleArticleText = await titleArticleEl.innerText; await t.expect(titleArticleText).eql(firstLinkText); });
Then became:
test('Access to an article from the home page', async t => { await t.click(homePage.startBtn); await postsPage.isPageDisplayed(); const text = await postsPage.clickFirstLink(); await t .expect(await articlePage.title.innerText) .eql(text); });
Our second test was:
test('Access to the form and posting an article, coming from home', async t => { const startBtn = await Selector('a').nth(1); await t.click(startBtn); await t.expect(await Selector('h4').innerText).eql('Posts Page'); const links = await Selector('ul').child('a'); await t.expect(await links.count).eql(100); const addBtn = await Selector('button[aria-label=Add]'); await t.expect(addBtn.exists).ok(); await t.click(addBtn); const formEl = await Selector('form'); await t.expect(formEl.exists).ok(); const inputTitle = await formEl.find('#field-title'); await t.typeText(inputTitle, 'How TestCafe is awesome!'); const textAreaField = await formEl.find('#field-content'); await t.typeText(textAreaField, 'Bla Bli Blou'); const submitBtn = await formEl.find('button'); await t.click(submitBtn); const toastEl = await Selector('.Toastify').child('div'); await t.expect(toastEl.exists).ok(); });
Then became:
test('Access to the form and posting an article, coming from home', async t => { await t.click(homePage.startBtn); await postsPage.isPageDisplayed(); await t.click(await postsPage.addBtn); await addPage.isPageDisplayed(); await addPage.submitForm('How TestCafe is awesome!', 'Bla Bli Blou'); await toastPage.isToastDisplayed(); });
Hooks
TestCafe allows you to specify functions that are executed before a fixture or test is started and after it is finished. Exactly like when you write unit tests.
- fixture.beforeEach( fn(t) )
- fixture.afterEach( fn(t) )
- test.before( fn(t) )
- test.after( fn(t) )
Note that test hooks override fixture hooks.
Remember we need to click start button in each of our tests. So we can use a fixture hook that will execute code before each test:
fixture`Navigation` .page`http://localhost:3000` .beforeEach( async t => { await t.click(homePage.startBtn); await postsPage.isPageDisplayed(); });
Then remove the corresponding lines from our tests.
Debugging
The debug method of TestController
TestCafe provides the t.debug
method that pauses the test and allows you to debug using the browser’s developer tools.
Here we want to stop the scenario with a debugger after we enter title in the input and before we enter content in the textarea:
async submitForm(title, content) { await t .typeText(this.inputTitle, title) .debug() // the test stops here .typeText(this.textAreaField, content) .click(this.submitBtn); }
The debugger is visible in the console
The test has stop…
Debugging with screenshots on failure
Use the following command to get screenshots when the test failed:
testcafe chrome e2e/**/* --screenshots ./screenshots --screenshots-on-fails
Debugging with videos
Enables TestCafe to record videos of test runs and specifies the base directory to save these videos.
testcafe chrome e2e/**/* --video videos
Some options are available:
testcafe chrome e2e/**/* --video videos --video-options singleFile=true,failedOnly=true
See documentation about videos
Running Tests
Browser support: installed browsers
Most of modern browsers locally installed can be detected: Google Chrome (Stable, Beta, Dev and Canary), Internet Explorer (11+), Microsoft Edge, Mozilla Firefox, Safari, Android browser, Safari mobile.
If these browsers are locally installed then TestCafe will detect it.
Run testcafe --list-browsers
to list which browsers TestCafe automatically detects.
Browsers support: remote device
To run tests on a remote mobile or desktop device, this device must have network access to the TestCafe server.
testcafe remote e2e/**/* testcafe remote e2e/**/* --qr-code

Browsers support: chrome device emulation
To do this, use the emulation browser parameter. Specify the target device with the device parameter:
testcafe "chrome:emulation:device=iphone 6/7/8" e2e/**/* --video artifacts/videos
Browsers support: cloud testing services
TestCafe allows you to use browsers from cloud testing services.
The following plugins for cloud services are currently provided by the TestCafe team.
You can also create your own plugin: see the doc.
Some other CLI options
Concurrent test execution:
testcafe -c 2 chrome e2e/**/*
Speed:
testcafe chrome e2e/**/* — speed 0.5
Multiple browsers with or without headless:
testcafe chrome:headless,firefox e2e/**/*
See the documentation for more options
You can also use a .testcaferc.json file for the config.
Reporters
Reporters are plugins used to output test run reports in a certain format.
TestCafe ships with the following reporters: spec (used by default), list, minimal, xUnit, JSON.
Here are some custom reporters developed by the community: NUnit, Slack, TeamCity.
Read the documentation to know more
Continuous Integration
Prerequisites
Remember we should run yarn start
before running our tests because we target localhost:3000
as the start webpage.
Our goal is to test exactly the same code that will be deployed or not (depending on results). So we need to serve built content after we run yarn build
.
We will use the serve package, so install it with yarn add — dev serve
Add a npm script to the package.json file:
“serve:build”: “serve -s build”
This will run our app on localhost:5000
.
Modify our tests to target build sources
In e2e/index.js
, replace localhost:3000
on our fixture page by localhost:5000
. Then build the app with yarn build
.
Serve the built content in a terminal with yarn serve:build
then run testcafe in headless mode in another terminal:
testcafe chrome:headless e2e/**/*
The all in one with the appCommand
For now we need 2 terminals to run our tests. This is not ideal.
The appCommand of TestCafe executes the specified shell command before running tests.
So we can use it to serve our built app instead of running the yarn serve:build
manually. Add this script to the package.json file:
“e2e:ci”: “testcafe chrome:headless e2e/*.js — app ‘yarn serve:build’”
Now we’re ready for CircleCI integration. If you don’t have any account, create it before going to the next step.
Create config.yml in .circleci
version: 2.0 jobs: build: docker: - image: circleci/node:8-browsers working_directory: /home/circleci/project steps: - checkout - setup_remote_docker: docker_layer_caching: true # Download and cache dependencies - restore_cache: name: Restore Yarn Package Cache keys: - yarn-packages-{{ checksum "yarn.lock" }} - run: name: Install Dependencies command: yarn install --frozen-lockfile - save_cache: name: Save Yarn Package Cache key: yarn-packages-{{ checksum "yarn.lock" }} paths: - ~/.cache/yarn # Build - deploy: name: Build command: yarn build # End-to-end tests - run: name: Run e2e tests command: yarn e2e:ci
Add your repo to CircleCI
On your dashboard click on “add projects” in the left bar menu
Find your repo, then click on “Set Up Project”:
Start building the project on CircleCI
You should see your project running:
Detailed workflow on CircleCI
Use store_test_results
Add xunit reporter to the project:
yarn add — dev testcafe-reporter-xunit
Modify the CircleCI config:
# End-to-end tests - run: name: Run e2e tests command: yarn e2e:ci - store_test_results: path: /tmp/test-results
Modify the npm script as follows:
“e2e:ci”: “testcafe chrome:headless e2e/*.js — app ‘yarn serve:build’ -r xunit:/tmp/test-results/res.xml”
See the summary on your dashboard
Be proud! Add badges to your readme:
[](https://circleci.com/gh/proustibat/xke-introduction-testcafe/tree/master)
[](https://github.com/DevExpress/testcafe)
Now CircleCI is running for each pull request
Some features you should be aware of
Intercepting HTTP requests
Logging HTTP Requests:
RequestLogger is a hook that can be attached to either a test or a fixture. It allows to record sent or received requests.
For example, here we record http request accessing an article with a get method. Then we can use assertions to check the status code of the response:
import { RequestLogger } from 'testcafe'; // import // < model imports and instantiations here > const logger = RequestLogger( { url, method: 'get' }, { logResponseHeaders: true, logResponseBody: true } ); // < fixture code here > test .requestHooks(logger) // Attach our logger as a hook to the test ('Access to the form and posting an article, coming from home', async t => { // ... await t.expect( logger.contains(r => r.response.statusCode === 200) ).ok(); });
Mocking HTTP Responses
We can also mock requests with the RequestMock hook.
Here, I added Google Analytics to our project. We don’t want to send the page view event to our GA dashboard when the navigation comes from our tests. So we mock the request by intercepting it and sending a fake gif:
import { RequestMock } from 'testcafe'; // import // < other imports and models instantiations > const url = 'http://localhost:5000'; const collectDataGoogleAnalyticsRegExp = new RegExp('https://www.google-analytics.com/r/collect'); const mockedResponse = Buffer.from([0x47, 0x49, 0x46, 0x38, 0x39, 0x61, 0x01, 0x00, 0x01]); const mock = RequestMock().onRequestTo(collectDataGoogleAnalyticsRegExp) .respond(mockedResponse, 202, { 'content-length': mockedResponse.length, 'content-type': 'image/gif' }); fixture`Navigation` .page(url) .requestHooks(mock) // Attach our mock as hook to the fixture .beforeEach(async t => { await t.click(homePage.startBtn); await postsPage.isPageDisplayed(); });
Framework-Specific Selectors
TestCafe team and community develop libraries of dedicated selectors for the most popular frameworks. So far, the following selectors are available.
- React
- Angular
- AngularJS
- Vue
- Aurelia
TestCafe Studio
A cross-platform IDE
- Create, edit and maintain end-to-end tests in a visual recorder without writing code. See Record Tests.
- Generates a report with overall results and details for each test after completion.
- Code Editor with syntax highlight, code completion and parameter hints.
- Write code from scratch, convert recorded tests to JavaScript to edit them later.
Recording our first test
Running with remote
Recording our second test
Code editor
Links:
- Tech doc: https://devexpress.github.io/testcafe/
- TestCafe Studio: https://docs.devexpress.com/TestCafeStudio
- Continue the tests by yourself: https://github.com/proustibat/xke-introduction-testcafe
- The tested web app: https://xke-introduction-testcafe.surge.sh/
Commentaire