automated browser testing: bridging the gap between dev and qa

We recently were a part of a project with what was, in many ways, a typical successful startup. The company makes hardware for a niche market, powered by their own firmware and driven by a suite of web applications running both on a server and locally as Electron apps. They make a great product that is disrupting the space and they’re growing rapidly, both in company size and number of users.

What started as a small integrated team has spun up to several groups overseeing various aspects of the product and as that happened the developers became somewhat siloed from the QA folks. Each group had its own process for keeping the quality high in the face of rapid growth, namely thorough unit tests on the development side and a series of step-by-step documents used by a number of testers to manually go through every page and every button of the web applications. Releases were coming quickly and the testers were spending hours upon hours methodically testing only to have to start all over again when another release came out of development. They were overworked and almost overwhelmed, and called Infinity for help.

automated browser testing to the rescue

We all agreed that automated browser testing was the place to start, and the hows and whats were left up to me. I’d done some testing like this before, using that old workhorse Selenium to drive Firefox in a dedicated remote testing environment. I knew it would do the job, but setting it up from scratch is rather daunting, especially for a modern JavaScript web application. During an early meeting, one of the developers mentioned Cypress.io so I went and read up on that – and promptly fell in love.

Cypress is a standalone testing tool that can automate any application that runs in a web browser. There’s no environment to set up, no special drivers to integrate with a browser. Using it in a JavaScript project is as simple as npm install cypress. It’s free and licensed under the MIT license. It uses the Mocha testing framework and the Chai assertion library, so it’s likely easy to use right out of the box by any JavaScript developer. It’s capable of controlling any Chromium- or Firefox-based browser, which leaves out only IE and Safari from cross-browser testing. There are many tutorials and blog posts out there about technical aspects of Cypress, so I’ll skip over those. Suffice it to say in an afternoon I had it installed and was running simple heartbeat tests of their web applications.

delving deeper in the cypress forest

More useful tests required just a bit more work. Cypress works via DOM manipulation and has a robust API to help navigate through a page. It’s possible to give it general directions to the elements you want tested, such as “the second child of the fourth div”, and I used that quite a bit in my early proof of concept tests. You can target elements by tag, class, id, text content, etc., so example tests for existing code are extremely quick to set up.

As I’m sure you can guess, keeping up with that in a rapidly iterative application is a sure way to get brittle tests. While you can do something like cy.get('.btn.btn-large').click() to grab that large button on your page, styling class names are certain to change down the road and break all your tests. A better approach is to give the important elements unique identifiers. I fell into the habit of doing that in my early days as a Rails developer, but it’s not often the default behavior of JavaScript frameworks. You can use the id or name attribute, like cy.get('[name=submission]').click(). That might be fine if you’re hand-crafting every element, but oftentimes those get generated dynamically (or not generated at all) by the JS framework. An even better approach is to use a special data attribute to contain your identifiers, like cy.get('[data-cy=submit]').click(). What data attribute you use is configurable, so you can go with something easy like data-cy.

Bloated javascript code is a real issue, and we do not want all these testing tags to slow down our users. When you minimize and package your code for production, your tools can be configured to strip out these dedicated data attributes. Your tests still run in development and staging environments without affecting the size of your production application. Cypress even comes with an interactive view of your pages called “Selector Playground” that can generate these attributes for you as you mouse over elements. However you get the job done, once you have the important elements identified, scripting actions on the page becomes a breeze.

translating human language to automated scripts

This is where the step-by-step documents from the QA folks came into play. On this project, the test plans were already in the form of “navigate to this page, click on that input field, type this value, click that button, and you should see this happen” and that translated directly to my Mocha and Chai tests:

context("The Landing Page", () => {
  before(() => {
    // The url to be tested
    cy.visit("/this-page");
    // What browser viewport?
    cy.viewport("macbook-15");
    // Say there are pop-up terms & conditions we need to agree to
    // before we can see the page, so find the button that says "YES"
    // & click it. Force it in case it's invisible to the test runner.
    cy.contains("YES").click({ force: true });
  });

  describe("The initial form", () => {
    it("types some text and clicks the button", () => {
      // Look for the element with my data tag
      cy.get("[data-cy=that-input-field]")
        .click()
        .type("this value")
        .blur();
      // Find the button with the data tag
      cy.get("[data-cy=that-button]").click();
      // Go down to the new tagged div that shows the thing that
      // happened and make sure it exists
      cy.get("[data-cy=that-div]").should("exist");
    });
  });
});

[my-awesome-app]  The Landing Page
[my-awesome-app]     The initial form
      ✓ types some text and clicks the button (37ms)

This direct translation is a perfect opportunity to bring the developers and the QA folks together. Theirs can be a naturally adversarial relationship, and opportunities to directly collaborate like this should be welcomed. In my case, QA already had a pile of step-by-step documents they were using to manually test, and passing those along to the developers gave them an opportunity to get comfortable with the Cypress API in a way that immediately removed a time-consuming burden from QA. It was easy to see how for future development QA would write the human scripts and pass those to the developer to turn them into Cypress scripts. That extra bit of communication at the start of the development cycle has the added benefit of making sure the developers explicitly know how the changes should behave and hopefully cut down on the back and forth that can happen during acceptance testing.

bells and whistles

The QA documents generally have a list of users to log in as, for use in testing user roles and permissions. They often have a separate set of instructions for setting up data to be used later on in the testing process. Developers get to use fixtures with their unit tests and can be blind to the possibility that QA can need a few hours of setup time before they can even begin to do their testing and QA might not know that fixtures can be of benefit to them, too. Cypress has all of the fixture functionality developers expect and they can be loaded in before running the integration tests or brought in as needed, giving all the flexibility that a person at the keyboard and mouse would have.

Another feature Cypress has that can aid in this communication is automatic screen recordings of failed test runs. Like everything I encountered with Cypress that behavior is configurable, but out of the box, mp4 files are created for every failure. Those can get passed along from the developer to the QA folks or the designers or whomever so they can see exactly how the application misbehaved. A picture is worth a thousand words (and static images can get generated too) but a quick little movie is even better.

Writing these integration tests may be more up-front work for the developers, but they’re already writing similar unit tests so it’s familiar work. And where there is a lot of overlap, that’s a sign that perhaps the unit tests can be simplified. Since Cypress.io is installed as part of your application, it has access to both the front and back ends, and the API lets you make the most of that. Let’s say that a developer wants to be sure a method is called with the correct arguments when a specific button is pushed. The test can “spy” on that method and assert what should be happening there. Similarly, tests can stub out methods, resolve promises, and even manipulate the JavaScript clock to test debounces and timeouts.

And of course, developers may want to automate the automation, so tests get run automatically as code is written, committed, and deployed. It’s simple to add Cypress in with the existing IDE, Git hooks, and Continuous Integration process you might have, so the additional overhead is low.

big bang for the buck

Automated browser testing can easily get dismissed as something that would be nice to have, but not worth the effort. Folks with scars from projects using Selenium might be especially hesitant to implement this step in the development cycle. But as you’ve seen above, browser testing gives you far more than a robot to push buttons on a web page. It brings QA and dev staff together, it produces more robust code, and it saves time throughout the development process. And with the ease of installation and use, Cypress.io provides a high leverage point lifting your project to success.

Tags: technology javascript testing communication qa