Cypress vs. Selenium: Our path to finding the right testing fit

As a QA tester, I want to share a real-world journey that my team and I took, to find the best testing solution that suits our needs. We were looking into two popular tools for testing websites: Cypress and Selenium. Think of this like a story from someone who tests software, explaining how we figured out which tool was best for our project management app.

Cypress and Selenium are both big names in the world of testing web apps, but they’re pretty different in how they work and what they can do. Cypress is like an all-in-one package for testing modern websites from start to finish. Selenium is more about giving you a bunch of tools to automate tasks in browsers, and it works with many programming languages.

Our journey wasn’t just about picking between two options. It was about finding the perfect match for what our project needed, how we do things, and how we can keep things running smoothly in the long run.

Developing a project management app

We were working on a project management app designed to plan, track, and manage tasks across various browsers and mobile devices.

The application comprises 3 parts.

• The marketing site, which was created in Gatsby, serves to inform you about the app and to create an instance for yourself.
• The second part is the back office. We created it using Symfony PHP, and we used it to manage instances of clients, subscription payments, etc.
• The third part, the application itself, where the frontend of the application is made with React.js and the backend is created in Symfony PHP.

Our goal was to make sure this application runs smoothly, regardless of whether you're using Chrome on your laptop or Safari on your phone.

Beyond cypress: Seeking versatility

Initially, I leaned towards Cypress because we already had some API tests in our project that were using it. However, when expanding these tests to more complex functional and end-to-end scenarios, I hit a few roadblocks.

Cypress had limitations, especially with browser support and the removal of certain plugins, like the one for XPath.

For instance, when locating elements, the Cypress had some problems when I tried with attribute-class or with any attribute value (because of the way the html code is written), so then the only reliable solution for me to locate the elements was with XPath.

But when I wrote the tests, Cypress didn't support the XPath plugin. Initially, in HTML there wasn’t the attribute-id for the elements that I needed to interact with in the tests.

Identifying those elements with attribute-class was hard and slow, so the next way was to locate them with XPath.

Cypress doesn’t support locating elements with XPath, so I had to look for another solution. Also Cypress doesn’t support Safari without external WebKit.

After thorough discussions and exploring alternatives, our team concluded that, while other tools had their strengths, it was Selenium that offered the comprehensive features and flexibility we needed. This decision came after evaluating what each tool could bring to our project, with Selenium ultimately aligning best with our specific requirements.

This is an example (code snippet) of locating the element using XPath with Selenium

// Locate an element using XPath
// Replace '//your/xpath/here' with the actual XPath of the element you want to interact with
let element = await driver.findElement(By.xpath('//your/xpath/here'));

// Perform actions on the element, e.g., click, getText, sendKeys, etc.
// Example: to get text from the element
let text = await element.getText();
console.log(text);

Setting up Selenium

Since the Cypress framework is a JavaScript-based end-to-end testing framework built on Mocha (a feature-rich JavaScript test framework running on and in the browser, making asynchronous testing simple and convenient), I've made a setup with Selenium also using JavaScript Mocha framework. In both cases it is necessary to install node.js

Steps to install and setup Selenium

Create a new directory for your project and initialize a Node.js project:

mkdir my-selenium-project
cd my-selenium-project
npm init -y

Install Selenium WebDriver, which is necessary for browser automation:

npm install selenium-webdriver --save-dev

You need to install drivers for the browsers you want to automate (e.g., Chrome, Firefox). Here's how you can install the ChromeDriver as an example:

npm install chromedriver --save-dev

Install Mocha, the testing framework:

npm install mocha --save-dev

In your package.json file, add a script to run Mocha tests:

"scripts": {
  "test": "mocha"
}

Key differences between Cypress and Selenium

Setup

Cypress: Cypress is relatively straightforward to set up, as it automates much of the configuration process. It excels in simplicity, although it is primarily optimized for Chrome and Electron. This focus means those requiring extensive cross-browser testing may need to undertake additional preparations.

Selenium: The setup for Selenium requires manual configuration of WebDriver for each browser. Besides setting up the WebDriver, Selenium tests also demand the initial definition of assertions and other pre-test setups, which can seem complex, especially for those new to the framework.

This additional initial setup is necessary to tailor Selenium for a wide range of testing scenarios and environments.

Stability and Performance

Cypress: Generally offers more stable tests due to its automatic waiting mechanism and less complex setup, but may face limitations in more extensive testing scenarios.

Selenium:  Offers robust testing capabilities, though it occasionally necessitates extra configuration for element synchronization and performance tuning to maintain test stability.

Syntax

Cypress: Offers a more fluent and easier-to-read syntax, which is especially beneficial for JavaScript developers familiar with Mocha/Chai.

Selenium: While powerful, may require more verbose code, demanding a steeper learning curve for those not familiar with its syntax.

Browser Support

Cypress: Primarily supports Chrome and Electron, with limited support for other browsers, potentially requiring additional work for extensive cross-browser tests.

Selenium: Supports a wider range of browsers out of the box, making it a preferred choice for comprehensive cross-browser testing.

Comparison of simple tests in Cypress and Selenium

To understand these differences better, let's compare the implementation of a simple login test in each tool. We'll create a simple login test scenario where a user enters their email and password and then clicks the login button.

For example, login test in Cypress:

describe('Login Test', () => {
  it('Successfully logs in with valid credentials', () => {
    // Visit the login page
    cy.visit('https://our-project.com/login');

    // Fill in the email address
    cy.get('.form--email').type('[email protected]');

    // Fill in the password
    cy.get('.form--password').type('yourpassword');

    // Click the login button
    cy.get('.login__button').click();

    // Add assertions here to verify successful login
    // For example, checking if the URL has changed or a specific element is visible after login
    cy.url().should('include', '/dashboard');
    cy.get('.some-element-after-login').should('be.visible');
  });
});

This test script does the following:

• Visits the login page.
• Types a given email address into the email field.
• Types a given password into the password field.
• Clicks the login button.
• It is recommended that you add assertions to verify that the login was successful. This might include checking if the URL has changed to a dashboard or if a certain element is visible after logging in.

And this is the same test written using Selenium:

const { Builder, By, Key } = require('selenium-webdriver');
const assert = require('assert');
const chrome = require('selenium-webdriver/chrome');
const path = require('chromedriver').path;

let driver;

describe('Login Test', function() {
  this.timeout(30000);

  before(async function() {
    let service = new chrome.ServiceBuilder(path).build();
    chrome.setDefaultService(service);

    driver = await new Builder().forBrowser('chrome').build();
  });

  it('Successfully logs in with valid credentials', async function() {
    // Replace with your login page URL
    await driver.get('https://our-project.com/login');

    // Fill in the email address
    await driver.findElement(By.className('form--email')).sendKeys('[email protected]');

    // Fill in the password
    await driver.findElement(By.className('form--password')).sendKeys('yourpassword');

    // Click the login button
    await driver.findElement(By.className('login__button')).click();

    // Add assertions here to verify successful login
    // For example, checking if URL has changed or a specific element is visible after login
    await driver.getCurrentUrl().then(url => {
      assert.strictEqual(url, 'https://our-project.com/dashboard');
    });
  });

  after(async function() {
    await driver.quit();
  });
});

In this script:

• Selenium WebDriver is set up with Chrome.
• The script navigates to the login page.
• It fills in the email and password fields.
• Clicks the login button.
• It is recommended that you include assertions to verify a successful login, such as checking if the URL has changed to the dashboard page or if a certain element is visible.

These are just simple tests with basic functionality, just to show how extensive Selenium is and that it is necessary to define several things at the beginning in order to run the test successfully.

Remember, these are simplified examples. In real-world scenarios, you'll need to handle more complex elements like waits, handling dynamic elements, and possibly using Page Object Models for better test maintenance.

Advantages gained with Selenium

Advantages gained with Selenium decided a testing framework for our application. It stood out for several reasons:

Cross-browser testing

Our application's functionality and performance needed to be consistent across a variety of browsers and devices. With its robust cross-browser testing capabilities, Selenium allowed us to test our application on popular browsers like Chrome, Firefox, Safari, and on a range of mobile platforms. This was essential to cater to our diverse user base and ensure a seamless experience for all users, regardless of their preferred browsing environment.

Ecosystem and community

The strength of Selenium lies not only in its technical capabilities but also in its extensive support ecosystem. The platform benefits from a large, active user community, which makes problem-solving more efficient and accessible. This community support, combined with a rich assortment of libraries, facilitated the development and maintenance of our tests, saving us time and resources.

Flexibility

Our application comprises various technologies and needs to function seamlessly across different browsers and devices. Selenium’s flexibility was a significant advantage. Its support for multiple programming languages meant we had a greater choice of development tools.

Selenium’s ability to interact effectively with different components of the application, regardless of the underlying technology, provided us with the versatility necessary to tackle complex testing scenarios head-on.

In summary, Selenium's adaptability to diverse environments, its strong community support, and its flexible nature made it the optimal choice for our application's testing requirements. Its ability to meet our specific needs ultimately led to our decision to use Selenium as our primary testing tool.

The verdict: Selenium this time!

For our project, Selenium emerged as the better fit. It aced the cross-browser testing and offered the flexibility we needed for our diverse technology stack. Plus, its large community meant we weren’t alone when we ran into issues.

But hey, this isn’t a one-size-fits-all verdict. Cypress has its strengths, especially in ease of use and modern JavaScript support.

The key takeaway? Your choice should hinge on what your specific project demands. If it's broad browser support and flexibility, Selenium might be your top pick.

But for a more streamlined, JavaScript-focused experience, Cypress could be the way to go. Remember, there’s no “best” tool, only the right tool for the job at hand. So, consider your project’s needs, and choose wisely!