If you’ve ever heard about end to end testing before (e2e testing), then you probably heard about how difficult it is to maintain such tests and how flaky they can be. People usually complain about how much work it is to implement end to end testing. Fortunately for us developers, things have changed for the best in the past few years.
In this article, we are going to take a look at the current state of end to end testing and see if it might work for you and your organization.
Why should I consider end to end testing (e2e testing)?
Most developers are familiar with unit tests, which focus on testing single units of code. Some developers are also familiar with integration tests, which test the interactions between different units of code. While these two types of tests are great to validate code as it is being written, such tests do not validate an application.
This is where end-to-end tests come into play. An end-to-end test is all about testing an application from the end user’s perspective. As a result, an end-to-end test does not know about the internals of the application. It is a black box type of test. For instance, e2e tests simulate a user clicking on a button, opening a menu, or entering some text in a form.
Real-life user scenarios can be very difficult to implement from a unit-test perspective. On the other hand, user scenarios are fairly straightforward to reproduce when switching to an end-to-end perspective.
Example of end-to-end test code
Here is an example of an e2e test. In this example, we simulate a user click on a drop-down. Then the user selects an item in that dropdown, and we check that the new selection has the expected effect on the web application. In that scenario, the price of a product changes when we select a different currency:
it('supports multiple currencies', () => {
driver.findElement(By.css('app-license-plate:nth-of-type(1)')).contains('$8');
driver.findElement(By.css('app-currency-switcher')).click();
driver.findElement(By.text('EUR')).click();
driver.findElement(By.css('app-license-plate:nth-of-type(1)')).contains('€9.12');
});
A few things come to mind when reading the above code:
- The code is easy to read and easy to understand
- It is almost more concise to write the test than to describe that scenario with sentences.
What is important to understand is that there are many key benefits to writing such end-to-end tests. First, we can have hundreds of such real-life scenarios translated into scripts that will run in just a few seconds or minutes. Second, we can repeat such exhaustive tests as many times as we want, such as several times per hour if need be. A human being couldn’t possibly repeat as many test runs at that speed without making any mistake. Not to mention that such repetitive tasks are extremely boring for a human being to do.
Is E2E a lot of work?
End to end testing requires an upfront investment in terms of time and knowledge. That said, it is an investment that will pay off very quickly. With such tests, we can catch bugs and regressions before making their way to a production environment, which is always costly in terms of company image, developer stress, and overall development team productivity.
For instance, compared to unit-tests, end-to-end tests cover a lot more code from the start as they need to run the entire application. A few end-to-end tests can achieve 60 to 80% of code coverage with minimal effort. Getting similar numbers with unit tests would require hundreds of them and a lot more code being written.
The real difficulty lies in providing a stable, dedicated environment for your end-to-end tests. Failure to do so results in flaky tests that randomly fail as your database gets updated by someone else.
Is there any way to record test scenarios instead of writing code?
TestProject is one of the few end-to-end testing technologies that comes with a Selenium AI-powered test recorder (in addition to their open source OpenSDK supporting Java, Python and C#). Using that recorder, you can quickly generate lots of different test scenarios by interacting with your user interface just like a regular user would.
While a developer might not need a recording tool, a Q&A person is likely to welcome such a feature. Using a recorder is more productive than having to learn a new software development language and library. Also, writing code can be a scary endeavor for people not familiar at all with software development.
My team does manual Q&A. How would end-to-end tests help?
While manual tests are always needed, they are never fully exhaustive. Automation testing enables the repeatability of hundreds of tests. It also enables the ability to translate complex testing scenarios into simple, repeatable steps.
Manual Q&A usually focuses on testing new features. Regressions are much harder to detect manually because they’re usually unexpected. This means that a change in the header of a website might get the attention of a Q&A tester on that header only, while a regression might have happened in a completely different place on that website.
Speed is also an important factor to consider. Automated end-to-end tests can validate hundreds of scenarios in a few seconds or minutes. Human beings can’t possibly be that fast and are also subject to making mistakes. One wrong click is all it takes to derail a testing scenario. From that perspective, an end-to-end test is going to be a lot faster and a lot more stable. After all, we invented machines to work on complex, repetitive, and boring tasks. End-to-end automated tests are no exception there.
Conclusion
While there is no silver bullet or one-size-fits-all solution in software development, end-to-end testing is definitely an area that is growing extremely quickly. The tools at our disposal are better than ever, the learning curve is getting shorter, and the developer/tester experience is definitely going in the right direction. As we saw in this article, there are several benefits to end-to-end testing. Teams are more productive. They detect regressions and issues early rather than when the code goes to production. As a result, both confidence and morale increase, which results in less stress and fewer emergencies to work on at a later stage.
I certainly hope that this introduction to end-to-end testing sparked some curiosity and excitement for you to give it a try ✨ Happy E2E Testing!