In our daily work, we rarely have the luxury of focusing on one task at a time, we must be able to handle multiple priorities & parallelize between a bunch of tasks. You too are most likely multitasking much of the time in order to keep up with the modern world and deliver great products at speed 🏃♀️
Since we’re used to parallelizing EVERYTHING from professional work to our personal lives, parallel execution could be the cure 🧪 In this article, we will discuss the power of parallel testing and how to achieve it using the following parallel execution solutions: Selenium Grid, Docker, or TestProject 💪 Let’s drill-down into the advantages and disadvantages of each solution and figure out which one will be the perfect fit to our needs.
Parallel Testing in a Nutshell
Parallel testing is a test method that allows test cases to run simultaneously on multiple combinations of browsers or mobile devices (Physical devices, Android emulators and iOS Simulators). There are many parallel testing benefits, the main one being that it helps us move faster in our overall test automation execution time & thus allows for faster time to market. However, it’s important to remember that in order to perform parallel testing well, there are some barriers we must overcome in order to guarantee quality & simple experiences.
Parallel Execution IN 👍 Serial Execution OUT 👎
When it comes to serial execution vs. parallel execution – there’s no doubt that parallel testing outshines the first by far, helping testing teams achieve faster time to market while increasing the testing coverage – Now, that’s magical 💫 But let’s face it, it doesn’t come easy. Here are 3 parallel execution solutions that will help you achieve the parallel testing speed that everybody is talking about:
We all know & love Selenium, the popular choice of framework among testers for decades. Selenium allows us to perform parallel testing using Selenium Grid, which specializes in running multiple tests across different browsers. Selenium Grid is a smart proxy server that uses routing commands to remote web browser instances, where one server acts as the hub. This hub routes test commands that are in JSON format to multiple registered Grid nodes. To dive deeper into Selenium Grid’s new architecture, check out Rex Jones II’s article about Selenium 4 new features.
Using Docker is another solution to achieve parallel test execution. Docker is the de-facto standard to build, run and share containerized apps from your desktop to the cloud. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.
However, the effort to scale Selenium or Appium automated tests is challenging. Spinning up Docker containers, configuring Selenium Grid, running Kubernetes cluster… 😵 There are a few barriers we need to take into consideration and see how to overcome them, or maybe there is an alternative way❓
Before the release of TestProject 3.0, each Agent could run a single Test or a Job at a time. When an additional Test or a Job was submitted, it was placed into a queue and sent to the Agent after the last reported that it’s idle. Starting TestProject 3.x, a single Agent can execute more than one Test or Job in parallel. This can be either multiple tests on one or more browsers or devices, multiple serial Jobs, or multiple parallel Jobs. This all occurs on-premise and without any complex configuration of 3rd party tools (such as Selenium Grid or Dockers) and without any 3rd party integrations to device farms (Sauce Labs/BrowserStack).
To dive even deeper and see how easy it is to get started, here‘s a detailed documentation for parallel execution with TestProject.
Parallel Execution Solutions: Selenium Grid vs. Docker vs. TestProject
Now that we’ve covered these 3 parallel execution solutions individually, let’s dive into this comparison I’ve conducted between Selenium Grid vs. Docker vs. TestProject:
- Setup Time: With Selenium Grid the setup time is quite long since you have to install Java, setup the Hub, the Nodes. The same goes for Docker, where you also have to invest a lot of time since it requires to setup Docker containers & using an orchestrator (i.e Kubernetes). However, with TestProject the setup time is just a few minutes – all you need to do is download and install the Agent, and that’s it!
- Complexity: Both Selenium Grid and Docker require advanced knowledge and a high set of skillset to set them up. With TestProject anyone can get started and you don’t need any preliminary knowledge before getting started.
- Velocity: The velocity in Selenium Grid is limited by the number of nodes that you configure, and in Docker it’s limited by the number of containers. In TestProject the only limitation is your own hardware, nothing else.
- Skills: With Selenium Grid and Docker you need to have a pretty high skill set, whereas with TestProject everything is already bundled up for you and we took care of all the heavy lifting for you so you can get started right away with parallel testing.
- Maintenance: There is high maintenance required for both Selenium Grid and Docker, you need to invest resources and capabilities to maintain their infrastructures. However, in TestProject there are no maintenance hassles. We take care of keeping the Agent always up to date and all the required capabilities come out-of-the-box bundled into the TestProject Agent.
- Mobile: Selenium Grid can run Appium mobile tests, with Docker it’s a bit limited and possible only with Android. With TestProject you have it all – parallel testing for Android, iOS and even iOS on Windows!
- Web (Browsers): Selenium Grid obviously supports web browsers, with Docker it’s quite limited, and with TestProject you can run tests in parallel on any major browser of your choice.
- Headless / UI: Headless and UI options are possible in both Selenium Grid and TestProject, whereas with Docker it’s partially possible because Dockers are headless dockers and usually limited to headless browsers. But finally, with TestProject X you can overcome these test automation barriers and push versions at speed.
I hope you find this comparison practical to be able to choose wisely the best solution for your requirements and skillsets available within your testing team. No matter which parallel testing route you select, it’s crucial for creating a smoother and more flexible work process, with minimum complex configurations & maintenance efforts, and maximum quality delivered at speed.
What’s your go-to solution for parallel testing? 🧐