Announcing TestProject X – Parallel Execution & Version Control

TestProject 2.0 NextGen was just a minute ago, but we’re back with another major release: TestProject X (v3.0) 🎈 This exciting version breaks some of the biggest barriers in test automation today 🔨 As it provides two of the most anticipated feature requests that came from you – our amazing community 🌟

Starting from TestProject 3.x you now have the following powerful capabilities:

1. Parallel execution on a single Agent, across multiple targets, without any Docker or Selenium Grid required!
2. Version control using built-in GitHub integration, to seamlessly connect TestProject with your GitHub repository and keep track of your test history.

🔽  Get TestProject X Today 🔽

This new TestProject version enables you to drastically reduce time to market and deliver high-quality software at speed, no matter what’s your skill set. Finally, everyone can easily achieve fast release cycles (beginners as well as experts!). TestProject v3.0 aligns your testing teams to fast-paced agile development life cycles and enhances whole-team collaboration between testers and developers.

Table of Contents – TestProject X New Features (version 3.0)

1. Live Announcement Event [Webinar recording & presentation slides]
2. Parallel Execution on a Single Agent
   • What is Parallel Execution and its Benefits?
   • Parallel Execution vs. Serial Execution
   • Parallel Execution Solutions: Selenium Grid vs. Docker vs. TestProject X
3. GitHub Integration
   • Example CI flow with TestProject-GitHub Integration
4. Take TestProject X for a Spin

---

Live Announcement Event

The big TestProject X reveal webinar on June 22nd was on FIRE 🔥 It was super exciting to share it with you & your feedback was incredible! Get the presentation slides here and check out the full recording below!

📌 For those of you who missed it or for those of you who attended but still have more questions – we’re hosting a live Q&A session all about TestProject X on June 29th. Save your spot and get a chance to ask us anything! 🧐

Parallel Execution on a Single Agent

Before the release of TestProject 3.0, each Agent could run a single Test or a Job at a time (also known as serial execution). When an additional Test or a Job was submitted, it was placed into a queue and sent to the Agent after the last reported that it’s idle.

Starting TestProject 3.x, a single Agent can execute more than one Test or Job in parallel (also known as parallel execution). This can be either multiple tests on one or more browsers or devices, multiple serial Jobs, or multiple parallel Jobs. This all occurs on-premise and without any complex configuration of 3rd party tools (such as Selenium Grid or Dockers) and without any 3rd party integrations to device farms (Sauce Labs/BrowserStack).

Besides being able to run multiple Tests and Jobs in parallel, TestProject provides multiple parallelization options:

1. For “ad-hoc” tests execution: Targets – Browsers or Devices (including virtual)
2. For Jobs:
   • Targets – Browsers or Devices (including virtual)
   • Tests & Data-Driven Tests

What is Parallel Execution and its Benefits?

Parallel execution is a test automation method where test cases are simultaneously run on multiple combinations of browsers or mobile devices (including Android emulators and iOS Simulators). Parallel testing increases the testing coverage and allows faster time to market. With this parallel execution capability now introduced in TestProject v3.0 you can benefit from the following, using just a single Agent, all 100% free:

• Increase testing speed and release versions more frequently.
• Reduce the cost of creating and maintaining infrastructure.
• Cover more targets (browsers/devices) on every cycle.
• Run multiple data-driven cycles in less time.
• Simplify integration into existing CI/CD pipelines.
• Test on-premise, yet allow scalability.
• Avoid Dependencies between Tests

In TestProject a parallel task on a single Agent is called Agent Worker. It can be a test/job execution or a recording session. Running tests and jobs in parallel requires additional resources and capabilities from the machine they are running on. On a standard desktop configuration you will be able to execute 3-4 workers simultaneously, while on a high-end desktop configuration you can execute 8-10 workers, which can speed up the testing process anywhere between 3x to 10x!

Parallel Execution vs. Serial Execution

To fully comprehend the power of parallel execution, let’s also understand the difference between parallel execution vs. serial execution. Serial execution is a method where tests run one after another, meaning that a previous test must finish before another one gets started. When a test execution time on average is 1 minute, executing 10 tests, with different parameter values (data-driven testing), will require 10 minutes! So, you can only imagine what happens when you have thousands of tests, right? 😨

❌ Serial Execution: 1000 tests * 3 browsers * 10 data-driven iterations = 20 days!

VS.

⚡ Parallel Execution: 1000 tests * 3 browsers * 10 data-driven iterations = Only 2 days!
(when using 10 workers for example)

Meaning that with parallel execution the time spent can be up to 10x less than serial execution (when using 10 workers for example)! Using serial testing method makes it impossible to achieve complete coverage.

Luckily TestProject Agent 3.x acts as a worker lab, providing the power to execute tests in parallel on a single Agent, saving up a BUNCH of time! 🎉

• DevOps free
• On-premise
• Small footprint
• Without Docker
• Without Selenium Grid
• Simple and 100% free!

To dive even deeper and see how easy it is to get started, here‘s a detailed documentation for parallel execution with TestProject.

Parallel Execution Solutions: Selenium Grid vs. Docker vs. TestProject X

Parallel test execution doesn’t come easy for all, thus causing the effort to scale Selenium or Appium automated tests very challenging at times. Spinning up Docker containers, configuring Selenium Grid, running Kubernetes cluster… 😵 But finally, with TestProject X you can overcome these test automation barriers and push versions at speed.

Below is a comparison between Selenium Grid vs. Docker vs. TestProject 3.x:

• Setup Time: With Selenium Grid the setup time is quite long since you have to install Java, setup the Hub, the Nodes. The same goes for Docker, where you also have to invest a lot of time since it requires to setup Docker containers & using an orchestrator (i.e Kubernetes). However, with TestProject the setup time is just a few minutes – all you need to do is download and install the Agent, and that’s it!
• Complexity: Both Selenium Grid and Docker require advanced knowledge and a high set of skillset to set them up. With TestProject anyone can get started and you don’t need any preliminary knowledge before getting started.
• Velocity: The velocity in Selenium Grid is limited by the number of nodes that you configure, and in Docker it’s limited by the number of containers. In TestProject the only limitation is your own hardware, nothing else.
• Skills: With Selenium Grid and Docker you need to have a pretty high skill set, whereas with TestProject everything is already bundled up for you and we took care of all the heavy lifting for you so you can get started right away with parallel testing.
• Maintenance: There is high maintenance required for both Selenium Grid and Docker, you need to invest resources and capabilities to maintain their infrastructures. However, in TestProject there are no maintenance hassles. We take care of keeping the Agent always up to date and all the required capabilities come out-of-the-box bundled into the TestProject Agent.
• Mobile: Selenium Grid can run Appium mobile tests, with Docker it’s a bit limited and possible only with Android. With TestProject you have it all – parallel testing for Android, iOS and even iOS on Windows!
• Web (Browsers): Selenium Grid obviously supports web browsers, with Docker it’s quite limited, and with TestProject you can run tests in parallel on any major browser of your choice.
• Headless / UI: Headless and UI options are possible in both Selenium Grid and TestProject, whereas with Docker it’s partially possible because Dockers are headless dockers and usually limited to headless browsers.

GitHub Integration

In addition to the parallel execution capabilities, TestProject X also includes a brand new GitHub integration, enabling you to now benefit from version control and access previous versions of your tests. Each change that you make in your test, will automatically be stored and documented so you can seamlessly keep track of your progress.

With this integration you will be able to:

• Synchronize entire projects with GitHub
• Seamlessly keep and access test history
• Run any previous test versions using TestProject CLI
• Import various test versions back into TestProject
• See what has changed in your tests directly on GitHub
• Integrate your CI flow directly into GitHub repos

To dive even deeper and see how easy it is to get started, here‘s a detailed documentation for connecting TestProject with GitHub.

Example CI flow with TestProject-GitHub Integration