9 Desktop Test Automation Tools That Won’t Cost a Penny (Updated 2020)

Deciding on the right testing tool for a desktop app might be a difficult task. Let’s dive into the pros and cons of some of the popular open source tools for testing desktop apps.

Pros and Cons of  Top 9 Open Source Test Automation Tools for Desktop Applications

1. WinAppDriver
2. AutoIT
3. Winium
4. TestStack.White
5. LDTP
6. Sikuli
7. Pywinauto
8. Robot Class 
9. Jubula

---

WinAppDriver

WinAppDriver (short for Windows Application Driver) is a free test automation tool for Windows desktop apps developed by Microsoft. It can run as a standalone as well as a plugin for Appium. Thus, if you are already running Appium’s server, then a request for Win10 app automation will launch WinAppDriver.exe.

Pros

• It’s developed by Microsoft (which in itself is awesome!), and it makes perfect sense to test Windows desktop apps using a tool from the same company.
• It uses WebDriver protocol which means that if you’re from the web/mobile automation world and you already know how to develop using the WebDriver environment, then the transition to WinAppDriver will be easy, quick and clear for you. 

Cons

• WinAppDriver is a free tool, yet currently its code is not open source. Samples, tests and related tools are open source, and it’s mentioned that they are considering the option to open source the WinAppDrive code.

Bottom Line:  A fast-growing tool that will provide you with better support for using Appium to test Windows apps.

AutoIT

AutoIT is a standalone (doesn’t require any configuration) and small footprint tool, that simulates mouse and keyboard clicks . It activates the binary files of the tested app using a Reflection.

The AutoIT comes with dedicated IDE, and is compatible with recordings and coding in its own scripting language (very similar to BASIC syntax).

Pros

• Allows exporting an Executable file which we can activate independently outside the project (might be very convenient in case of 3^rd party tools integration and CI).
• Supports Regular Expression.
• The possibility to develop dedicated addons for dedicated needs (with a version called– ‘AutoItX’).

Cons

• Based on a unique coding language (that requires learning).

Bottom line: No wonder AutoIT is one of the most popular tools in its field.

Winium

Winium is a new open source framework, that’s based on Selenium and consists of 3 parts:

1.  Supports test automation on Windows applications.
2.  Supports test automation on Windows Phone operating system context (home button, notifications bar, toggles etc.)
3.  Supports test automation on Windows Phone applications.

Pros

• In case you are already using Selenium WebDriver and familiar with its API, using  Winium is straightforward.

Cons

• As a result of it’s being a new project, few struggles come in the way. Its API still isn’t complete and not on the same maturity level as other tools in this category.

Bottom line: In case you are considering Winium, my advice is to wait until it’s more baked.

 

TestStack.White

White is a library for automation of desktop apps. It started as a small open source project and then became a part of TestStack which consists of a variety of open source code projects for automated and manual testing.

White supports a variety of automation technologies: Silverlight, WPF, WinForms, Win32 and SWT in Java. It’s possible to write White tests in any language supported by .NET.

Pros

• Allows Automation on a variety of technologies.
• Supports all .NET programming languages.

Cons

• Requires installation of UI Automation Verify library.
• It is deprecated – no longer actively maintained.

Bottom Line: One strength among many is the impressive API that is highly extensive and simple to use.

 

LDTP (Linux Desktop Testing Project)

Although the project started for Linux, nowadays it supports MAC (PyATOM version) and also Windows OS (Cobra version). The LDTP comes along with an Editor of its own and among other activities it also supports recordings.

Pros

• Its library is very rich and is compatible with a wide range of environments/ languages compared to other tools in this list. LDTP allows performance monitoring (Memory and CPU) while executing tests on an application.

Cons

• There isn’t enough documentation, many pages are broken, which makes learning API much more difficult.

Bottom line: An amazingly capable tool, but has disadvantages that lower its rank because it is immensely hard to learn.

 

Sikuli

What makes Sikuli exceptional is the object recognition by images, while other tools in the list recognize the objects by objects properties. Clearly, there are advantages and disadvantages for both methods. The image recognition is performed by an background engine that is  capable of comparing a pixel to a pixel at BitMap level.

When working with Sikuli, initially we’ll form a Repository of pictures (screenshots from the application) which will become the inspiration (Expected Result) for determining points and rectification which will be activated while running the program (Actual Result).

The tool comes with an IDE of its own, which includes a unique scripting language and an API to which it’s possible to connect in Java and C#. Additionally, Sikuli offers visualization and functional tests.

Pros

• Since the technique of comparing pictures doesn’t rely on object properties, Sikuli can interpret everything we’ll create – while executing one test, it is possible to check a DB table, a Windows window, also a mobile application and a response from the server without dedication any efforts or special configurations.

Cons

• Constant maintenance of the existing pictures stock for each product change.

Bottom line: The recommendation is to implement this tool in specific occasions only.

 

Pywinauto

The PyWinAuto is a Python library that provides a collection of functions that make operations on Windows (controls and windows dialogs). The library presents a wide set of operations, is clear and user friendly.

Pros

• Pywinauto is library, meaning it doesn’t require installations of any external tools.
• Familiarity with Python, simplifies the implementation of this library.

Cons

• Those who don’t already use Python,  will need to open a new Python project.

Bottom line: A library with a variety of capabilities which is highly recommended especially for Python experienced professionals.

 

Robot Class / Win.form Class

These tools are distinctively different than the rest, in the way that they aren’t directly destined as test automation tools for desktop applications, but act as classes to simulate activities of the keyboard and mouse. Whereas, the Robot Class could be used with  Java and Win.form is for .Net.

Pros

• Solely a class.

Cons

• Solely a class.

Bottom line: We’ll use the class for uncomplicated scenarios only ( for instance, choosing an image from the File System), no QA is needed.

 

Jubula

Jubula is an open-source automated functional GUI testing tool for Java desktop apps. Jubula is part of the Eclipse package and it mostly used for functional testing.

Pros

• Does not require coding effort.
• Supports SWT/RCP/GEF/Swing and HTML apps and works on operation systems Windows, Linux/Unix and Mac.
• Heuristic object recognition.
• Continuous integration by command-line clients.
• The version control is via export/import.

Cons

• Supports only Java.
• A relatively small community that makes it difficult to find documentation or reach out for help.

Bottom line: If you don’t have coding skills or you are not a technical tester and looking for a free tool to create automated scripts for functional testing.

---

Reference