This article has been updated on January 21st 2020.
Deciding on the right testing tool for a desktop app, might be a difficult task. These are the-
Pros and Cons of 9 different open source desktop test automation tools for desktop applications, written in WinForms/ WPF:
WinAppDriver (short for Windows Application Driver) is a free test automation tool for Windows desktop apps developed by Microsoft. It can run as a standalone as well as a plugin for Appium. Thus, if you are already running Appium’s server, then a request for Win10 app automation will launch WinAppDriver.exe.
- It’s developed by Microsoft (which in itself is awesome!), and it makes perfect sense to test Windows desktop apps using a tool from the same company.
- It uses WebDriver protocol which means that if you’re from the web/mobile automation world and you already know how to develop using the WebDriver environment, then the transition to WinAppDriver will be easy, quick and clear for you.
- WinAppDriver is a free tool, yet currently its code is not open source. Samples, tests and related tools are open source, and it’s mentioned that they are considering the option to open source the WinAppDrive code.
Bottom Line: A fast-growing tool that will that will provide you with better support for using Appium to test Windows apps.
AutoIT is a standalone (doesn’t require any configuration) and small footprint tool, that simulates mouse and keyboard clicks . It activates the binary files of the tested app using a Reflection.
The AutoIT comes with dedicated IDE, and is compatible with recordings and coding in its own scripting language (very similar to BASIC syntax).
- Allows exporting an Executable file which we can activate independently outside the project (might be very convenient in case of 3rd party tools integration and CI).
- Supports Regular Expression.
- The possibility to develop dedicated addons for dedicated needs (with a version called– ‘AutoItX’).
Based on a unique coding language (that requires learning).
Bottom line: No wonder AutoIT is one of the most popular tools in its field.
Winium is a new open source framework, that’s based on Selenium and consists of 3 parts:
- Supports test automation on Windows applications.
- Supports test automation on Windows Phone operating system context (home button, notifications bar, toggles etc.)
- Supports test automation on Windows Phone applications.
In case you are already using Selenium WebDriver and familiar with its API, using Winium is straightforward.
As a result of it’s being a new project, few struggles come in the way. Its API still isn’t complete and not on the same maturity level as other tools in this category.
Bottom line: In case you are considering Winium, my advice is to wait until it’s more baked.
White is a library for automation of desktop apps. It started as a small open source project and then became a part of TestStack which consists of a variety of open source code projects for automated and manual testing.
White supports a variety of automation technologies: Silverlight, WPF, WinForms, Win32 and SWT in Java. It’s possible to write White tests in any language supported by .NET.
- Allows Automation on a variety of technologies.
- Supports all .NET programming languages.
Requires installation of UI Automation Verify library.
Bottom Line: One strength among many is the impressive API that is highly extensive and simple to use.
LDTP (Linux Desktop Testing Project)
Although the project started for Linux, nowadays it supports MAC (PyATOM version) and also Windows OS (Cobra version). The LDTP comes along with an Editor of its own and among other activities it also supports recordings.
Its library is very rich and is compatible with a wide range of environments/ languages compared to other tools in this list. LDTP allows performance monitoring (Memory and CPU) while executing tests on an application.
There isn’t enough documentation, many pages are broken, which makes learning API much more difficult.
Bottom line: An amazingly capable tool, but has disadvantages that lower its rank because it is immensely hard to learn.
What makes Sikuli exceptional is the object recognition by images , while other tools in the list recognize the objects by objects properties. Clearly, there are advantages and disadvantages for both methods. The image recognition is performed by an background engine that is capable of comparing a pixel to a pixel at BitMap level.
When working with Sikuli, initially we’ll form a Repository of pictures (screenshots from the application) which will become the inspiration (Expected Result) for determining points and rectification which will be activated while running the program (Actual Result).
The tool comes with an IDE of its own, which includes a unique scripting language and an API to which it’s possible to connect in Java and C#. Additionally, Sikuli offers visualization and functional tests.
Since the technique of comparing pictures doesn’t rely on object properties, Sikuli can interpret everything we’ll create – while executing one test, it is possible to check a DB table, a Windows window, also a mobile application and a response from the server without dedication any efforts or special configurations.
Constant maintenance of the existing pictures stock for each product change.
Bottom line: The recommendation is to implement this tool in specific occasions only.
The PyWinAuto is a Python library that provides a collection of functions that make operations on Windows (controls and windows dialogs). The library presents a wide set of operations, is clear and user friendly.
- Pywinauto is library, meaning it doesn’t require installations of any external tools.
- Familiarity with Python, simplifies the implementation of this library.
Those who don’t already use Python, will need to open a new Python project.
Bottom line: A library with a variety of capabilities which is highly recommended especially for Python experienced professionals.
Robot Class / Win.form Class
These tools are distinctively different than the rest, in the way that they aren’t directly destined as test automation tools for desktop applications, but act as classes to simulate activities of the keyboard and mouse. Whereas, the Robot Class could be used with Java and Win.form is for .Net.
Pros: Solely a class
Cons: Solely a class
Bottom line: We’ll use the class for uncomplicated scenarios only ( for instance, choosing an image from the File System), no QA is needed.