KDT (keyword driven testing) Vs. Test Automation Scripts

KDT (Keyword-driven testing) has been around for over two decades, surprisingly it is still being presented as cutting edge by some of the testing services providers. In a matter of fact, some enterprises develop proprietary KDT frameworks for years, and possess KDT solutions with vast keyword test and functional capabilities. Unfortunately, they aren’t open source, which prevents them from becoming a good alternative to keyword driven testing solutions available today.

Another interesting fact is that only lately the KDT methodology was presented as a part of the test automation solution by premium tool vendors. It definitely indicates that KDT is a well-recognized testing methodology and it has won an honorable place in the testing hall of fame. But does it meet the target and do testers use it effectively without scripted test automation? Doubtful.

Before diving any deeper, let’s explain what exactly KDT is. The idea behind the keyword driven testing approach is the ability for test engineers to design test cases using keywords that actually represent actions. This is done by implementing an abstract logical layer on top of an automation core framework – i.e. a set of functions & methods. In some cases the core framework is responsible for the actual execution, while in others it interacts with 3rd party APIs and tools. KDT architecture allows test engineers to create different variations of automated test cases using fixed set of keywords, without requiring any coding skills. The result is exponential growth in automated test cases and testing coverage.

In most cases, KDT frameworks are based on three major factors:

1. Vocabulary (keywords and phrases) based on the needs and terminology, sometime called Business Readable DSL (Domain Specific Language).
2. A parser (compiler), which is responsible for the keywords aggregation, syntax checking and data driven. Usually the input is based on a known format such as CSV, XLS, XML, Dynamic SQL and more.
3. Automation core framework – a set of generic service functions and element based functions, which is responsible for the test execution and reporting. In some case interacts with 3rd party tools for the actual execution.

A typical KDT test will look like this:

Keyword	Object	Value
Open Browser	Chrome	www.google.com
Set Text	Search TextBox	“What is KDT?”
Press Button	Search Button

For more than one reason, KDT automation won’t be the default choice for many automation engineers. The required effort to establish an effective KDT environment is huge, and in most cases requires a dedicated team which might not be trivial for the common R&D organizations. In addition, the learning curve for proprietary vocabulary (keywords and phrases) will most likely be the same as learning a new scripting language (Python/Javascript/VBS). In fact, learning proprietary keywords can be more complicated in times, and be less flexible than an existing scripting language.

There are a lot of discussions online, both Pros and Cons, for the KDT approach. Below are some of the key factors that should be considered before implementing one of the two approaches (KDT vs. scripting):

Factor	KDT	Scripting
Maintenance	Relatively low (in the long run)	May grow (in the long run)
Skills and Expertise	Test cases may be written by a manual QA engineer.	Requires knowledge of the used scripting language, which usually forces the tests to be written by an experienced automation engineer.
Change request	Test cases are easy to modify	Test cases can take time to modify (depends on the complexity of the script)
Scalability	Allows exponential growth by design.	Depends on framework & test scripts design.
Investment	Huge initial investment	Some investment in test automation infrastructure (e.g. generic service functions).
Delivery time	Initial test cases delivery time is longer in KDT framework, since it takes time to establish. Afterwards it can even be more than double over scripting.	Test cases are written immediately.
Training	Ramp up time for a new QA engineer may take more than usual (Proprietary keywords)	Ramp up time will be up to test automation engineer skills, the test scripts implemented in standard scripting languages

Eventually, you may ask, which of the two approaches is better?

There is no definite answer – it depends on multiple factors:

• Test automation goals
• Product life cycle and timelines
• Stakeholders skills and organization culture
• Budget

As a rule of thumb, KDT will probably fit enterprises, while automation scripting small-to-medium companies & startups (which are more agile by nature). 

General guidelines:

1. When considering a KDT framework, you should check open source solutions such as cucumber combined with Appium or Selenium. The alternative can be a commercial solution which already implements KDT principles such as UFT or TestComplete KDT. In any case developing a KDT framework from scratch probably won’t be a wise idea.
2. When establishing a proprietary framework for headless or API test automation, using a scripting language (Python/Javascript/VBS) should be considered as the KDT base for test business logic, and a higher level programming language (C++/C#/Java) as the core automation framework for test execution. By separating the two technologies, it’s easier to overcome the majority of the problems in the test automation world.
3. If agile development and continues integration is a factor, KDT is probably not an optimal choice (unless the framework is there to begin with). Using tools such as Microsoft TFS, JUnit or Robot Framework with the combination of Jenkins or Bamboo as the build/CI servers will usually prove as a better solution.
4. Planning is important. It’s crucial to define what the automation targets are, what to automate and who will take part in it.