Automation Test Strategy: Identify Your Goals and Ownership

Designing and implementing an automation test strategy can be overwhelming. There are multiple points to consider and if you’ve ever researched the topic online, you know there are many different points of view to think about. How you develop and implement an automation strategy should be unique to your organization’s needs and abilities.

In this multi-part blog series, I will provide some in-depth thoughts about how I’ve designed and implemented test strategies, things I’ve learned, and other industry gurus’ thoughts on the topic. At the end of these installments, you’ll feel comfortable enough to design your own automation test strategy!

In this particular post, we will discuss WHY you want to start automating (goals) and WHO will own it (ownership). While those two points seem straightforward, there are elements to consider that may be unique to your circumstances or points that you may not have considered at all yet.

Table of Contents – Automation Test Strategy: Identify Your Goals and Ownership

You’re here → Chapter 1 – Identify Your Goals and Ownership
Chapter 2 – What Should I look For In a Tool?
Chapter 3 – What Should We Automate

Identifying Your Automation Goals

Answering why you want to automate is an important, foundational question. Once you identify this you’ll want to continuously reference the answer throughout this journey. Ensuring you’re implementing techniques or tools that allow you to achieve your goals should always be a priority. Here are some common goals you may consider:

Faster execution times: or, faster speed to market. The quicker your team can validate code changes, the quicker that code can make it to customers. Automating your test suite can greatly reduce the amount of time it takes to complete testing.
Improved accuracy: Any time a tester manually executes testing, a risk of error is present. We make mistakes and one way to reduce the risk of mistakes is to create scripts of your testing (automation) to allow you to execute the same test every time.
Better use of testers: Once you have testing covered via automation, your manual test engineers can focus their efforts on complex, human-centric driven testing. Driving this culture shift will improve end-user experiences within your application.
User load simulation: Having the ability to automate load on your application is imperative to ensuring your systems can handle high peak times. Unfortunately, having manual load added to your application is quite often not enough.
Quicker feedback loops: Automation, when effectively developed, allows you to get feedback to the developer quickly. If testing does reveal a bug, the developer is still freshly involved in the changes and can quickly fix the issue. Longer feedback loops often leave developers switching context back and forth as the testing results on the original story often happen too far after the development work has concluded.
Higher-quality code to QA: Designing your automation to execute in a CI/CD pipeline allows it to run when a developer checks their code in. This gives them rapid feedback and confidence before QA even touches it.
Increased test coverage: Manually executing test cases can limit the amount you’re able to cover during testing, especially if you’re working with reduced testing cycles. By automating, you’re able to cover more testing right away.

Identifying the WHO

Now that you have the why defined, the next crucial part is the who. Defining the initial who, as well as the long-term who is important to the success of your test automation strategy. Let’s discuss the areas you’ll need to define a who for and some options you have for each.

Scripting and Maintenance

There are a few considerations to make as you’re defining who will own the scripting of your tests. You have short-term needs which are essentially automating your regression suite upfront. In addition, you have your long-term needs such as adding new features to your automation suite, maintaining existing scripts, executions, and validations.

Defining who will own this is a decision your organization will need to make together. My personal preference is to have dedicated automation engineers for both short-term and long-term needs. Here are some common options to consider:

Dedicated automation engineers or SDETs (software developers in test): Having dedicated automation testers will allow you to quickly get your automation suite up and running. These folks are experienced and skilled when it comes to setting up an automation framework and scripting tests.
Once your suite is up and running, you’ll want to consider the effort needed to maintain the scripts. Having full-time, dedicated people to focus on these efforts ensures your automation suite stays stable, reliable, and trustworthy. Please note this option can be costly as you may have to hire specific people for such roles.
Developers: You may choose to ask your development team to get your automation scripted and running. However, this can take their time away from new feature development. In addition, the time required to keep these scripts maintained and executing smoothly is more than a developer has to give.
I would suggest considering that your development team owns the unit testing aspect. It’s the largest group of tests and the foundation of the automation testing pyramid. We will review the testing pyramid in a later post in this series when discussing what to automate.
Manual test engineers: Maybe you’ll want to utilize manual testers to complete this task. They’re already on the team, so there would be no additional resource cost. Beware though, implementing an automation framework and scripting your regression suite is a task that is time-consuming and takes highly developed skills.
Asking someone without those skills to develop them on the fly might be too overwhelming for some. Also remember, one of your goals for implementing automation testing in the first place may be to reduce testing time.Asking someone to not only manually validate changes but also focus on developing skills for automation and scripting an entire regression suite may lead to unexpected difficulty and possibly even failure in achieving that goal.

Test Execution and Validation

When you first implement an automation framework, you won’t typically be implementing that into a CI/CD pipeline. As such, you’ll need to consider who will be responsible for executing the scripts and validating the results. Here are some common options to consider:

Dedicated automation engineers or SDETs (software developers in test): If you’re choosing to hire dedicated automation engineers for scripting and maintenance, it might make sense for those same people to own the execution and validations of the testing.
If they have to feel the pain of validating false failures during testing, that might give them enough motivation to get those fixed quicker.
Developers: Asking developers to execute testing and validate failures will certainly take them away from their primary goal of developing new features for your customers. In addition, the task of executing and validating failures doesn’t take into account specific development skills like scripting might.
So using them in this capacity may limit the amount of work your teams can get completed. The only exception to this, in my opinion, would be if they owned the unit testing. If those fail, developers should be the ones who confirm the failure.
Manual test engineers: Having manual testers validate automation test executions can be a good fit for some organizations. Using their skills to dig into the failures and validate them is an extension of their day-to-day testing activities.
However, if the person who owns the script isn’t motivated to fix the bad tests, your manual testing team may spend too much time validating false failing tests instead of using their super-skills to explore and test new features.

Automation Testing Process

Selecting Testing tools

Successful test automation depends on selecting the appropriate test automation tool. By selecting the right test automation tools, you can efficiently achieve the target of your project. There are some important steps that should be followed while selecting the automation tool:

Select the tool according to your project requirements.
Evaluate the tool.
A cost-benefit analysis should be performed.
The final step is to have a good knowledge of how the tool is used.

Determining Automation Scope

The Scope of Automation testing is where your application is under test to automate its features. The scope of the automation can be determined in the following ways;

To know about the functionalities of your project.
Automation testing is applicable for applications that have a large number of data.
The complexity of test cases.

Planning, Designing, and Development

During this phase, you can decide your planning, designing, and development of your automation test scripts which contains the following details:

Designing of frameworks and their features.
Automation Test environment setup.
Maintaining the timeline for execution.
Delivering the automation test scripts.

Executing Tests

Automation test scripts are executed in this phase. It is the process of executing the code and checking the expected results by comparing them with the actual results. Test automation scripts are executed through the automation tools.

Maintenance and Monitoring

During this phase, you can monitor whether the newly implemented functionalities added to the applications are working or not. The maintenance approach is followed when new automation test scripts are added with new features of the application.

Benefits of Using Automation Testing

In the software testing process, automation testing has a lot of benefits than manual testing. The benefits of automation testing are as follows;

It increases the productivity of test automation team members.
Its cost-efficiency.
Automated tests are time-saving and deliver fast execution of test scripts.
Higher test coverage.
Quickly determines the stability of your build.

Conclusion

After reading this article, you should now feel comfortable having these discussions among your team and identifying the goals you’re trying to accomplish on your journey to create an automation test strategy. In addition to the goals, make sure you’re agreeing on who will own specific areas such as scripting, maintenance, and failure validations. Identifying this in detail upfront will save you time and headaches in the future.