Designing and implementing an automation test strategy can be overwhelming. There are multiple points to consider and if you’ve ever researched the topic online, you know there are many different points of view to think about. How you develop and implement an automation strategy should be unique to your organization’s needs and abilities. In this multi-part blog series, I will provide some in-depth thoughts about how I’ve designed and implemented test strategies, things I’ve learned, and other industry gurus’ thoughts on the topic. At the end of these installments, you’ll feel comfortable enough to design your own automation test strategy!
In this particular post, we will discuss WHY you want to start automating (goals) and WHO will own it (ownership). While those two points seem straightforward, there are elements to consider that may be unique to your circumstances, or points that you may not have considered at all yet.
Table of Contents –Automation Test Strategy: Identify Your Goals and Ownership
- You’re here → Chapter 1 – Identify Your Goals and Ownership
- Chapter 2 – What Should I look For In a Tool?
- Chapter 3 – What Should We Automate
Answering why you want to automate is an important, foundational question. Once you identify this you’ll want to continuously reference the answer throughout this journey. Ensuring you’re implementing techniques or tools that allow you to achieve your goals should always be a priority. Here are some common goals you may consider:
- Faster execution times: Said differently, faster speed to market. The quicker your team can validate code changes, the quicker that code can make it to customers. Automating your test suite can greatly reduce the amount of time it takes to complete testing.
- Improved accuracy: Any time a tester manually executes testing, risk of error is present. We make mistakes and one way to reduce the risk of mistakes is to create scripts of your testing (automation) to allow you to execute the same test every time.
- Better use of testers: Once you have testing covered via automation, your manual test engineers can focus their efforts on complex, human-centric driven testing. Driving this culture shift will improve end-user experiences within your application.
- User load simulation: Having the ability to automate load on your application is imperative to ensuring your systems can handle high peak times. Unfortunately, having manual load added to your application is quite often not enough.
- Quicker feedback loops: Automation, when effectively developed, allows you to get feedback to the developer quickly. If testing does reveal a bug, the developer is still freshly involved in the changes and can quickly fix the issue. Longer feedback loops often leave developers switching context back and forth as the testing results on the original story often happen too far after the development work has concluded.
- Higher-quality code to QA: Designing your automation to execute in a CI/CD pipeline allows it to run when a developer checks their code in. This gives them rapid feedback and confidence before QA even touches it.
- Increased test coverage: Manually executing test cases can limit the amount you’re able to cover during testing. Especially if you’re working with reduced testing cycles. By automating, you’re able to cover more testing right away.
Now that you have the why defined, the next crucial part is the who. Defining the initial who, as well as the long-term who, is important to the success of your test automation strategy. Let’s discuss the areas you’ll need to define a who for, and some options you have for each.
Scripting and Maintenance
There are a few considerations to make as you’re defining who will own the scripting of your tests. You have your short-term needs which are essentially automating your regression suite upfront. In addition, you have your long-term needs such as adding new features to your automation suite, maintaining existing scripts, executions, and validations. Defining who will own this is a decision your organization will need to make together. My personal preference is to have dedicated automation engineers for both short-term and long-term needs. Here are some common options to consider:
- Dedicated automation engineers or SDETs (software developers in test): Having dedicated automation testers will allow you to quickly get your automation suite up and running. These folks are experienced and skilled when it comes to setting up an automation framework and scripting tests. Once your suite is up and running, you’ll want to consider the effort needed to maintain the scripts. Having full-time, dedicated people, to focus on these efforts ensures your automation suite stays stable, reliable, and trustworthy. Please note this option can be costly as you may have to hire specific people for those roles.
- Developers: You may choose to ask your development team to get your automation scripted and running. However, this can take their time away from new feature development. In addition, the time required to keep these scripts maintained and executing smoothly is more than a developer has to give. I would suggest considering that your development team owns the unit testing aspect. It’s the largest group of tests, and the foundation, of the automation testing pyramid. We will review the testing pyramid in a later post, in this series, when discussing what to automate.
- Manual test engineers: Maybe you’ll want to utilize manual testers to complete this task. They’re already on the team, so there would be no additional resource cost. Beware, implementing an automation framework and scripting your regression suite is a task that is time-consuming and takes highly developed skills. Asking someone without those skills to develop them on the fly might be too overwhelming for some. Also remember, one of your goals for implementing automation testing in the first place may be to reduce testing time. Asking someone to not only manually validate changes, but also focus on developing skills for automation and scripting an entire regression suite, may lead to failure in achieving that goal.
Test Execution and Validation
When you first implement an automation framework you won’t typically be implementing that into a CI/CD pipeline. As such, you’ll need to consider who will be responsible for executing the scripts and validating the results. Here are some common options to consider:
- Dedicated automation engineers or SDETs (software developers in test): If you’re choosing to hire dedicated automation engineers for scripting and maintenance, it might make sense for those same people to own the execution and validations of the testing. If they have to feel the pain of validating false failures during testing, that might give them enough motivation to get those fixed quicker.
- Developers: Asking developers to execute testing and validate failures will certainly take them away from their primary goal of developing new features for your customers. In addition, the task of executing and validating failures doesn’t take specific development skills like scripting might. So using them in this capacity may limit the amount of work your teams can get completed. The only exception to this, in my opinion, would be if they owned the unit testing. If those fail, developers should be the ones who confirm the failure.
- Manual test engineers: Having manual testers validate automation test executions can be a good fit for some organizations. Using their skills to dig into the failures and validate them is an extension of their day-to-day testing activities. However, if the person who owns the script isn’t motivated to fix the bad tests, your manual testing team may spend too much time validating false failing tests instead of using their super-skills to exploratory test new features.
After reading this article, you should now feel comfortable having these discussions among your team and identifying the goals you’re trying to accomplish on your journey to create an automation test strategy. In addition to the goals, make sure you’re agreeing on who will own specific areas such as scripting, maintenance, and failure validations. Identifying this in detail upfront will save you time and headache in the future.