When you are starting your test automation effort for a brand new project, you need to pick the right tool to perform automation with. Doing so requires a thorough analysis, where you weigh the pros and cons of a list of proposed frameworks 🔍
Of course, you sometimes choose based on unproven information, since you might have never worked with some of these tools before.
Analysis is a great way to differentiate between possible tools to use, but simply doing this analysis cannot be the sole basis for selecting the right one for your project. To prove that a promising tool can be used successfully for your testing, you should build a Proof of Concept (POC).
Decide what test scenarios to automate
For this purpose, you will need to select a handful of promising tools and start with the one that looks best for your project, from an analysis perspective. Apart from this, you also need to select a set of test scenarios you want to automate.
The goal is to evaluate how capable the selected tool is to automate the proposed scenarios. Along the way, there will be quite a few lessons learned 🧐
First of all, how will you choose what to automate? Think about your product. What are the most critical scenarios you want to automate? What are the most complicated scenarios that need automation? And what are those scenarios where certain, more difficult to test technologies are used in the product implementation?
Pick a few of each of these scenarios. The goal is to automate all the selected ones.
Learn how to use the tool chosen for your POC
Now you can get started with the automation. Having picked the most promising tool/framework, you will initially have to learn how to use it, in case you haven’t already. This means you need to read the official documentation and any other helpful resource that can get you to understand how this tool works.
This is another aspect you can further analyze when it comes to the selection of the right tool:
- Does it have good documentation?
- Is it easy to learn?
- Is it similar to anything you have already used?
- Would it be easy to onboard anyone without previous knowledge of this tool?
- Is there knowledge in the company for this programming language, if one is used?
Of course, if the answer to all these questions is ‘yes’, this tool looks even more promising 🔥
The next thing to figure out is how to set up your existing project, or a new one, to support the selected tool. The answer should lie in the documentation. But in some cases, your project has some specific setup, which is so different from the documentation, that you need to figure out by yourself how to integrate this tool in your project. This again means reading, learning, research, and trying out different things.
This aspect should also be evaluated when it comes to picking the right tool:
- Is it easy to set up?
- Is the setup required only once; and if not, how difficult is it (and how long does it take) for anyone in the team to have it set up and working properly?
The easier it is, and the less time it takes to set up the project, the more promising the tool is.
Start writing the automated tests
Once the setup is complete, you can actually start to write your automated tests. Because this is a proof of concept, you don’t need to spend a lot of time on having the cleanest code ever. The idea is to ‘fail fast’: if the tool is not right for your project, you don’t want to have spent too much time on it.
If the tool is the right one, and you will pick it for your future automation efforts, you will later allocate the time to improve the code from your POC. But of course, try not to write the worst code ever, if you are writing code 👩💻
When picking which scenarios to start automating, I recommend you start with the trickiest ones. Maybe ones that no other tools were able to automate, and of which you are unsure that are even automatable.
This will give you a good idea about how many test scenarios might be in jeopardy of not being automated, in case they are similar to the ones you picked for the POC.
It’s important to see the perspective: can we automate all/most/some/none of our scenarios? Let’s set aside some manual test cases for those scenarios we cannot automate.
In some cases, we can observe that certain scenarios are not automatable due to how the implementation was made. Some tweaks might be needed, which would enable the scenarios to be automated. And these tweaks might be easy to make: you can simply request the developers to implement these tweaks, and off you go.
Run the tests
Once you developed your tests, make sure to run them, and run them a lot. Writing them is one thing, but running them multiple times might reveal a weakness in the tool, in regards to your scenarios.
In case such a weakness exists, you need to know before adopting the tool if there are ways to improve its behavior, or if you can work around the weakness. If not, clearly you will have unreliable tests and it’s not worth investing effort into implementing such tests.
When you are done evaluating your chosen tools or frameworks, make sure to compare all the pros and cons for each. I gave you a few tips about what you should look into. But, you might also have some specifics, like cross-browser support or cross-OS support. Make sure to include these criteria in your evaluation too.
Conclusion
The biggest win out of doing a POC is selecting the most appropriate tool for your test automation. In some cases, you will be lucky and able to pick a tool or framework which supports 100% of your automation. In other cases, there will be some need for workarounds or manual work. That is because there simply might not exist a tool that suits all your needs.
Either way, once you are done with the POC, you will select a tool having a lot of insight into how to set it up, how easy it is to work with it, how well it fits your needs.
And as an added bonus, you already have a set of tests you can run right away since you built them as part of the POC 🤗