There’s a trend that I see with test automation on many QA teams – not following software engineering best practices. They have a large number of tests that all pass. Or some tests that flicker between pass and fail. Or some that fail all the time. But the biggest problem is that they rarely remove any tests. Some people fill their house with things they never get rid of. A team’s automated test suite can be the same, piling up with low value tests. I call this phenomenon, “Automation Hoarding,” and it can cause many teams’ effort to grind to a halt. Are you an automation hoarder? Here are the signs and solutions if you are:
Usability as a Service
When building automation frameworks or optimizing existing ones, I use the “Three Pillars”. These are three software engineering best practices I’ve learned that great frameworks live by. The three principles are: Usability, maintainability and extendability.
When it comes to hoarding tests, I find it’s because of low usability. This shows up when it takes a lot of effort, or technical expertise, to build a test. Then, a mental block forms to deleting tests, and that’s understandable. Few people would drop tests that took them hours, days or weeks to construct. However, not dropping some tests leads to long-running test suites and high maintenance.
The product under test is always changing, so your tests should too! If they don’t, or can’t, there’s a good chance there are stale tests in your suite eating up time, energy, morale and money. If that’s where you are now, someone on your team may be an automation hoarder. It might even be you! But that’s ok, because there’s hope. Here’s how to break out of that mindset.
Breaking Out
The key to breaking out of this anti-pattern is to increase the usability of your automation. This can be done by decreasing two things:
- The technical expertise required to write tests.
- The tribal knowledge about the test framework.
Decreasing Technical Expertise
Many test frameworks need developer-level skills to write tests. Many testers aren’t developers, and that’s a big problem if your goal is to get good automated tests written. The very people who have the knowledge and strategy for that can’t do so, due to the technical hurdle.
To decrease that hurdle, reduce your suite’s complexity. If you’re writing code, or selecting tools, have testers be your target audience. Choose tools or patterns that get all the technical stuff out of the way, so that testers can get the most out of their own skill. Then, testers, developers and automation engineers will all have a better time. When tests are written using tools like this, they take much less time to build. Then, it’s easier to delete tests when they’re no longer useful.
Decreasing Tribal Knowledge
How many times have you worked on something, only to find out it was already done somewhere else? I know I’m not the only one. This is tribal knowledge—Someone, somewhere, knows this information, but we don’t have the time to ask everybody, or to ask about who knows about this. The result is duplicated effort of course, but it also causes hesitation: “Is this already written? What if I’m reinventing the wheel on this?”. That hesitation causes more time to be spent, which results in not wanting to delete the test later.
To prevent this, remove tribal knowledge entirely. Choose or build tools that let you write tests as close to your native language as possible. For example: With Web UI testing, reduce the test down to simple actions: Click a button, enter text into a field, validate a particular result, etc. Smaller parts are easier to remember. Make it obvious what the test is supposed to be doing, and it will take much less time to write it.
Moving Forward
If your automation effort is slowing to a crawl, it might be due to hoarding tests. If it is, then hopefully this article shows you how to get a handle on it. When you’re free to add or remove tests quickly, great things happen.
Happy automating! 😎 –Fritz
Please share in the comment section below your opinion on “automation hoarding”? Are you one of them?
Love this post! I would like to add that in order to decrease technical expertise you must sometimes sacrifice simplicity and maintainability in the framework in order to make tests simpler to write and more maintainable. This is fine, because you are trying to make your tests easy to write for someone who may not be a developer. It’s just backwards from how you want to write code as a developer. Break that mental barrier. Also in the effort to decrease tribal knowledge my key is a good wiki. First make your workflows and methods very easy to find in code and very clearly defined so that people know what they do based on the name. After that a good wiki does wonders. It takes work to maintain it, but if you put the right mechanisms in place to ensure that framework changes are reflected in the wiki as they happen it makes it a lot easier on test writers. I recommend making it part of the merging process for code changes in the framework. Like code doesn’t get merged if the wiki is not updated.
A wiki is a REALLY good idea. And when you said “mechanism” it triggered a deep memory.
I work a lot in Ruby, so I used to use a gem called “YARD” which, when you run it against a codebase, it goes through and generates documentation for you. You might have seen examples of this before, where you click on a class, and it takes you to a page with all the methods/classes in there and you can keep drilling down.
It’d be simple to set up a Jenkins job to run YARD against the framework code as soon as any check-ins are detected. Thank you very much for the reminder–time to dust off some neurons!