Fully automated testing!
Is this the holy grail of testing? Is it even possible? Does the statement itself even make sense? It seems that if you want to generate a lot of heat and discussion around testing you just need to bring up the idea of fully automated testing. I don’t want to dive into those debates, but I do want to discuss what things you should automate. No matter what your opinion is on the value of automated testing, surely we can all agree that some things make more sense than others to automate! But how do you go about deciding what to automate? 🤔
Automate the Boring Things
One heuristic that I have found really helpful is the idea of automating boring stuff. If I have a task that I need to repeat over and over, I will probably get bored of doing it pretty quickly 🥱 If I pay attention to the boredom signal, it is usually a pretty good indicator that I have something I could automate 💡
Like I said, this heuristic has worked well for me. It has helped me see test automation as more than just creating scripts to do regression testing. Although, let’s be honest here, running through regression scripts over and over will usually trigger the boredom response after a while. There are other ways that our jobs get boring though. Creating the same data over and over again. Filling out the same fields in a form time and time again. Manually kicking off the same build steps every day. Whatever it is, these boring, repetitive tasks can often be automated.
So this heuristic is a helpful one, but let’s not forget that it’s still just a heuristic. It is not an immutable law. In fact, if you aren’t careful, it can even cause problems.
Don’t Automate some Boring Things
I had a boring task I had to do. I had to copy some data from a couple of systems into one place so that we could create reports. Since it was boring I applied the automate-the-boring-things heuristic. I created a script that would automatically copy and format the data so that I didn’t have to do it anymore. It took me a few hours to put this script together, and it probably saved me 10-15 minutes of boring work each week. A great win right? 🥇 I had to make a fix or two to the script every once in a while, but within a couple of months it had more than paid for itself, plus I no longer had that boring task on my list. Everybody is winning, right?
Or are they?
There is an important person missing from the story above. Why was I copying that data? It was to make a report. But why was I making that report? Well someone in management wants to know something about how the system was working. Where is that person in the story above? 🔍
I automated away the boring work, but I never asked the much more important and fundamental question: what problem am I trying to solve? You see, if I had taken a few minutes to reach out to the people that the report was for, I would have found out that they didn’t care about most of the report. I found this out when my script broke one day and a manager reached out to me to get the data. It turned out the only data he wanted was data that was already reported in another place. I just pointed him to that data and he was happy. All the rest of that data that I was pulling together was useless to him.
At that point, I stopped running the script altogether. I realized that I had solved the wrong problem. The problem I had solved was the problem of getting rid of things that bore me. The real problem I should have been focused on solving was the one of figuring out how to solve the problems my “customers” were having.
Therein lies the fundamental flaw of the automate-the-boring-stuff heuristic. It focuses on the wrong person. The focus is on me and my boredom. Successful and helpful work is work that helps other people solve their problems. Of course, there still is value to getting rid of boring work through automation, but only if that will help to better solve the problems your customers are having ❕
Automating Regression Tests
Let’s take this back to the classic test automation job: automating the regression tests. It is very easy to focus on solving the wrong problem here. So many times when creating automated regression tests we try to solve the problem of “Running these tests is slow and boring”. When we try to solve this problem we get test suites that take the manual regression test cases and create scripts that execute each of the actions in them. But what if that is the wrong problem to try and solve? What if instead, we were to try and solve the problem of “Our clients experience too many regression bugs.” This should change the way we think about it ✅
Perhaps some of the manual tests can be removed if they aren’t valuable which would free up time to do other things. Or perhaps there are things we can do with test automation that can’t be done manually at all. Of course, perhaps the problem is best solved by having some of the manual regression tests automated to speed up the feedback loop. Whatever the particular solution is, at least we are now solving a problem that explicitly includes our clients 🤗
Focus on solving the right problems & the solutions will fall into place 💪
So what problem are YOU trying to solve❓