logo logo

Test Automation Mentality Gone Wrong

Imagine you live on an island in the Pacific Ocean, and it’s the middle of World War II. [You probably don’t actually know the war is going on, being that you live in the middle of the Pacific Ocean, but play along.] One day some metal bees come out of the sky and shoot out some people, who get to work.

They cut down trees and smooth the land to make a flat, straight path that looks, to you, like a big waste of time. But pretty soon, you see more metal bees coming, and shooting out cargo (image #1 below) – you know, food, clothing, ammo, and supplies to build more stuff. You figure, Hey, neat trick! Let me get some of that.

So you start building your own runway (image #2 below). You clear the trees and smooth the land, expecting that this activity is what’s going to call the big metal bees again to shoot out some food and building supplies for you, too.

You wait and you wait, and you wait, and nothing happens.

But, why? Didn’t you do everything right?

Didn’t you cut down the trees, just like those other people did? Didn’t you smooth the land, just like those other people did? So why didn’t the metal bees come and give you piles and piles of big boots and tents and canned tomatoes, like they got?

Test Automation Mentality Gone Wrong

What happened?

Well, what happened is that the people idolized the wrong thing. They thought that it was building the runways that made the metal bees come (image #2 above), not knowing that there was a bigger purpose all along. The cargo was coming to the island not because of the runway, but because there was a war going on and that cargo needed to be staged there for later use.

This is an illustration of the many “cargo cults” that sprung up during that time, and the people worshipped the planes that came and they worshipped the activity that made the runways. So in order to get some of that cargo for themselves, they did everything they could to replicate the actions of the successful people.

“Ah, but Fritz,” you say, “isn’t this supposed to be an article about test automation?” 🤨

Ha ha! I say, It is! I’ve just added a colorful story to illustrate the problem I see happening in many dev shops.


“Cargo Cults” Worship the Process, Not the Outcome. That’s Bad.

Let’s think about what those cargo cults are trying to do. They’re trying to get cargo to come to their island. But it’s missing the bigger picture perspective. In software development terms, that’s like trying to improve your test automation, for the sake of improving test automation. It misses the bigger picture (image #4 below – similar to image #2 above).

Remember, the ultimate goal of software development is satisfied customers. That’s the end result. Not simply more automation (image #3 below).

End goal: Customer Satisfaction. What are the ways to do that? Well, one is to ship product (software updates) faster 🚀 How do you ship faster? Well, one way is to have a shorter testing cycle. How do you have a shorter testing cycle? One way is to automate tests.

When you look at it like this, you realize that test automation, and really any automation, is just a tool that’s used to drive the process towards the ultimate end goal.

Test Automation Mentality Gone Wrong - Customer Satisfaction

Put Customer Satisfaction at the Center of the Mind Map

When you’re looking at test automation for the sake of test automation, you’re worshipping the wrong thing. You’re putting the focus in the wrong area 💡

We’ve all done a mind map, right? You write your ultimate goal in the middle of the page, and draw a circle around it. In this case, it would be Customer Satisfaction. And then branching off that, like little mushrooms or speech bubbles, are the things that directly affect Customer Satisfaction. Off of each one of those is another set of mushrooms or bubbles that are the things that affect the things that affect Customer Satisfaction, and so on.

Customer Satisfaction should be at the center of this mind map, and Test Automation should be one of the little bubbles off to one side, maybe as part of its own branch. Sure, there are going to be elements that feed in to Test Automation, but it’s not the center of the picture. If you force the focus on to Test Automation, you’re going to take it away from Customer Satisfaction. That means you’re inverting the priorities of these two.

And if you do, it’s no wonder that you’re not getting the satisfied customers that you want. How can you? That’s not what you’re focusing on. You’re focusing on some arbitrary percentage of coverage or automation that makes your numbers go up, but doesn’t actually solve the problem. In other words, you’re building more runways without knowing why.


Work Backwards to See Where Automation Makes Sense

If you’re finding your shop giving in to the lure of more test automation, because that’s an easy way to measure a metric, you might be in the cult. Want to get out? It’s going to take some work.

First, you’re going to need to reset your perspective and goal. Instead of forcing more and more automation just to have the numbers go up, reframe your view with the end-user in mind. This might require getting some of them in a room and, gasp! horror! actually talking to them 🙇‍♂️

I know it sounds heretical, in this day and age when the only people who talk to clients are sales people or account managers. But it’s essential to have a good perspective of the end-user’s goals and needs when it comes to design, development, testing, documentation, and support. If you’re getting all of those needs filtered through a few other channels, you’re likely going to forget about that end user and start focusing only on your job – automating (building runways) – which isn’t going to be productive.

Once you have a good feel for the end user’s needs, work backwards. What are all the elements that affect those needs and goals? What are the steps you need to take in order to efficiently create those elements? What has to happen before that? And before that? And before that? Where is development? What’s to be tested? What makes sense to automate, knowing the end of the line?

This perspective will help you remember that automation, like development, is a tool. It’s one of many that’s used to enhance the customer experience. It’s not a solution in and of itself.

If you forget that, and join up to the “cargo cult of test automation”, don’t be surprised when the metal bees don’t start magically dropping cargo (more satisfied customers) right at your door.

You’ve been warned 🔮


About the author

Michael Fritzius

Founder of a boutique consultancy based in St. Louis, MO. Specializing in providing automation for people to help them recapture their most valuable resource: time. More info at: https://archdevops.com/

Leave a Reply