logo logo

Maintaining Value – Automation’s Forgotten Cost

main post image

Most first-time automation endeavors lack planning around maintenance cost. We typically start an automation endeavor like this:

  • Decide to automate.
  • Choose a tool and start automating.
  • Demonstrate our automation success (hopefully).
  • T-shirts for everyone!

Soon, we begin development for the next release of the product. Depending on how we decided to automate, there might be major changes required in our tool, our scripts, or both. The scripts may need to be rerecorded or rewritten. Is that effort a big deal? Maybe, maybe not; It depends on our chosen tool, our implementation, and the quantity of automation that has to be reworked.

Sadly, this chain of events causes many automation endeavors to be labeled as failures due to maintenance costs. When planning an automation endeavor, several things are often forgotten:

  • Licenses and support agreements for tools.
  • Script maintenance.
  • Tool and framework maintenance.
  • Documentation updates.

License and support agreements should be very straightforward, but often they are not; the details are in the license agreements or contracts we sign. The total cost of our licenses can increase or decrease depending on how many users we have. Some products provide perpetual licenses, meaning that even if we decide not to renew our support agreement, we can still use the products at no additional charge; Some products require the support agreement to continue their use. Neither of these support approaches is inherently bad, but one may be more preferable to our particular organizations. In any case, if we need to pay annual fees for our automation tools, we must account for those fees during our initial planning; We don’t want any surprise bills at the end of the year.

We have to remember that our automation is code. In the same way we plan for upkeep on our product code, we need to maintain our automation code, including any scripts that we create. If we change our product, we may have to change or extend our existing automation. We will also have bugs; We’ll find issues in the automated scripts which need to be debugged and fixed. Finally, taking upgrades of our automation tools may require changes to our scripts as well: Sometimes releases are not 100% backward compatible.

“But we use a tool that is code-less (or script-less)!”

So? It’s still code. Somewhere, under the hood, in that stack, there is some software. Additionally, using a non-code interface for a tool still requires us to sequence a set of commands that will perform an activity on our behalf. I submit, that’s code and therefore software, which will require some kind of maintenance. Users of this kind of tool may not have the same considerations as users of a script-based tool, but they will likely have some that are unique to their tool and implementation choices.

As much as our scripts are code, any tools or frameworks that we create for ourselves are equally so. In addition to the maintenance items related to scripts, there are other items that can cause maintenance to our tools or frameworks, including operating system or programming language upgrades; New features, new fixes, and new issues can necessitate changes in our code.

As the automation we create gets used, the documentation we write for it must evolve. We will add features and fix issues; These activities often result in needing to update our automation documentation. We should also consider updating the documentation to reflect how our tools and scripts are being used; Things that one team is doing may benefit other teams. FAQs and troubleshooting are the most likely candidates for updates from this emergent learning.

As usual, not all of us will have the same maintenance needs, however, neglecting to consider and account for them can wreck our budgeting or planning processes. We need to think about and attempt to account for as many of these items as possible. If we don’t, we run the risk of our automation being labeled as a failure, when in truth it was our planning that was at fault.

You are welcome to share your thoughts on test automation maintenance in the comments below!  🙂 

Paul Grizzaffi – Blog & Twitter

Avatar

About the author

Paul Grizzaffi

Paul Grizzaffi is a Principal Automation Architect at Magenic – https://magenic.com/. His career has focused on the creation and deployment of automated test strategies, frameworks, tools, and platforms. He holds a Master of Science in Computer Science and is a Certified ScrumMaster from Scrum Alliance. Paul has created automation platforms and tool frameworks based on proprietary, open source and vendor-supplied toolchains in diverse product environments (telecom, stock trading, E-commerce, and healthcare). He is an accomplished keynote speaker who has presented at both local and national meetings and conferences. He is an advisor to Software Test Professionals and STPCon, as well as a member of the Industry Advisory Board of the Advanced Research Center for Software Testing and Quality Assurance (STQA) at UT Dallas. Paul looks forward to sharing his experiences and expanding his automation and testing knowledge of other product environments.

Join TestProject Community

Get full access to the world's first cloud-based, open source friendly testing community. Enjoy TestProject's end-to-end test automation Platform, Forum, Blog and Docs - All for FREE.

Join Us Now  

Comments

10 1 comment
  • Java Testing Weekly 13 / 2018 March 26, 2018, 6:17 am

    […] Maintaining Value – Automation’s Forgotten Cost is an interesting post that identify things which are often forgotten when people start to write automated tests for the first time. […]

Leave a Reply

Join TestProject Newsletter

Join a 20K community of readers! Always stay up-to-date with all the latest test automation trends, best practice and tips shared by leading software testing community experts across the globe!

FacebookLinkedInTwitterEmail