A few years ago a colleague of mine said to me, “You’ve got some excellent material in your presentations. Have you ever thought about writing a book?”. At first I didn’t take the suggestion seriously, but later on, while readying another talk for a conference, I noticed there was a theme developing in and from my presentations. I was focusing on the front-end work and processes related to implementing automation in testing.

The reason for this was that I saw a lot of articles, whitepapers, and blog posts written about tools, frameworks, and how to write automation code. But they were always minimal in content (or none at all) about what information to gather first and the steps needed to take before getting that deep into the coding process. Everyone was jumping in feet first and building scripts as fast as they can. This then leads to articles, whitepapers, and blog posts written about automation implementation projects either outright failing or needing severe rework.

The thing that amazed me the most is that they all asked the same basic questions of “Why did we fail? Where did we go wrong? How are we going to fix this?” And the typical answer was, “We should have planned better, and we need a deeper understanding of this type of work.”

They all kept “putting the cart before the horse.” As I was someone who had made the same mistakes early in my experiences implementing automation in testing, I found the impetus to write the book my friend had suggested.

Before The Code: First Steps to Automation in Testing

The following are summaries and conglomerations of the topics and information in my book.

Chapter 1 starts with the process of implementing automation in testing. There are multiple things to consider, like why the company/team wants to implement automation. What should or should not be automated. Who is going to do the work and how best to do it.

Typically the “Why” question is answered with “we want to make testing faster.” The “What” question is responded to within multiple ways based on what type of testing is to be done and at what level. The “Who” question is answered in various ways, as well.

An outcome of these questions come in the shape of an implementation plan and budget. The plan provides the roadmap for the tasks to undertake and their estimated timelines and dependencies. The budget can be designed for staffing using FTE or Contract labor, which can be mixed teams if needed. Outlining the budget upfront is useful when talking with management about automation effort and getting their support.

Chapters 2 through 4 cover the topics of myths & misconceptions regarding automation in testing, how the Automation Pyramid is misunderstood, and finally how to educate management (and other groups) about the myths and misconceptions surrounding the implementation of automation.

The myths & misconceptions include things such as:

• You can automate 100% of your tests.
• Automation allows Management to reduce test staff.
• One tool is all you need.
• Automation will speed up the testing process.
• Anyone (untrained/unskilled) can do automation.
• Open-source tools are “free.”
• “Codeless” and “Scriptless” tools can do all the work.
• Automation can be created quickly.
• Only hire a Software Design/Development Engineer in Test (SDET).
• Automation provides an immediate return on investment (ROI).

The Test Automation Pyramid should be considered more of a concept than a rule. Common misunderstandings of it include the need to de-emphasize GUI level tests for various reasons. Another one is the use of percentages of implementation at each layer. The originator of that idea later reversed their statements and said that they were not advocating for exact ratios. Even Mike Cohn, in his original work, did not say anything about percentages between the layers of the pyramid. His emphasis upon Unit Tests was to push for reliability and robustness of the code. He felt that testing at a more atomic level would help to eliminate the need for similar type tests at higher levels.

Educating management, and other groups, about these particular myths and misconceptions surrounding test automation, gives you a shot at success. It provides the understanding and opens lines of communication that will benefit you and your work.

To achieve this, use selling techniques and learn how to talk in the “language” of your target audience. Doing so provides better communication and builds relationships with other groups outside of testing. It demonstrates testing being an integral part of the process and leads to more involvement in the earlier stages of the project cycle. Perception can be everything, and one that isn’t well known or thought of is that testing is the center of a wheel. It is a hub of activity and information. As such, a lot of information flows through it and needs to be communicated to the appropriate groups, in the proper time, and in a way that is easily understood using the correct language.

Chapters 5 and 6 cover the topics of staffing and training them for automation work. Typically companies that are just starting out implementing automation will look outside of their doors. Because of that, they need to make sure they create the proper job description for the position. Too often, job descriptions are “shopping lists” of skills and try to find someone who has it all. Realistic expectations need to be set from the beginning. These include a focused and precise job description, knowing that it will take time to find the right person, finding the right staffing firm and getting qualified candidates from them, and making a realistic offer to that person (don’t lowball). This is true for either a contractor, consultant, or full-time employee (FTE). You get what you pay for, and implementing automation in testing is a long term investment.

If they decide to recruit an internal person, they need to take the same steps as listed above. But there are a couple of differences to know. First, the person is a known commodity; their abilities and work are known. Their financial costs are known and need to be adjusted accordingly. Make the change in position and work worth it. Second, the company has to know it is shifting a resource from one role to another. You can’t expect this person to do two different jobs at the same time. They need to backfill the position or shift the responsibilities to other people. They also need to make sure the person recruited to do the automation work gets training if they don’t have all the requisite skills.

Training itself can be a broad subject. Companies need to realize the diverse set of skills and knowledge a person is going to need to work with automation in testing.  The topics range from programming skills and languages to testing skills, to an understanding of automation frameworks and their architecture, and technology skills and knowledge.

Training of staff is an essential step on the road to a successful implementation of automation in testing. It needs to be done so that everyone involved understands the work and effort involved, along with the technical challenges that will be present. There is a need for both disciplines of development and testing to be merged to form a hybrid that can understand and address the conditions of the work.

Chapter 7 talks about the basics of automation development work.  The concept of a “script” is discussed and how certain practices need to be utilized to ensure the script code is modular, maintainable, and reusable. Principles such as DRY, KISS, and YAGNI are discussed. The chapter talks about building software to test other software to create “Testware.”  Not only does this cover the different types of programming languages to use for automation, but also the various technologies in the application stack. Including GUI vs. Database vs. Operating System vs. Web vs. Services vs. API’s. All of these require that you understand how to hook the software under test and interact with it, and at what level.

Other automation practices, such as SEARCH (Setup, Execute, Analyze, Report, Cleanup, and Home) are covered. The chapter closes out with automation patterns and the four basic categories of them and how they apply to implement automation in testing.

Chapter 8 is about the selection of tests for automation in testing.  The first factor in deciding which tests to automate is determining if it is the right one and has the necessary information to allow the development of an automated test from it. This speaks to the idea of a tests “appropriateness” for automation. Is it a good idea to spend the time and effort automating the test? You need first to review your tests and determine which ones are worth the effort to convert to execution by test automation.

As part of the process, the “Readiness” of the test needs to be considered. How well the test lends itself to automation. Is there enough information and detail describing what the test is to do and how to do it. Criteria for readiness includes things such as:

• Does it have enough information on the process so that it is clear what to do during the test, and does it answer comprehensibly how?
• Does it clearly describe the data, and their conditions, that are needed to perform the test?
• Does it express an endpoint for the test?

Next comes the principles of Categorization (used for grouping of tests into suites) and Prioritization of tests. These allow for better management and organization of the work according to the need and focus of the project. Different visualization methods are discussed regarding how they can aid in the process of designing and selecting tests for automation.

Chapter 9 is about Testability and the factors & items that need to be implemented to aid in the process of automation in testing. The section talks about the different levels of the application stack and which factors to consider to make them more testable and automate-able. This includes the GUI, API/Services, and Code/Unit levels of the software under test. “Baking in” Testability is used to build in properties in the software to make it open to be more testable. It’s done to ease the testing of software and allow increased use of tools in the testing effort.

Chapter 10 discusses the need and purpose of a Proof of Concept (POC) when implementing automation. Why perform one, how to ready and perform it, and how to present the findings to the appropriate parties. Performing a Proof of Concept is one of the most important things to do. It ensures you’ve picked the right tool, framework approach, staff, and skills/training needed for the automation implementation itself. A POC and its review ensure that you are not going to be wasting time and money.

Chapters 11 and 12 are about automation framework concepts and the basics of framework architecture, respectively. There are frameworks for each level of testing, and within them, there are different types of framework approaches or designs, and patterns within them. These include Unit, API/Services, and GUI level.

Frameworks are the software infrastructure upon which to create test automation. They support the implementation of automation at the different levels of the code and software. There are different types of automation frameworks for the different kinds of testing on a project. Within a framework, there are different levels and types of capability and functionality. They allow for the creation of the necessary assets and artifacts to build automated tests with. And in a way that is flexible, reusable, robust and maintainable. Frameworks enforce good programming practices and treat test automation as software development.

Chapter 13 talks about where to run the automated tests and how to set up your test suite for distributed unattended execution. How to set up a “Test Lab” and the benefits it can provide. This includes how to run them in parallel, running them unattended, on-demand at any time of day, and as part of a CI/CD process. How a “Test Lab” provides flexibility in the types of configurations of test environments utilized and can offer a cost-effective way to scale up the testing effort and add to the overall ROI for test automation.

Chapters 14 and 15 discuss Self-learning and Craftsmanship. As discussed briefly in a previous chapter, there are specific skills and items of knowledge to have or attain to be effective with implementing automation in testing. A person needs to become a hybrid of tester and developer. They need to have an understanding of the models of software development, its techniques for programming, and the principles that should govern them. The chapters talk about understanding the different types of testing and their techniques, along with the methods that support them. They end with how Craftsmanship is honing one’s abilities but also learning and applying other subject’s knowledge as well. It is the ability to take the abstract concepts and constructs from those other subjects and using them in the current context of the work at hand. Craftsmanship is the process of going from Novice to Expert, and along the way of learning being able to apply the information proficiently to eventually being able to teach it to others.

 

As a thank you to the test automation community, here’s a coupon for you to use to get the book at a discounted price: https://leanpub.com/beforethecodefirststepsforautomationintesting/c/TestProject

If you have already read the book – I would love to hear your feedback in the comments below  💡 

 

---