There are some terms we hear often in the world of Automation: Page Object Model, Behavior Driven Development, Selenium, Page Factory and others. We listen to them when we look for how to do automation testing following what we know as best practices. And why do we do this? It is because, as the name implies, the best practices are there to give us a structure to work efficiently, with methods already proven by countless people in the same situation in which we are about to embark.
But what happens when these best practices are not the best option for the problem we are trying to solve? Is that even possible? Are best practices something universal or are they something based on context? Today I come to tell you about past experiences where, the solution, was quite the opposite of what is posed in the utopian vision of test automation and, even so, they were resounding successes.
Test Assets: Custom Solutions for Custom Needs (Selenium, JMeter & Grafana)
In one of my last clients, there was a need to create a test asset that would serve to monitor how long it took for a user to log in to production. For that, it was required to be something that reflects the customer experience. That is, it had to be through the UI, without logging in with a simple request to the API and had to validate some things on the landing page.
On the other hand, the solution had to be packaged to run externally outside the company, so it required either a JAR or another way to run it without building or downloading anything extra like dependencies.
Finally, the solution had to generate data points in a database to be the reference for a Grafana dashboard in which graphics with the trends in the measured time of each page, the login, and other key actions would be displayed.
What was the solution that best suited these particular requirements? Create a class in JMeter, with Selenium and PhantomJS (it is deprecated now thanks to Chrome’s headless ability, but at the time it was one of the best options) and take advantage of the simple integration with Influx dB to send data points that were to be picked by Grafana to create a monitoring dashboard with exactly what the client needed.
As you know, JMeter is an ideal tool for Performance Testing, not so much to create a simple test to log in to a page through the UI and, on top of this, with Selenium. You will also know that, when using Selenium, it is recommended to use the Page Object Model so that maintenance is done without problems in the long term. So … why use Selenium within JMeter and without using the Page Object Model, creating web elements, actions and test steps within the same class?
Well, the truth is that this Artifact Test was not going to require any maintenance or added scenarios after being created. In addition, it was requested to be the indispensable minimum given the technical limitations of the agent in which it was going to be executed. JMeter entered the field given its simple integration with Influx dB and the ability to extract data points very easily from the code. The only class instead of the multiple page classes suggested by the POM was another consequence of needing to simplify something that was going to be used just for that and that was not going to need more work later.
For the record, I did try the best practice approach: Selenium, with Gradle as the build tool, the page object model, Cucumber for the reporting (which wasn’t needed given this was intended for Grafana feeding) and using a myriad of other dependencies to get the timing of the request triggered from the UI as a HAR file that I needed to parse… As you can see, an overkill for the quick task they needed done.
The result, which I’m still proud today, was to have those dashboards in pretty much every floor of one of the most important clients of my company, with the management and countless testers, devs, performance teams and others using it as one of the pivotal tools for their work.
When Do Best Practices Work?
It is important to understand that, best practices, work only depending on the context. One has to know them and, above all, understand them, to know when it is a good idea to implement them. Better yet, separate the best practices in modules that can be fitted as if they were gears in the big scheme of Automation, in order to find the most efficient solution without using them for the mere fact of doing so and unnecessarily increasing complexity.
Having said this clarification, I wanted to tell you when it is a good idea to follow these best practices. When they work, we will realize they really do because this will save us a lot of work and time. And this is because, in general, these guidelines exist precisely for that: To organize, structure and, ultimately, prevent us from repeating ourselves on building bridges to the common automation pitfalls. Not following them will not make a project not compile, but it will take us to an early cul-de-sac that will not allow us to move forward and will demand that we rethink the engineering to come up with a better model, losing huge amounts of time in the process.
Also, when the context is appropriate, making use of what is considered standard will make everyone being on the same page and the workflow is enhanced with this synergy. Think for a moment about a double-edged sword in the scene of Automation testing: Cucumber.
Cucumber, when used by a Business Analyst to create the scenarios relying on the tool to follow a Behavior Driven Development methodology, achieves the utopian reality of having a living documentation, readable by less technical people and with all Automation machinery behind.
On the contrary, when used only by the fact of using it, it ends up adding layers of complexity that do not facilitate the understanding of what is being tested, often with scenarios written by the same testers that, rather than fulfilling the role of BA, simply want to have a text to put the code that defines it behind.
When They Do Not work?
As I always say, best practices are not something holistic that can be applied to all kinds of situations and hope they just work. It is similar to diets: Not all bodies and metabolisms have the same requirements and, applying a diet without first studying what the body needs to be applied to can lead to results far from those expected.
For example, let me tell you about a case in which I worked relatively recently. There was an application under test, web, that required several scenarios to cover what manual testing did. All these tests were performed on a particular page. As the Test Engineer on duty used to create the framework using dependency injection, with the world object for creating instances of page classes, explaining this to the testers responsible for automating later was a different task. When I asked why they used dependency injection, they all said the same thing: “because this person created the framework like this.” After explaining the most common uses in Automation and getting them to understand that, if they only used a Step Def class there was no need to inject because it only added complexity without solving any problems, they got rid of this way of dealing with the classes in that project, bringing much more fluency at work and avoiding people breaking their heads to understand how to do what is usually done when automating with Selenium and Cucumber.
What Should I Do Then?
We will have to advocate for quality. What does this mean? That the practices we use to carry out our work should not get in the way and hinder quality assurance. Part of being a testing professional is to observe the context, the technologies involved, the needs of the client and, with this information, make the decisions that give the best results. And sometimes, best practices do not lead to the best results, especially when the context is not the right one!