Reducing Production Bugs Using Agile Test Automation
In this magazine article Mr. Shalev Utnik, a QA Group manager at AlgoSec shares his thoughts regarding test automation methodologies for complicated cyber security products, test automation tools and other important topics in the field of automated testing.
“At Algosec, automation testing has statistically proved to detect high severity bugs in real time and before moving to production…”.
AlgoSec empowers enterprise organizations and service providers to manage security at the speed of business. Through its application-centric approach, AlgoSec automates and simplifies security policy management across virtual, cloud and physical environments to accelerate application delivery while ensuring security. Over 1,500 of the world’s leading organizations, including 20 of the Fortune 50, rely on AlgoSec to optimize and streamline changes to the network security policy, mitigate risk and ensure continuous compliance.
So, no more manual testing?
“Test automation does not take into consideration UX aspects, or highly complicated scenarios that require creativity and exploration (in our case – highly complex firewall scenarios), therefore thinking “out of the box” of a manual software engineer is highly important. I believe that replacing the human creativity with automation testing procedures is a difficult task, and it’s important to rely on human software engineers alongside with test automation procedures.”
“At AlgoSec, test automation developers are responsible for the test script development, maintenance and suggestions for additional capabilities to existing tests. The manual software engineers are responsible for deep root cause analysis, analyzing the outputs and bugs verification in addition to manual testing efforts”.
3rd party vs. internal tools
“Speaking from my personal experience, when you start your very first test automation project you begin by writing your own tools, and only later on understand that by using existing frameworks you could’ve saved a lot of time and effort. Re-inventing the wheel is typically much harder than relying on a working one, which is usually more stable, providing extended functionality “out of the box” and reducing the maintenance efforts.
When selecting our tools we looked for these features:
- Lightweight with good performance
- WEB driver that supports multiple browsers
- Well debugged and stable
- Cross platform with support for multiple programming languages
Most of our tests are written in Java and are based on Selenium. We use the TestNG framework for executing the tests and producing reports for later analysis, as for visual testing, we use Aplitools. Finally, we use Jenkins CI server to support E2E continuous integration process.”
What were the goals behind implementing automation testing in your organization?
“The idea was to help the organization in releasing high quality products, and to increase the test coverage in our limited timeframes, which otherwise would require much additional effort from a manual QA engineer. The main challenge was to map the most critical and common user scenarios and bringing them closer to the test automation world.”
“The more effort we put into developing automated tests, the more stable tests will eventually execute and will increase the overall product quality”.
“According to my experience, an agile environment is the ultimate solution for test automation, since it helps to quickly reproduce test failures & product regressions. Also, the bug fixing cycle decreased since we started implementing automation testing.”
Any tips for test automation rookies?
“I believe that implementing test automation in growing companies is a must have goal and must be treated as a software development project including spending the proper time for designing the project and each task”. As an organization, if you are a beginner and don’t have any previous experience in this field, I strongly recommend getting assistance or consulting from professionals. Designing the ultimate test automation framework is crucial and will have direct impact on your scalability and maintenance in the future.
Let us know what you think, 🙂
Do you implement these tools in your company? or others?