Recently, I realized that I wasn’t taking advantage of as many automated validation tools as I could be. Hopefully, after reading this post you will question whether you are incorporating enough automated validation tools in your testing process.
The testing process provides one way of reducing risk on our projects. “There is a risk that we have bugs in the software”. Therefore, perform software testing to identify problems, which we then fix, to reduce the risk that the user will find the bugs in the software.
When testing, “there is a risk that our testing misses issues”, these are some reasons for that:
- Lack of time.
- Prioritizing areas of the software means some parts don’t receive as much attention.
- Lack of technical knowledge.
- etc.
Given this, we want our software testing process to increase its effectiveness and find issues that only the testing process could find. We really don’t want issues being found by a tool, implementing a series of automated validation rules.
We easily check our compiled code for syntax errors. The compiler does that for us. Parts of our application do not go through this process.
As an example, I recently wrote a simple ‘joke’ application to generate the type of slogans in my book “Dear Evil Tester“.
Despite having only 15 lines of HTML, I managed to write it in such a way that it was buggy on iOS devices.
- None of the browsers I used for testing rendered it poorly.
- None of the browsers I used for testing reported an issue.
- When I reviewed the code it looked fine, given my understanding of HTML.
But, it failed to render one of the buttons on iOS. I could have found this problem early by running the HTML through a standards automated validation tools e.g. :
In 2 minutes, either of these free tools could have found the error and I could then quickly fix the code to make the HTML standards compliant.
Had I used these tools, any browser that claims to adhere to the HTML standards should have no problem rendering the application. I could have done this for free, prior to my testing and certainly prior to release.
Many parts of our applications have tools that can apply validation and review rules automatically:
- Interpreted languages like Ruby and JavaScript have linters and static analysis tools.
- CSS has validation tools.
- HTML has other validation tools.
- Compiled languages have tools that go beyond the compiler e.g. findbugs, checkstyle.
If we do not use these tools, then we increase the following risks:
- Our testing finds defects that could have been found earlier and automatically.
- We have to repeat testing after ‘minor’ defects have been discovered and fixed.
- We miss very small ‘compliance’ issues which only manifest on certain devices.
- We don’t use the devices that the errors manifest on, and therefore release defect to production.
I suggest that for each of the technologies that you use on your project, you spend time looking for static analysis tools that you can apply. Mitigate the risk that your development process would allow simple issues to slip through that can decrease the effectiveness and cost efficiency of your testing.