Exploratory testing is one of the most important approaches in software testing. It comes in complement to automation testing and test case-based testing, and can help uncover software defects that are usually missed by the other approaches. In this article, I will show a quick exploratory testing guide 🧾
What is Exploratory Testing?
Exploratory testing is a software testing approach where the testers design and execute test cases simultaneously, based on their knowledge, experience with the SUT, and results from previous tests. It is done without executing previously written test cases and scenarios. While it may seem that the testers perform random actions in the application, that is not the case. In fact, exploratory testing is defined as an experienced-based type of testing. It requires good knowledge of the application under test.
It is a very useful approach when the project lacks documentation or when the documentation is not very detailed. In these cases, it can be difficult to extract test cases before the application is ready for testing because we don’t have the requirements until then. It can also be extremely useful when we have limited time for testing. Executing the tests while designing them helps save a lot of time, since we can identify defects faster than we would if we wrote the tests first and executed them after.
Challenges
- When the project lacks documentation, we may be unsure of the expected results for certain scenarios. This may lead to invalid bugs or a lot of time spent discussing whether a functionality works the way it should.
- Because the tests are unscripted, it may be difficult to record every step performed during testing. This can make defect replication harder.
- It requires good knowledge of the SUT. If you are new to the project, or a beginner tester, it might be difficult to perform exploratory testing efficiently.
- It also depends on the tester’s skills. It’s important to be a creative person, flexible, and to pay attention to details.
Benefits
- A research published in 2007 revealed that exploratory testing found up to 11% more defects than test case-based testing. It also found 29% more “obvious” defects (such as a missing button) and 33% more hard-to-detect defects (defects that require multiple user steps). Furthermore, it produced fewer false reports (meaning invalid, duplicated, or otherwise non-existent bugs) than scripted testing.
- Yes, test automation is great, but it cannot test everything. Exploratory testing can cover scenarios that cannot be automated. It also provides a “human” touch, because automated tests can only check what they are instructed to check. However, with exploratory testing, testers can look further than the test cases or requirements, and find inconsistencies they would otherwise miss. It is also great for usability testing, where test scenarios are hard to identify before test execution and even harder to automate.
- It provides quick feedback. Because test execution happens at the same time as test design, testing can begin as soon as the implementation is ready. This means that defects can be quickly identified, which makes them easier and cheaper to fix.
- For the same reason above, it supports Agile development. It provides flexibility and allows quick or unplanned changes in the requirements, without the rework needed to rewrite existing test cases.
- If done right, it can help understand the SUT better because it leads to flows that are not usually explicitly covered in the requirements and test cases.
- It encourages the tester to use their skills, creativity, and intuition. This means that more scenarios can be identified, that would otherwise be missed with more traditional testing approaches.
- As the pesticide paradox principle states, when executing the same test cases repeatedly, they will stop uncovering new defects without updating them. When tests are done exploratory, there will always be some deviation from the exact scenario, which means new information can be found with each new execution.
How To Perform Exploratory Testing
There are multiple ways to perform exploratory testing. The best thing about it is that each tester can use their own skills and preferences and adjust the testing process accordingly 💪
Charters
Testing charters are a mission statement for exploratory testing, as defined in the book “Explore it!“. Each charter corresponds to one or more features that will be tested in one session. The session should be time-boxed, and the tester(s) should focus solely on this purpose and not get interrupted by any other tasks, such as answering emails or going to meetings. The charters will have a clear focus, and also a clear statement of what should not be tested.
By the end of the sessions, testers should have documented what was tested, what issues or defects were encountered, and the conclusions of the session, including overall results and findings 📝
Persona Testing
This is an approach that can help uncover more unusual testing scenarios. The idea is to identify the personas of the potential users of the AUT and test from their perspective. Each type of persona has a different behavior, different interests, and different ways of interacting with the application. Once you create the personas, you put yourself in their shoes and use the application as they would.
Here’s an example for an e-commerce application:
Rebecca is an 18-year-old high-schooler. She uses a fast internet connection and uses the app from her phone. She wants the application to move fast and doesn’t have the patience to browse through many pages to find a product she is looking for. Therefore having filters and sorting is important.
Malcolm is a 62-year-old who is not passionate about gadgets or technology. He browses the app from a PC with a slower Internet connection.
You can see how the two people would use the application in two very different ways. So as a tester, you should test the application in two different ways. Once, to make sure that everything works on a mobile device 📱 and that the navigation moves quick so the user can reach the product of interest fast. And once, to check how the application works with a slower connection and confirm that all visuals are correctly loaded and there are no responsiveness issues.
Pair-Testing
In this approach, testers team up to test the application together. Since each of us has our own experiences, each brings a unique point of view to testing as well. The testers take turns performing the test execution and analyzing and reviewing the results. It can be particularly beneficial to less experienced testers, who can observe their senior counterparts at work and learn from their experience and knowledge.
Good Practices in Exploratory Testing
- Keep the sessions focused on one area, one functionality, or one purpose. Don’t try to cover the entire application in a single session, but have dedicated sessions for each area that needs testing.
- Time-box the exploratory testing sessions. Don’t let yourself be interrupted by other tasks during the sessions.
- Start testing early ⏳ This applies to all types of testing, but as I already mentioned, exploratory testing can provide quick feedback therefore it can identify defects as soon as the testing activities start.
- Document the test activities and the findings.
- Use your creativity while testing. This is the best time to use your imagination and try to “break” the application in less obvious ways than the classic boundary and equivalence partitions tests.
Conclusions
Exploratory testing can be one of the most beneficial ways of performing tests in your project. While it does not require any programming knowledge, it doesn’t mean that it is easy. The more experienced the tester, the more value this type of testing will bring 💡