Quality is king when it comes to mobile application testing. When we download apps that don’t work as they should, the most common response is to uninstall the app from our devices. In fact, 80% of mobile users will abandon an app permanently if they encounter a bug 3 times.
Even some of the most used apps, like Microsoft’s Skype, can suffer from critical bugs that escape testing into production, ultimately impacting their usability and security.
Quality is an ongoing issue, with a majority of mobile users reporting they have had an app crash in the last 6 months. Apps that don’t undergo proper testing are especially liable to have unforeseen issues. Mobile apps can only be released confidently when the proper testing has been performed prior to going live.
With 1,000’s of variants in the market, Android is typically the area most companies are struggling to test adequately. With this in mind, we will focus our attention on Android, but first, let’s consider the overall approaches we should take with testing our application.
Should I Automate My Android App Testing?
The age old question remains – should I test my android app manually, or should I invest in Android UI automation? Manual testing is the process of testing the app with a human tester, just as a user would. Automated testing uses tooling to drive interactions with the UI programmatically.
One of the main benefits of mobile automation testing is the ability to run tests reliably and repeatedly. Human users are susceptible to making errors and entering incorrect information. Furthermore, properly implemented Android UI automation can allow for testing of multiple devices and OS types at a single time, in parallel. Therefore, using Android UI automation can result in better test coverage and overall user ratings, leading to an app’s success in the marketplace.
With these pros and cons outlined, we will focus on UI automation for Android, and some quick hacks we can implement to improve our experience with this approach.
10 Android UI Automation Testing Hacks
1. Use automation testing tools that fit your landscape
One of the most popular testing tools, Appium, can be used to create UI automation for both iOS and Android apps. There’s no question that Appium is very useful if you are releasing apps across both platforms.
But if you are only releasing Android apps, Espresso and UIAutomator may be your best bet since they’re created by Android’s own creator, Google. They are deeply integrated into the existing Android development tools, which will make testing even more time-efficient and productive.
❗ You might find this speed benchmark quite interesting when considering such various tools for mobile automation testing: Fastest Mobile Automation Testing Tool – Appium vs. XCUITest vs. TestProject vs. UiAutomator vs. Espresso.
2. Automate first around critical aspects of the app
Understand the most critical paths for your customer, and automate around them. For example, photo editor apps require accuracy to touch commands. So, you should focus your UI automation efforts first around the touch accuracy. Eventually, you should strive to automate all of the major functionality, but starting with the most essential tests will ensure you don’t have any critical regressions. While users don’t find any bugs enjoyable, they will certainly be most impactful when they block key functionality in the app.
3. Utilize customer data analytics to focus device coverage
Automated app testing on all mobile devices and operating systems is nearly impossible, especially with Android’s thousands of device and operating system combinations. Instead, focus on the key device and operating system variants to see if you can regularly test on enough devices to cover ~80% of your user base or more. The Google Play Console provides a great overview of the device and operating systems that your app customers are using.
4. Prepare for the latest OS releases
New OS versions are constantly emerging, and users will be frustrated if their brand new device isn’t supported. Make sure to track the Android Developers Previews to get a head start on understanding changes in the latest Android version.
5. Create a backlog specific to UI automation
Building Android UI automation is a development task, just like writing code to create a new feature. As such, we should make sure we track all of these efforts, keep them properly prioritized, and understand the effort of these tasks before diving into the work.
6. Automation requires the tester and developer to communicate
To make UI automation testing time-efficient, developers should have constant communication with the testing team while developing the app. Often times, developers can tweak the app in certain ways to make it more “testable”, by providing streamlined ways to set up users, identify locators, or use dummy payment methods.
7. XPath: use with caution
Google doesn’t provide XPath type queries natively, and writing proper XPath queries can be useful when deployed properly. Approach with caution though, as performance will often suffer and the locators can be brittle, resulting in slow running tests with frequent failures. Instead, investigate using easier to identify options such as accessibility IDs as your locators.
8. Organize selectors for rapid element identification
This follows the previous tip in regards to selectors. Organize selectors to prioritize those you use most often, so if you were to use accessibility IDs first, prioritize those in the list. This will ensure quicker test creation, test execution, and test repair.
9. Create atomic tests where possible
It’s true, most of your test scenarios might start with a shared step such as logging in to the application or creating a new account. However, this doesn’t mean every test should need to start with this step, because doing so will not increase test coverage and will only add time to your test execution. Use end-to-end tests sparingly. Find ways, through API’s or other methods, to skip repeated steps where possible and build tests that can cover the specific functionality that needs to be tested.
10. Don’t let your device farm hold you back
All too often, teams will not test on a common device or operating system simply because they did not have it available in house. Today, there are plenty of solutions to overcome that problem – whether with device emulators like Genymotion, or with real device clouds such as BrowserStack and Sauce Labs. Even if you can only afford to test some device variants every once in a while, some testing is better than no testing at all.
❗ You might enjoy reviewing this comparison article of 5 cloud solutions: Top 5 Solutions for Executing Test Automation in the Cloud (Comprehensive Comparison).
Conclusion
Android UI automation can be a difficult task, but it is an important one which can determine the success or failure of your app. With the right tools, knowledge, and mindset, Android UI automation will become a manageable routine for your testing team.
Got any hacks we missed? We’d love to hear about them! Share in the comments below 👇