Web and mobile applications are incredibly important and present in tech in recent years. More than just tech, it looks like every company nowadays has some sort of mobile or web application, sometimes even both. During the start of my career, I thought I would never work on mobile apps as I didn’t perceive them to be as important as web apps.
This point of view is not so rare when talking to fellow colleagues in my profession, but I believe it is a specific bias based on a worldview that is not true anymore. During my youth, all families had computers, and only a few people had smartphones. Now the number of smartphones is definitely higher.
I started working as a back-end developer for web applications, moved to web testing, and two years ago I got moved to the mobile squad. At first, I wasn’t excited to be a part of the mobile team, but later on, I realized that perhaps I was working on the most used project in the entire company.
Since I had started my career thinking only about web apps, I had some issues when I pivoted to mobile. So if you are thinking about a change to mobile testing, here are some important points to keep in mind ✅
Mobile Apps Have Versions
Imagine the following situation: You are testing a new feature on the test environment for a web app. Assuming your workflow is not out of the ordinary, it should look like this- test the feature, find a bug, fill a report, assign the report to the dev, the dev fixes the bug, opens a bugfix, merges it, and then you retest. Very simple, straightforward, and efficient.
Now imagine you are testing a similar feature in a mobile app. You follow every step of the workflow but when you get to report the bug you get bad news. Turns out this bug was already detected, but it was fixed in a new version that became available 15 minutes ago.
This is probably one of the most frustrating experiences when testing mobile apps. If it were a web app you wouldn’t have this problem, since every fix becomes available almost instantly after the bugfix has been merged.
In web apps, the state of the development of the system is the same state that you are testing in. In mobile apps, the state of the development is always ahead. A bug could be fixed and merged to a new branch, but a new version was not generated yet.
So when testing mobile apps, the version in which you are testing in is a constant concern, given that the testing outdated versions doesn’t give you the correct idea about how many bugs the current version has. To circumvent this you need really good communication between members of the team, easy access, and alerts to new versions.
Another thing that is important to notice on a website is that the time between testing a bug and retesting it after a fix has been deployed can be really quick. Let’s say there is a typo in a button that you were able to detect.
The time needed to write the code to correct this and deploy the fix is so small you could go to the cafeteria after writing the report, grab a latte, and when you get back the typo has been fixed ⌚ Downloading the app takes a little time, so you would usually wait for a few bugs to be fixed so it is worth it to generate a new version and only then retest it.
At first, it doesn’t look so different, but the time between reporting a bug and retesting it can be as long as days (assuming it takes a lot of time to fix it), after this time you could even forget what the bug was in the first place and need to look at the report you filled yourself to know what it was about.
Furthermore, this new version can have new features, integrations, and other things that can interfere with the feature previously tested, which can cause unexpected behaviors or bugs. Keeping good track of bugs with some bug tracking tools and keeping track of the versions can be of great help.
Features Take More Time to Get to the Clients
For better or for worse it takes longer to publish a new feature in a mobile app than on a web app. The reason is simple, you need an approval from the Apple/Play store, which can take days if everything is ok, or longer if something is wrong.
Reasons can be, for example, your new feature is based on Bitcoin, and Bitcoin is forbidden in some countries that you selected to distribute. This increased time to the client makes getting the feedback slower.
Usually, this just slows the development a little and does not affect testing too much. The problem surges when a certain bug appears right when you are testing a new feature. While planning your testing schedule (testing new features or new bugs), it is harder to know in what hour or day your new feature is going to reach your clients.
The good thing is that getting the feedback is somehow easier than getting it on a website. Users of your app can give you reviews on the Play store, alerting you of bugs or bad UX, making your testing and monitoring easier.
To make things faster try to connect the Play store to your most used communication channel so everyone is aware of the reviews as soon as possible. These reviews are most of the time great tools to catch bugs, check the client’s happiness, and help users that are stuck in registration.
Just keep in mind some clients will complain about unfair pricing and other things that are unrelated to the quality of the app, and will give you a one-star review because the competitors charge 50 cents less 🙈
Automated Mobile Testing Has Some Specific Challenges
Probably the most painful aspect of testing mobile applications is just the enormous number of different devices, OS versions, hardware capacities, and resolutions you have to deal with. In my days of web testing, we would usually test for Chrome and Firefox, perhaps some different resolutions, but that was it.
In the end, it didn’t matter if our client was using a Dell notebook or an HP PC, because it wouldn’t affect our application. It didn’t matter if he/she was using Windows 7 or 8. Web applications are just more resilient to hardware and OS changes.
Getting into the mobile app nowadays we need to check Android and IOS, so we would need 2 devices, but add in some different OS versions and this number can increase to 10, and add in some different smartphone manufacturers and this number can increase even more.
At some point, it becomes impossible to have all these different devices and to manually test the test suites by hand. At this point, we use automated tests and device farms. Automated tests are of common use in web applications, but in general, we don’t have the same case for device farms 📱
As I said previously, web applications are more resilient to hardware and OS changes, so it is usually a good call to skip these kinds of tools as the costs (monetary and people’s time) are bigger than the rewards. In mobile apps these kinds of tools are essential and the investment is justified.
A device farm is a collection of devices that could be used for testing. Big companies own physical farms but most companies use farms from their favorite cloud provider.
These farms are essential to test mobile. Many times a bug is exclusive to android and not present on iPhones, or only exist for big screens while small screens do just fine. So be sure to add a task to implement one in your backlog as it will certainly avoid a lot of bugs from getting to production.
The State of the Phone May Cause Bugs
More than the OS versions or browsers that you use, there are even more additional challenges in mobile testing. Your user could be traveling with an unknown location, his battery could be running out, his internet could be slow or unavailable, and his phone’s fingerprint could be broken along with his camera.
There are just so many variables that you have to keep in mind, and know how each of them can affect your app’s performance. There is no easy answer as to how to prepare for all these scenarios, but we can try to identify which are more important and need more attention 👀 Usually, some scenarios are more critical than others, depending on your application.
For a food delivery app, location is crucial information and so testing how the app handles failures in GPS is important. In banking applications locations are still important but not as much. Internet speed is important for streaming apps but not so much for single-player offline game apps. Give attention to the ones identified as necessary and ignore the ones that have the least impact on the final user.
Conclusion
In this article, I showed some of the challenges that surprised me when I stopped testing web apps and moved to mobile. In the end, mobile testing still is a really fun experience with tons of things to learn and appreciate.
The feedback you can get from app reviews is gold, and using device farms is an exciting challenge. So if you are thinking about asking your boss to be transferred to the mobile team, don’t be afraid if some things turn out to be out of your comfort zone. After all, these are the kind of things that make us evolve as testers! 💪