A few years ago I met a professor who told me that mobile games are not just video games today- people connect to them emotionally. This line stayed with me until today, even though a lot phased out from that time and I never thought of mobile games so deeply back then.
A Guide to Mobile Games Testing – Table of Contents
- Mobile Games Testing vs Mobile App Testing
- Key Areas to Look Out for in Mobile Games Testing
- Testing Mobile Games with TestProject
Researching about their impact on our lives today, I found out that in 2018, mobile games were downloaded 7.1 billion times on the Google play store and Apple’s app store. Three years later i.e. today, the number of downloads exceeds 11.5 billion 📈 Even if I dig down to the application level, “mobile games” is the fifth most downloaded category in the list of apps that do not come pre-installed.
When billions of devices are involved with a single software, even a small bug can bring down the reputation and business within seconds. The newer you are in the market, the faster you would fall. The main aim is to have a high number of active users on the application.
The more downloads, the more active users, and the more business that will drive. All of this creates a lot of pressure on the team behind releasing the software especially the testers.
A developer can make sure a module or functionality is working but the tester needs to confirm whether it works correctly or not. Such a situation calls for a strategy and a guide that can help you in mobile games testing 📙
This post will walk you through mobile games testing. We will understand how mobile games are different from mobile applications and cannot be treated that way for testing.
While a simple app can afford a simple module failure, games work in synchronization that leaves no room for bugs as one small bug can shut the complete application down. With this note, let’s see some of the things involved in mobile games testing and the roles they play.
As a software tester, you might think that I have been working in the testing field for x years and know automation scripts too. I am a fit candidate for this job role. On the other hand, as a recruiter recruiting for the games testing department can make you feel exactly the same way. But that’s actually not true.
Game testing and mobile application testing differ a lot. If you start considering mobile games equivalent to mobile applications, you may miss a lot of the things and may end up assembling the wrong team or you may land in a place not suited to you.
In this section, I will highlight the major points that split these two types of testing in the arena 💡
The first and most prioritized thing that comes to mind while thinking about mobile games is their current style of operation. They have changed a lot today compared to eight years back when flappy bird became a hit. Before that time, it was mostly only the player who enjoyed the game and shared screenshots of his scores to his friends.
Today, things have moved ahead to multi-player games (including profile modes) and they have been in demand because of high participation and multiple people playing with and to each other.
This makes a lot of things hard for mobile game testers. For example, a simple scenario is the position of each player. If the positions of people to each other or to some object are not accurate in all of the devices, the game will attract negative feedback. The same goes for consistent statistics displayed in different players’ profiles.
In the current software development life cycle processes, the release cycles have become too short in most of the product releases. In this scenario testers sometimes fall short of testing the software in the given time, and as a result are generally a sprint behind from production.
This is perfectly fine as long as you run a lot of automated tests and CI/CD pipelines on the software and make sure the newly built code works fine. But such post-production testing mechanisms are not an option in mobile games testing.
It is a very fragile process. If you are serving millions and billions of users, a small bug can make headlines the next day. The amount of risk involved while releasing a mobile game is much higher than releasing a mobile application. Therefore, it becomes challenging for the testers to complete their tasks in time before the release.
A recent research paper on image-matching techniques to test mobile games describes how we can make use of image-matching to determine the location and type of object in a mobile game. This works on the user’s actions and graphical elements of the game. Such types of analysis are extremely great for simulators as they provide accurate data in their execution.
On the contrary, what if I use the simulator again to determine the battery drain rate on the mobile device? 🔋 Would that be accurate? Or would a simulator work in such a situation?
Simulators do not play such a significant role when it comes to mobile games. The reason is that mobile games depend a lot on the type of hardware the mobile device has. A simple mobile application like e-com behaves almost similar up to a threshold hardware specification.
But you might hear someone purchasing a mobile device just to play a mobile game on it or people asking for a “gaming phone” in the shops. This is because the hardware has a major impact and simulators cannot provide satisfying data in this.
You must also consider things such as network variations and mobile screen frame per second specs that can only be evaluated through a mobile device.
Another major point in mobile game testing that you might find different from mobile app testing is the number of things one tester works on and the number of testers who work on one thing. Mobile games, as I said in the introductory section, have modules that work together in synchronization.
For example, consider the Amazon application. If the section “Your Orders” is not working, you might not be able to see your past orders done through it. That does not mean you won’t be able to place an order. So a tester who is testing “Your Orders” doesn’t need to deviate from this section and can write scripts specific to this module.
Mobile games testing is a bit different. For example, you are playing a game in which you can buy special powers from the game’s personal store which you can also boast off in front of your friends while playing. Now if you just test whether the power comes to the player’s account or not is not enough. He should be able to make use of it perfectly while playing.
This makes your involvement deeper into the application that you might not have to go through if testing other types of applications. Hence, you must be an all-rounder tester to dive into mobile games testing.
In the last section, I mentioned how mobile games involve one tester in too many places and occupy too many testers in one place. While the first part is discussed above, the latter one calls for teamwork.
When you are involved in multiple modules, you need to work with a lot of people who may have different thought processes, different approaches and different methods of working. As a recruiter, you should always look out for testers with such skills.
Apart from these major differences, you may find mobile games testing a bit complex, hectic, and focused too much on efficiency rather than functionality.
Mobile games testing has a lot more components than mobile app testing. Once we have passed the “figuring out” phase of determining the changes we will see from mobile app testing, we can focus on the mobile games and their effect on the users.
In 2021, mobile games generated a revenue of 90 billion dollars. Would you consider the same old testing pyramid graph or do you think there are several sections that need special focus in testing the mobile games? 🤔
This section will help you in determining the most important areas when you start testing mobile games. Some of them may be new and some of them might be enhanced as compared to the mobile applications. But eventually, they are the prime reasons for generating over 90 billion USD in a year.
The graphical performance is one of the most important considerations for both testers and the player. Gamers love being involved in high graphics supported by the device and accompanied by the gaming engine. No game can survive today in the market without being graphically excellent.
The performance metrics here are not a single parameter but a collection of a lot of them that every player keeps in mind. For example the battery drain rate. If your mobile game is draining the battery at an unusual rate, players tend to move away from such games.
The next factor is the load a GPU that will take place while running your game. GPUs run on high clock rates when you’re playing, which generates a lot of heat during the process and ultimately degrades the battery life.
GPUs are also equally important for other applications, and if your game starts to steal too much from other apps, the device will slow down 📱 This should always be avoided while testing the graphical performance.
One important factor that I can’t miss here is the frame-per-second aspect of a mobile game. If you have played a good graphic game on your mobile you will probably see the settings to reduce or upgrade the frames per second (or FPS) for the game.
As a tester you need to go for FPS testing and whether it enhances the experience on the appropriate devices or not. Other factors such as lagging on high FPS and no support on the device can be ignored here.
To summarize the graphical performance, the tester needs to ensure that it remains consistent on all the similar target devices. If I am getting a different graphics output and hardware utilization while my friend with similar specifications is getting different, gaming communities are very quick to pick that.
Next comes the user interface of the game. A lot of people put the UI on top of the graphical performance in their analysis. I do not feel that way. Being with a lot of gamers I have seen how people ignore the UI if the game seems promising or has that special thing that can keep them hooked (like counter-strike a long back).
Recently there has been too much focus on the “looks” of the game. This has been a direct result of an increased set of specifications that come equipped in budget devices today (refer to the above RAM capacity graph).
In 2013, Flappy Bird stole the show when it appeared on the play store. It had terrible UI (below image). If you somehow were able to avoid its success at that time, you would be surprised to know that Flappy Bird was earning $50,000 per day just 9 months after its release.
A mobile game consists of so many elements that work together to perform one action. For example, a game with a character inside does a lot of stuff. It can run, fight, shoot and sometimes even go to the gym and exercise 🏋️♀️
All these actions are extremely complex and require accurate positioning to make it look like a real person. User interface testing takes care of such things and guarantees that the interface is smooth as the developers claimed.
One small difference you might notice is how we talk about UI/UX together in a mobile application but the “user experience” part has made it to a separate section here. This is important because the user experience and the user interface are so different things in a mobile game that it’s best if we talk about them separately.
Most people spend 1 to 5 hours a day playing mobile games. I realized this high intensity when I noticed PUB-G started giving warnings when you have played for two hours in a day and even blocking you for 1 day after crossing the 5-hour mark ⏰
Today, mobile games are not just some application that you explore in your free time. People are deeply involved mentally in the games that it has become something more – an experience perhaps connecting people emotionally (as my professor said). An experience that cannot be taken for granted while testing it for players.
A bad user experience on a mobile game can prove to be as fatal as an application frequently crashing down. A tester is responsible for verifying a consistent experience in all the devices for all the players. A lot of things come under user experience for mobile games.
For example, tutorials are one of the first things a user experiences when they install the game for the first time. If it isn’t a lucid take on what the game offers, a player may take a lot more time to understand the rules and interface which is frustrating.
Another take on the user experience is the navigation of the menu and other options provided inside the mobile game. Testing for these elements is extremely important to understand how a player will perceive your game 2 days after installing.
So, do you think the best way to judge the user experience is through automation scripts or through automation plus user participation? I believe user participation plays a very important role in determining how your users will feel when they start playing your game. This is similar to usability testing except that this time the users should be chosen wisely.
The first slot should be users that have never played a game – the ones that can tell you a user experience for a beginner. The second slot should be the users that have played games but from different genres than your game. The final slot can be the users that play games and are accustomed to other games in the same genre.
You can take the feedback from the gamers and report it to developers or any other necessary team. This can be laid down as a better guide for your user experience testing. If you have anything to add to it, feel free to comment down below with your thoughts 👇
In January 2021, Samsung released their flagship mobile device called S21. I witnessed that great event while Samsung showed off the powerful device they were about to make public.
While I had seen their previous launch events too, one thing was different in this one. They had a separate section for introducing mobile games on the device. In that segment, Samsung announced their partnership with Microsoft to “enhance the gaming experience” for their users.
This intrigued me. I knew that Microsoft already has a gaming segment of their own called “Xbox” and I was interested to know how this can be merged. The answer was merging the Xbox controller to the S21 device.
Now you could see and experience the game on a mobile device and control it using your favorite and comfortable controller 🎮 This is one example of how external device testing plays a part in shaping the future of your mobile game. External devices have evolved just parallel to the mobile game evolution.
While earlier the player had just two controls – left and right, today you will see all your fingers busy in playing. For a tester, this might seem like a nightmare, but you never know when a user starts using another external device.
What if he gets accustomed to another game with some other external device and now feels so comfortable that wants to try everything with this? They always expect that to be perfect.
While working on mobile testing, always make sure all the external devices work, are supported, and most of all, they do not degrade the user experience in any way.
The final key area we need to focus on is security. One of the major reasons that make security testing important for mobile games testers is the use of third-party codes. Third-party code is the reusable code developed by someone else than the mobile game developer 👩💻
When we use third-party code, we don’t need to develop that module from scratch and it’s a very good practice in software development as it promotes the use of open-source code.
In a mobile game, you will find advertisement code or logger third-party code, etc. that is integrated to make the experience better or just earn some money. This heavy usage of third-party libraries resulted in the addition of a new section in the General Data Protection and Regulation (GDPR) focusing on this.
As a mobile games tester, you should be aware of the vulnerabilities of using such a third-party code. For example, in future releases, the third-party code may start fetching the data from the user and start saving it in their database. You can avoid this by only updating the library at your end after scrutinizing it.
Penetration testing is a good measure while thinking about the security of the user’s data and looking out for all the data leaks. Data protection is a very important thing. If you read recent headlines, you might understand why.
The last thing to discuss before wrapping up this guide is the tool that you can use to perform the testing of the mobile game. The tool will play an important part in the mobile games testing journey because of the number of devices you will target. Mobile devices are getting cheaper in the market while the mobile games requirement hasn’t been extremely high.
A mobile game with a requirement of a minimum of 2GB RAM can easily run at its full strength on a 4GB device. If your mobile is targeting around 25% of all the device holders, you still aim for at least 80 million devices in the United States alone, which is huge! Such a large target requires a capable tool that can manage all this.
TestProject is a good tool to start your journey with and stick to, as it is easy to understand for beginners and powerful to run complex test cases for experts. The primary benefit of choosing it is that it comes with codeless scripting capabilities. For someone who has a testing talent but is not able to write complex scripts, this can prove to be the best tool.
If you are someone who loves coding scripts, TestProject provides that option too. The best method is to go codeless for basic tests with the help of 1500+ automation actions that come with it and then use code to enhance your cases until you’re satisfied 💪
TestProject also comes equipped with offline capabilities (which stands a reason sometimes for a few testers) so you can save your tests locally. Once you are ready, sync the code and Git repositories with the TestProject account. Later, you can use the same tool for connecting CI/CD tools and pushing changes at a lightning speed. You can read more about it here.
The tool also delivers support for both popular mobile operating systems – Android and iOS. Along with this, it can integrate with BrowserStack and SauceLabs to give you access to use their thousands of real devices and TestProject’s power of automation.
This tool can boost up your mobile games automation testing journey and is free to use for all 🔥
Mobile games are rising 11% from quarter to quarter on Google play store. This staggering rise has been due to an ever-increasing number of gamers (especially Q1 of 2020) and feasible high-end mobile devices. While all this looks superb for a user, it increases the responsibilities of a mobile games tester year by year.
This post gave you an insight into what a mobile game tester has to look forward to and how it is different from the testing of mobile apps. We also looked at why these key areas deserve special attention and what different approaches we can follow in the journey.
In the end, good and effective testing cannot be done without good and effective tools. TestProject seems to fit the description here and can watch your back like a good friend. Surely there are other tools available and other platforms to put down your tests. But since this tool has got everything and is free to use, it deserves a shot.
Think like a gamer and you will find a lot of hidden issues that weren’t known before 💫