Android applications are perhaps the first choice to download, due to the market they share with the world. I have seen a lot of applications that have been developed and run on Android for a long time before launching them on the iOS store.
Although, the strict iOS scrutiny does play a role in the number of applications available on its store. Launching first for Android isn’t about scrutiny or being rejected, it’s just about how much you can reach with your launch and which store matters the most for the business.
According to GlobalStats, Android has 72.73% of the market share while iOS has only around 26%. In simple words, out of 100 people, 72 are using Android OS while 26 use iOS. So, if you need to reach a broader market, what would you prefer? πΉ
Here comes the importance of Android testing and the frameworks that can help us facilitate it. Since developing for Android comes with this advantage, it also comes with a high risk. If your application is not able to perform as intended and has several glitches, you are jeopardizing your business in front of the audience.
This could have fatal effects. The best way to minimize the risk is to use a framework suitable to your requirements and skills. On the same track, in this post, we will go through several popular Android testing frameworks and list their greatest strengths. Let’s get started!
Table of Contents
- Appium
- TestProject
- Espresso
- Calabash
- Selendroid
- Robotium
- Android automation frameworks at a glance
- Vote for your favorite Android testing frameworks
1) Appium
Appium is a multi-talented, versatile open-source testing framework that tests Android, iOS, and Windows applications through automation. The main aim of Appium is to relieve the users from being bound into a single technology stack and carry on with its anomalies.
Appium allows flexible test execution and has been in the community for a long time. A large community will help you in every corner of the Android testing process.
Pros
- No app recompiling: Appium lives on the philosophy that the user should not be required to recompile the application or modify it for testing.
- Tool independent: As I mentioned, it does not let you get bound into a single tech stack. It’s flexible and can operate with a lot of tools. So, if you have a favorite tool, you can integrate Appium into it and enjoy testing.
- Open-source: It is open-source and hence free to use for everyone.
- Language independent: Since Appium works with every testing framework, you can choose a framework that supports your language. Hence, it can support any language you want to write tests in.
- Cross-platform: It’s cross-platform in nature so you can use the same code among different operating systems. This also enhances the code reusability.
Cons
- If you are not using Selendroid, you may not be able to test for the Android version below 4.2.
- Appium has been observed to be a bit slower than other testing frameworks in test execution.
Code example
def getresults(self): displaytext = self.driver.find_element_by_accessibility_id("CalculatorResults").text displaytext = displaytext.strip("Display is " ) displaytext = displaytext.rstrip(' ') displaytext = displaytext.lstrip(' ') return displaytext def test_initialize(self): self.driver.find_element_by_name("Clear").click() self.driver.find_element_by_name("Seven").click() self.assertEqual(self.getresults(),"7") self.driver.find_element_by_name("Clear").click()
Bottom line: Appium is a great software that has existed for a long time and has proved its worth. It gives the flexibility of test framework, language, and platform which is everything a tester wants. Appium is a very useful tool for your Android testing and I am sure you will love it.
2) TestProject
TestProject is made for the community. Hence, it is free to use for everyone. TestProject is built on top of proven and trusted frameworks – Selenium and Appium.
Hence, you not only get the benefits of Selenium and Appium but also the added features of TestProject itself. The tool is available for Android app testing, iOS app testing (iOS testing on Windows too!), and even web app testing. It is powerful, crisp, and provides unparalleled advantages to the testers & developers.
Pros
- Recorder: TestProject comes with advanced recording capabilities that help in increasing the code reusability by extending the recorded tests across suites.
- Self-healing recorder: The recorder is artificial intelligence-powered and possesses self-healing capabilities. Therefore, any change in the location of the element is detected and the tests are adjusted accordingly.
- Free to use: It’s free to use for everyone and therefore, you can try it and continue your usage based on experience, without worrying about emptying your pockets.
- Dynamic element support: The automation framework comes with built-in support for dynamic elements.
- Adaptive-wait technology: The adaptive-wait technology adapts itself to the loading time of the web page. This prevents a few test cases from failing just because the web page took a little more time to load.
- Cloud-enabled: The tool comes with hybrid support for cloud-based testing along with local on-premise testing.
- Centralized dashboard: It has a very attractive and unique dashboard where a tester can analyze their tests in a single place, no matter where they executed them.
- Addons library: It comes with loads of addons to enhance the capabilities of the tool according to the tester or the project. This way you can also customize the tool as per your preferences.
- Schedule and run: The tool offers “schedule and run” features for the tests, benefiting from built-in CI flows.
Cons
- People coming directly from Appium might find some of their integrations missing from the tool.
- TestProject does not offer an in-house device farm solution (they do though have built-in integrations to either BrowserStack or Sauce Labs). You need to connect your physical real devices via USB cable or can use emulators/simulators.
Code example
driver.findElement(By.id("name")).sendKeys("John Smith"); driver.findElement(By.id("password")).sendKeys("12345"); driver.findElement(By.id("login")).click(); boolean passed = driver.findElement(By.id("logout")).isDisplayed(); if (passed) { System.out.println("Test Passed"); } else { System.out.println("Test Failed"); }
Bottom line: TestProject is a great free tool that is gaining appreciation from all over the world. Communities and forums have picked it up for its superior strengths and swiftness in the Android testing phase. No matter your skill set, if you love Appium, you are going to fall in love with TestProject’s extremely intuitive & powerful solution for your E2E mobile testing π²
3) Espresso
Espresso is an Android testing framework developed by Google and follows open-source development paradigms. Espresso uses JUnit as its base and is, therefore, a familiar and easy android testing framework for the testers. The tool is extremely efficient in the user interface testing of the android mobile applications.
Pros
- Easy and predictable: Espresso is easy to use and the scripts are predictable in nature because of their familiarity with the JUnit.
- Fast: This Google automation tool is faster in processing even with the “wait” and “sleep” in the code. This is achieved by automatically detecting the idle thread and when the tests are in other states.
- Highly stable: For the same reasons described in the previous point, Espresso is highly stable in nature.
- Open-source: It is an open-source framework and hence the abilities can be extended to other tools.
Cons
- UI-based only: This Android framework is just UI-based which raises the need to learn multiple frameworks for a completing testing phase. This can be a deal-breaker for some testers.
- Not cross-platform: It is not a cross-platform android testing framework. This means you have to construct multiple scripts for different platforms.
Code example
@Test fun greeterSaysHello() { onView(withId(R.id.name_field)).perform(typeText("Steve")) onView(withId(R.id.greet_button)).perform(click()) onView(withText("Hello Steve!")).check(matches(isDisplayed())) }
Bottom line: Espresso is a very comforting tool that asks for a minimum job from the tester. The problem is that it is just best for two things: Android and UI-based testing only. When other frameworks are offering much more, some testers are bound to look the other way even if Espresso’s synchronization features are extraordinary.
4) Calabash
Calabash is an automation testing framework used with both Android and iOS applications. The power of calabash is its test creation which does not require any coding skills from the tester π©βπ»
The framework works exceptionally well with real devices and provides real-time feedback for automation tests. Calabash is open-source in development nature and can work around with hybrid applications as well.
Pros
- BDD-enabled: Calabash uses behavioral driven development that is easier and faster to construct.
- Automatic UI interactions: It enables automatic UI interactions such as pressing a button, etc.
- Easy CI/CD integrations: Can be integrated with popular CI/CD tools very easily.
- Real device compatible: It has been praised for running on real devices by the testers. This feature allows the testers to assess the mobile device in real conditions which are better than simulators.
Cons
- Calabash server on Android can only be used to test the UI inside the application code. This problem does not persist with Appium.
- Notification testing is currently not available in Calabash.
Code example
## Invalid login ## Given /^I am about to login$/ do welcome = page(WelcomePage).await @page = welcome.wordpress_blog end When /^I enter invalid credentials$/ do @page = @page.login(USERS[:invalid]) end Then /^I am presented with an error message to correct credentials$/ do @page.assert_invalid_login_message screenshot_embed end
Bottom line: Calabash is a good tool for acceptance testing in Android. But, it comes with a few limitations that are hard to ignore. While I can praise Calabash for its real-time feedback and real device testing, it makes me think twice before using it. I find Calabash more powerful for iOS applications rather than for Android-based testing. Although it depends on the project and the team too.
5) Selendroid
Selendroid is a word originating from the amalgamation of Selenium and Android. You can also remember this framework as “Selenium for Android“. Since the framework is Selenium-based, it comes with all the powerful strengths of Selenium and this has been a USP in the market for it.
Selendroid uses Selenium 2.0 to write the test cases which a lot of automation testers are already aware of. Also, it can be used with android native applications or hybrid applications.
Pros
- Android webview enabled: Selendroid tests the mobile web using the Android mobile view app.
- AUI support: The framework supports advanced user interaction APIs and multiple gestures to facilitate efficient testing of the UI of the application.
- Hot plugging supported: It supports the hot plugging of the hardware device.
- Vast use: It can be used on simulators, emulators, real devices, and even with the Selenium grid which is very helpful.
- JSON Wire Protocol compatible: Selendroid is JSON Wire protocol supported which extends its functionality for the testers.
- Open-source: The framework is open-source and therefore can be used by anyone for free.
Cons
- Selendroid is a heavy framework when it comes to computer resource utilization. A minimum recommended RAM would be 4GB if you want to use it.
- Complained of being a bit slow but I guess that depends on the configurations and addons you have installed.
Code example
SelendroidCapabilities capa = new SelendroidCapabilities("io.selendroid.testapp:0.17.0"); WebDriver driver = new SelendroidDriver(capa); WebElement inputField = driver.findElement(By.id("my_text_field")); Assert.assertEquals("true", inputField.getAttribute("enabled")); inputField.sendKeys("Selendroid"); Assert.assertEquals("Selendroid", inputField.getText()); driver.quit();
Bottom line: Selendroid automation testing framework is made especially for Android UI testing. It leverages Selenium and if you are in love with it, you will definitely love Selendroid. The framework is open-source and you can give it a try and let me know how you feel about it.
6) Robotium
The last of the open-source Android testing frameworks I will be taking up in this list is the Robotium framework. Robotium is a high-performance framework to handle multiple Android activities π€ΉββοΈ
The main strength of the framework is the UI tests in which it is extremely proficient. It can be used to write functional, acceptance, and system tests in addition to the UI. Robotium comes with full support to native and hybrid applications written for Android operating systems.
Pros
- Minimum application knowledge: You don’t need to be an expert in the application code for testing it with the Robotium automation framework. You can test apk files or source code as you like.
- Ability to handle multiple activities: Robotium can handle multiple Android activities automatically.
- Time effective: The framework writes test cases that are readable, easy and fast to write, and maintainable.
- CI enabled: It can be easily integrated with Maven, Gradle or Ant for the purposes of continuous integration.
Cons
- Robotium is incapable of handling the flash components.
- It has a lot of issues when it comes to the notification or notification bar testing in android devices.
- Even though Robotium claims of being fast on its official page, the community has repeatedly emphasized the slower test executions.
Code example
* Constructs this object. * * @param viewFetcher the {@code ViewFetcher} instance * @param waiter the {@code Waiter} instance */ public Checker(ViewFetcher viewFetcher, Waiter waiter){ this.viewFetcher = viewFetcher; this.waiter = waiter; }
Bottom line: Robotium is more a UI testing framework than a complete solution in this field. Moreover, I have found a lot of people raising the issues of slow test execution and the inability to handle multiple activities. This can be a dealbreaker as it will waste time and raise a need to learn another framework.
Android automation frameworks at a glance
The following table summarises all the Android automation frameworks discussed in this post:
*W- Windows; M- macOS; L- Linux
π If you want to explore the best frameworks for iOS, you can check out my guide to iOS testing frameworks.
Vote for your favorite Android testing framework
This post walked you through the best Android automation testing frameworks that you can look forward to in your testing career. But, these are not all equivalent in power or capabilities. They even offer different features which are written in different languages. However, there is no “one” best framework among them.
Which one you should choose depends on the project and the requirements needed. It is therefore recommended to go through each of them and carefully analyze their powers.
I also understand that even if they are among the most popular Android automation frameworks in the community, they are not the only ones on that list. You may be working or have worked on a different tool that you found comparable to these and would love to see here.
I will leave you to decide which framework matches your team and project the best, and would love to know your recommendations in the comments π