Top Android Testing Frameworks You Need to Know

Android applications are perhaps the first choice to download, due to the market they share with the world. I have seen a lot of applications that have been developed and run on Android for a long time before launching them on the iOS store.

Although, the strict iOS scrutiny does play a role in the number of applications available on its store. Launching first for Android isn’t about scrutiny or being rejected, it’s just about how much you can reach with your launch and which store matters the most for the business.

According to GlobalStats, Android has 72.73% of the market share while iOS has only around 26%. In simple words, out of 100 people, 72 are using Android OS while 26 use iOS. So, if you need to reach a broader market, what would you prefer? 💹

Here comes the importance of Android testing and the frameworks that can help us facilitate it. Since developing for Android comes with this advantage, it also comes with a high risk. If your application is not able to perform as intended and has several glitches, you are jeopardizing your business in front of the audience.

This could have fatal effects. The best way to minimize the risk is to use a framework suitable to your requirements and skills. On the same track, in this post, we will go through several popular Android testing frameworks and list their greatest strengths. Let’s get started!

Appium
TestProject
Espresso
Calabash
Selendroid
Robotium
Android automation frameworks at a glance
Vote for your favorite Android testing frameworks

1) Appium

Appium is a multi-talented, versatile open-source testing framework that tests Android, iOS, and Windows applications through automation. The main aim of Appium is to relieve the users from being bound into a single technology stack and carry on with its anomalies.

Appium allows flexible test execution and has been in the community for a long time. A large community will help you in every corner of the Android testing process.

Pros

No app recompiling: Appium lives on the philosophy that the user should not be required to recompile the application or modify it for testing.
Tool independent: As I mentioned, it does not let you get bound into a single tech stack. It’s flexible and can operate with a lot of tools. So, if you have a favorite tool, you can integrate Appium into it and enjoy testing.
Open-source: It is open-source and hence free to use for everyone.
Language independent: Since Appium works with every testing framework, you can choose a framework that supports your language. Hence, it can support any language you want to write tests in.
Cross-platform: It’s cross-platform in nature so you can use the same code among different operating systems. This also enhances the code reusability.

Cons

If you are not using Selendroid, you may not be able to test for the Android version below 4.2.
Appium has been observed to be a bit slower than other testing frameworks in test execution.

Code example

def getresults(self):
        displaytext = self.driver.find_element_by_accessibility_id("CalculatorResults").text
        displaytext = displaytext.strip("Display is " )
        displaytext = displaytext.rstrip(' ')
        displaytext = displaytext.lstrip(' ')
        return displaytext


    def test_initialize(self):
        self.driver.find_element_by_name("Clear").click()
        self.driver.find_element_by_name("Seven").click()
        self.assertEqual(self.getresults(),"7")
        self.driver.find_element_by_name("Clear").click()

Bottom line: Appium is a great software that has existed for a long time and has proved its worth. It gives the flexibility of test framework, language, and platform which is everything a tester wants. Appium is a very useful tool for your Android testing and I am sure you will love it.

2) TestProject

TestProject is made for the community. Hence, it is free to use for everyone. TestProject is built on top of proven and trusted frameworks – Selenium and Appium.

Hence, you not only get the benefits of Selenium and Appium but also the added features of TestProject itself. The tool is available for Android app testing, iOS app testing (iOS testing on Windows too!), and even web app testing. It is powerful, crisp, and provides unparalleled advantages to the testers & developers.

Pros

Recorder: TestProject comes with advanced recording capabilities that help in increasing the code reusability by extending the recorded tests across suites.
Self-healing recorder: The recorder is artificial intelligence-powered and possesses self-healing capabilities. Therefore, any change in the location of the element is detected and the tests are adjusted accordingly.
Free to use: It’s free to use for everyone and therefore, you can try it and continue your usage based on experience, without worrying about emptying your pockets.
Dynamic element support: The automation framework comes with built-in support for dynamic elements.
Adaptive-wait technology: The adaptive-wait technology adapts itself to the loading time of the web page. This prevents a few test cases from failing just because the web page took a little more time to load.
Cloud-enabled: The tool comes with hybrid support for cloud-based testing along with local on-premise testing.
Centralized dashboard: It has a very attractive and unique dashboard where a tester can analyze their tests in a single place, no matter where they executed them.
Addons library: It comes with loads of addons to enhance the capabilities of the tool according to the tester or the project. This way you can also customize the tool as per your preferences.
Schedule and run: The tool offers “schedule and run” features for the tests, benefiting from built-in CI flows.

Cons

People coming directly from Appium might find some of their integrations missing from the tool.
TestProject does not offer an in-house device farm solution (they do though have built-in integrations to either BrowserStack or Sauce Labs). You need to connect your physical real devices via USB cable or can use emulators/simulators.

Code example

driver.findElement(By.id("name")).sendKeys("John Smith");
driver.findElement(By.id("password")).sendKeys("12345");
driver.findElement(By.id("login")).click();

boolean passed = driver.findElement(By.id("logout")).isDisplayed();
if (passed) {
    System.out.println("Test Passed");
} else {
    System.out.println("Test Failed");
}

Bottom line: TestProject is a great free tool that is gaining appreciation from all over the world. Communities and forums have picked it up for its superior strengths and swiftness in the Android testing phase. No matter your skill set, if you love Appium, you are going to fall in love with TestProject’s extremely intuitive & powerful solution for your E2E mobile testing 📲

3) Espresso

Espresso is an Android testing framework developed by Google and follows open-source development paradigms. Espresso uses JUnit as its base and is, therefore, a familiar and easy android testing framework for the testers. The tool is extremely efficient in the user interface testing of the android mobile applications.

Pros

Easy and predictable: Espresso is easy to use and the scripts are predictable in nature because of their familiarity with the JUnit.
Fast: This Google automation tool is faster in processing even with the “wait” and “sleep” in the code. This is achieved by automatically detecting the idle thread and when the tests are in other states.
Highly stable: For the same reasons described in the previous point, Espresso is highly stable in nature.
Open-source: It is an open-source framework and hence the abilities can be extended to other tools.

Cons

UI-based only: This Android framework is just UI-based which raises the need to learn multiple frameworks for a completing testing phase. This can be a deal-breaker for some testers.
Not cross-platform: It is not a cross-platform android testing framework. This means you have to construct multiple scripts for different platforms.

Code example

@Test
fun greeterSaysHello() {
    onView(withId(R.id.name_field)).perform(typeText("Steve"))
    onView(withId(R.id.greet_button)).perform(click())
    onView(withText("Hello Steve!")).check(matches(isDisplayed()))
}

Bottom line: Espresso is a very comforting tool that asks for a minimum job from the tester. The problem is that it is just best for two things: Android and UI-based testing only. When other frameworks are offering much more, some testers are bound to look the other way even if Espresso’s synchronization features are extraordinary.

4) Calabash

Calabash is an automation testing framework used with both Android and iOS applications. The power of calabash is its test creation which does not require any coding skills from the tester 👩‍💻

The framework works exceptionally well with real devices and provides real-time feedback for automation tests. Calabash is open-source in development nature and can work around with hybrid applications as well.

Pros

BDD-enabled: Calabash uses behavioral driven development that is easier and faster to construct.
Automatic UI interactions: It enables automatic UI interactions such as pressing a button, etc.
Easy CI/CD integrations: Can be integrated with popular CI/CD tools very easily.
Real device compatible: It has been praised for running on real devices by the testers. This feature allows the testers to assess the mobile device in real conditions which are better than simulators.

Cons

Calabash server on Android can only be used to test the UI inside the application code. This problem does not persist with Appium.
Notification testing is currently not available in Calabash.

Code example

## Invalid login ##
    Given /^I am about to login$/ do

      welcome = page(WelcomePage).await    
      @page = welcome.wordpress_blog
      
    end

    When /^I enter invalid credentials$/ do
      @page = @page.login(USERS[:invalid])
    end

    Then /^I am presented with an error message to correct credentials$/ do
      @page.assert_invalid_login_message
      screenshot_embed
    end

Bottom line: Calabash is a good tool for acceptance testing in Android. But, it comes with a few limitations that are hard to ignore. While I can praise Calabash for its real-time feedback and real device testing, it makes me think twice before using it. I find Calabash more powerful for iOS applications rather than for Android-based testing. Although it depends on the project and the team too.

5) Selendroid

Selendroid is a word originating from the amalgamation of Selenium and Android. You can also remember this framework as “Selenium for Android“. Since the framework is Selenium-based, it comes with all the powerful strengths of Selenium and this has been a USP in the market for it.

Selendroid uses Selenium 2.0 to write the test cases which a lot of automation testers are already aware of. Also, it can be used with android native applications or hybrid applications.

Pros

Android webview enabled: Selendroid tests the mobile web using the Android mobile view app.
AUI support: The framework supports advanced user interaction APIs and multiple gestures to facilitate efficient testing of the UI of the application.
Hot plugging supported: It supports the hot plugging of the hardware device.
Vast use: It can be used on simulators, emulators, real devices, and even with the Selenium grid which is very helpful.
JSON Wire Protocol compatible: Selendroid is JSON Wire protocol supported which extends its functionality for the testers.
Open-source: The framework is open-source and therefore can be used by anyone for free.

Cons

Selendroid is a heavy framework when it comes to computer resource utilization. A minimum recommended RAM would be 4GB if you want to use it.
Complained of being a bit slow but I guess that depends on the configurations and addons you have installed.

Code example

SelendroidCapabilities capa = new SelendroidCapabilities("io.selendroid.testapp:0.17.0");

WebDriver driver = new SelendroidDriver(capa);
WebElement inputField = driver.findElement(By.id("my_text_field"));
Assert.assertEquals("true", inputField.getAttribute("enabled"));
inputField.sendKeys("Selendroid");
Assert.assertEquals("Selendroid", inputField.getText());
driver.quit();

Bottom line: Selendroid automation testing framework is made especially for Android UI testing. It leverages Selenium and if you are in love with it, you will definitely love Selendroid. The framework is open-source and you can give it a try and let me know how you feel about it.

6) Robotium

The last of the open-source Android testing frameworks I will be taking up in this list is the Robotium framework. Robotium is a high-performance framework to handle multiple Android activities 🤹‍♂️

The main strength of the framework is the UI tests in which it is extremely proficient. It can be used to write functional, acceptance, and system tests in addition to the UI. Robotium comes with full support to native and hybrid applications written for Android operating systems.

Pros

Minimum application knowledge: You don’t need to be an expert in the application code for testing it with the Robotium automation framework. You can test apk files or source code as you like.
Ability to handle multiple activities: Robotium can handle multiple Android activities automatically.
Time effective: The framework writes test cases that are readable, easy and fast to write, and maintainable.
CI enabled: It can be easily integrated with Maven, Gradle or Ant for the purposes of continuous integration.

Cons

Robotium is incapable of handling the flash components.
It has a lot of issues when it comes to the notification or notification bar testing in android devices.
Even though Robotium claims of being fast on its official page, the community has repeatedly emphasized the slower test executions.

Code example

 * Constructs this object.
 * 
 * @param viewFetcher the {@code ViewFetcher} instance
   * @param waiter the {@code Waiter} instance
 */

public Checker(ViewFetcher viewFetcher, Waiter waiter){
  this.viewFetcher = viewFetcher;
  this.waiter = waiter;
}

Bottom line: Robotium is more a UI testing framework than a complete solution in this field. Moreover, I have found a lot of people raising the issues of slow test execution and the inability to handle multiple activities. This can be a dealbreaker as it will waste time and raise a need to learn another framework.

Android automation frameworks at a glance

The following table summarises all the Android automation frameworks discussed in this post:

*W- Windows; M- macOS; L- Linux

📌 If you want to explore the best frameworks for iOS, you can check out my guide to iOS testing frameworks.

Vote for your favorite Android testing framework

This post walked you through the best Android automation testing frameworks that you can look forward to in your testing career. But, these are not all equivalent in power or capabilities. They even offer different features which are written in different languages. However, there is no “one” best framework among them.

Which one you should choose depends on the project and the requirements needed. It is therefore recommended to go through each of them and carefully analyze their powers.

I also understand that even if they are among the most popular Android automation frameworks in the community, they are not the only ones on that list. You may be working or have worked on a different tool that you found comparable to these and would love to see here.

I will leave you to decide which framework matches your team and project the best, and would love to know your recommendations in the comments 😀

Top Android Testing Frameworks You Need to Know

Table of Contents

1) Appium

Pros

Cons

Code example

2) TestProject

Pros

Cons

Code example

3) Espresso

Pros

Cons

Code example

4) Calabash

Pros

Cons

Code example

5) Selendroid

Pros

Cons

Code example

6) Robotium

Pros

Cons

Code example

Android automation frameworks at a glance

Vote for your favorite Android testing framework

Leave a Reply Cancel reply