5 Simple Steps to Appium Mobile Automation Reports in C#

It’s no secret that these days mobile testing has an increasingly important role in many organizations, and Appium is one of the most popular tools for mobile automation. Combine it with TestProject, and you’ll get Appium powered by AI and Self Healing capabilities, as well as bonus awesome HTML or PDF test reports to showcase the results of your automated tests 💪📊

In this tutorial, I’ll walk you through all the steps to create and run a simple test on the Calculator Android app, and to generate the test report.

What is Appium?

Appium is an open-source framework that uses the WebDriver protocol to automate tests for native, hybrid, and mobile web apps. It can be used on Android, iOS, and Windows apps. For this tutorial, I will assume you already have Visual Studio with Android tools installed and have a basic understanding of tools such as Selenium and Appium.

Why use TestProject SDK?

An important part of the test automation process is being able to demonstrate what the automation framework actually tested, and what were the outcomes of the tests. Appium itself will not create any reports, so you’ll need some outside help. With TestProject, most of the work is done for you – after a quick setup, there’s no need to worry if the results will be saved or if they will still look good.

TestProject’s automation test reports are saved by default in HTML format in your account, and they are downloadable in PDF format. They are customizable and can be more or less detailed, depending on your needs. You can add custom project and job names, and include logging information or screenshots at the step level. You can also set the jobs to automatically send the reports by email or even by Slack.

5 Step Appium Mobile Automation Framework with TestProject

Set up the TestProject account

The first step is signing up to TestProject. It’s free forever, so setting up an account requires no credit card, and no additional info, just your name and email address. To get started, navigate to the Sign Up page and enter your profile information. Once connected, you will have access to the TestProject dashboard, where you can see all your projects, monitor your tests, and see your reports.

Download the TestProject Agent

Next up, you’ll need to download and install the TestProject agent, so the tests you run in your IDE will be recognized and monitored on the dashboard. This step is pretty straightforward, too, all you need to do is navigate to the Agents tab and click on Download an Agent, or directly select your operating system. You can install the TestProject agent on Windows, Mac OS X, and Linux, and it’s also available on Docker.

Once installed, start it, then reopen the Agents menu and select Register an Agent, give the agent a name and you’re good to go – you’ll be able to see your agent in the Agents list. You can see a screenshot of the Agents tab of my account, with my clever agent name – Andreea’s Agent. 🙂

It might not seem very important, but if you work in a team, it’s good to see which agent the test run used:

Write the first Appium test using TestProject

One option is to use TestProject’s record and play functionality. Make sure the test agent is running, create a new test in the TestProject app, record the steps, then either continue editing and running the test from the TestProject portal, or generate the code in Java or C#:

Or maybe you prefer to write your own code, in which case, follow the next steps. I will be using C# for this tutorial, but you can also do the same using either the Java or Python TestProject SDK.

First, create a new .NET Core project in Visual Studio. I’m a fan of NUnit, so I’ll create an NUnit project, but if you prefer a different test framework, feel free to use it.

You’ll notice that you already have a test class with a Setup and a Test method.

Next, add the TestProject SDK to the project – from the Solution Explorer, you can right-click the project name, select Manage NuGet Packages, and browse for TestProject SDK.

That’s all the preparation needed to get started. In the SetUp class, which runs before the test, create a new runner.

[SetUp]
public void SetUp()
{
    runner = new RunnerBuilder(DevToken).AsAndroid(DeviceUDID, AndroidPackage, AndroidActivity).Build();
}

Let’s also go through the variables and declare them, otherwise the build will fail:

The DevToken is s developer key provided by TestProject. You can find it in the Integrations tab:

The DeviceUDID is your phone’s ID (or the emulated phone ID, if you are using an emulator)
The AndroidPackage and the AndroidActivity are the .apk and the app activity you want to use in the test. If you’re not familiar with this part, you can check out the Appium documentation.

The variables should look something like this for the Calculator App:

public static string DevToken = "YOUR_DEV_TOKEN";
public static string DeviceUDID = "ANDROID_DEVICE_ID";
public static string AndroidPackage = "com.android.calculator2";
public static string AndroidActivity = "com.android.calculator2.Calculator";

And now, create a new class for the test itself. My test is to check that I get the correct multiplication result, so see my code below:

using OpenQA.Selenium;
using TestProject.Common.Enums;
using TestProject.SDK;
using TestProject.SDK.Tests;
using TestProject.SDK.Tests.Helpers;

namespace TestProject_Appium
{
    class AndroidTest : IAndroidTest
    {
    public ExecutionResult Execute(AndroidTestHelper helper)
    {
      var driver = helper.Driver;
      var report = helper.Reporter;

      driver.Timeout = 15000;
      
      report.Step("Reset App", driver.TestProject().ResetApp(), TakeScreenshotConditionType.Always);

      report.Step("Tap '5'", driver.TestProject().Tap(By.Id("com.android.calculator2:id/digit_5")), 
        TakeScreenshotConditionType.Always);

      report.Step("Tap 'multiply'", driver.TestProject().Tap(By.Id("com.android.calculator2:id/op_mul")), 
        TakeScreenshotConditionType.Always);

      report.Step("Tap '6'", driver.TestProject().Tap(By.Id("com.android.calculator2:id/digit_6")), 
        TakeScreenshotConditionType.Always);
    
      report.Step("Tap 'equals'", driver.TestProject().Tap(By.Id("com.android.calculator2:id/eq")), 
        TakeScreenshotConditionType.Always);
      
      report.Step("'result' contains text '36'?", driver.TestProject().ContainsText(By.Id("com.android.calculator2:id/result"), "30"), 
        TakeScreenshotConditionType.Always);

      return ExecutionResult.Passed;
    }
  }
}

Let’s pause and analyze it:

Since the test should run on Android, the test class needs to implement the IAndroidTest interface.
The AndroidTestHelper provides access to driver and other Android actions resources (such as the reporter).
Using the Step method, we can add for each step a description, a condition, which will tell if the step is passed or failed, and if we want to include a screenshot to the step. I want my report to include screenshots for each step regardless of the test result. You can choose to take a screenshot only for failed steps if that works better for you.

What we did here is run a test that calculates 5 times 6 in the Android calculator, and then checks that it returned 30.

Run the test

To actually run the test, go back to the first class created. In the [Test] method, add this line:

runner.Run(new AndroidTest());

and declare the runner variable at the beginning of the class:

Runner runner;

The class should like this now:

using NUnit.Framework;
using System;
using TestProject.SDK;

namespace TestProject_Appium
{
    public class Tests
    {

        Runner runner;

        public static string DevToken = "YOUR_DEVELOPER_TOKEN";
        public static string DeviceUDID = "device uid";
        public static string AndroidPackage = "com.android.calculator2";
        public static string AndroidActivity = "com.android.calculator2.Calculator";


        [SetUp]
        public void SetUp()
        {

            runner = new RunnerBuilder(DevToken).AsAndroid(DeviceUDID, AndroidPackage, AndroidActivity).Build();
        }

        [Test]
        public void RunAndroidTest()
        {
            runner.Run(new AndroidTest());
        }        
    }
}

I renamed the class and the test method so the names make more sense.

The last step – simply build your project and run the test. That’s it.

Check the test reports generated by TestProject

Now go to your TestProject account and open the Reports tab. You should see the result of the test. If all went well, the report will show a passing percentage of 100% of tests passed. On the right side, you can see all the steps descriptions, and the screenshots, if you decided to always include a screenshot as I did.

It’s always a good idea to test our tests, to make sure that we didn’t get a false positive. So replace this line:

report.Step("'result' contains text '30'?", driver.TestProject().ContainsText(By.Id("com.android.calculator2:id/result"), "30"), 
  TakeScreenshotConditionType.Always);

with this:

report.Step("'result' contains text '30'?", driver.TestProject().ContainsText(By.Id("com.android.calculator2:id/result"), "36"), 
  TakeScreenshotConditionType.Always);

Obviously, 5 times 6 is NOT 36, so the test will fail. Rebuild, rerun, refresh the Reports page:

The report for the new test run shows a 100% failing percentage, and you can filter the steps to see only the failed steps. Here you’ll have a screenshot of the failed step:

You can see some cool statistics for your tests run in the Trends and Reports page of TestProject, such as the ratio between successful and failed executions, distribution of tests executions per Agent, distribution of passed/failed/skipped per platform.

You can also proceed to download the reports in PDF format if you want them stored locally or shared with other members of your team.

The takeaway

In 5 easy steps, you can automatically get great reports for your Appium mobile automation, stored on the cloud in HTML format, or locally as PDFs 💥