logo logo

How Can You Scale Your REST API Testing?

main post image

REST API is a powerful way to run tests, however it’s not always enough. There are some pitfalls that cannot be spotted by REST API testing alone. And here comes the importance of end-to-end test automation.

For example, during the process of writing this article, I tested NASA’s public API and discovered a bug in one of their API schemas. I could not have discovered this bug unless I would have gone through the entire cycle of end-to-end testing. This clearly emphasizes the vast importance that end-to-end testing has in a product’s life-cycle, and we should not underestimate the power of combining API and UI automated testing for better coverage and quality.

TestProject‘s end-to-end test automation framework just released a new open source addon for RESTful API clients. It provides actions to send HTTP/S requests using GET, POST, PUT and DELETE methods and is completely open source. You can see the documentation and source code for it here.

In this article, we will discuss how to create automated REST API tests with TestProject and we will:

  1. Send API commands to the server and validate responses.
  2. Use values from responses as parameters in test steps.
  3. Combine REST API and recorded UI steps within the same automated test to achieve end-to-end testing.
  4. Analyze reports.

To start working you will need to make sure that:

  1. You have a TestProject account, you can get it for FREE here.
  2. You have a TestProject agent installed, watch how to install a TestProject Agent here.
  3. You have installed the REST API Addon, you can find it in the TestProject addons library here.

 

Assembling Our End-to-End Automated Test

We will use NASA’s APIs as an example, not just because we LOVE science but since it is also publicly available. The Near-Earth Object Web Service which is a RESTful web service for near earth Asteroid information, allows users to search for Asteroids based on their closest approach date to Earth.  

Our end-to-end test will contain 6 simple steps:

  1. Step 1 – Open your TestProject account and create a new recorded test.
  2. Step 2 – Retrieve all of the available asteroids by using GET API command and extracting the first asteroid ID (https://api.nasa.gov/neo/rest/v1/neo/browse?api_key=DEMO_KEY).
  3. Step 3 – Create a dynamic endpoint URL according to the first asteroid ID retrieved, analyze the server’s response and extract the value of “$.nasa_jpl_url”, which represents the URL to a web page that provides information about the asteroid.
  4. Step 4 – Replace the URL with a valid structure (adding ‘s’ to the HTTP string) – this is not something that you should do in a real-life scenario. However, we did it to overcome the bug we found in the response schema.
  5. Step 5 – Navigate to the asteroid URL (the correct one we fixed). Then, validate that the asteroid ID label that is presented in the web page matches the ID that was extracted in step 2.
  6. Step 6 – Analyze reports of our recorded test.

Step 1 – Create a New Test

Open your TestProject account and create a new test.
In our example, we will choose a web test (but of course you can also create a mobile test in the same manner and even a coded test):

TestProject_E2E_1

 

 

 

 

 

 

 

 

 

 

 

Then, you will need to provide test details, such as the test name and its description and supply the tested URL.

Since our test will mostly rely on REST API commands, which contains the URL as part of the steps, the initial application URL doesn’t play a major role here, yet should be supplied as part of the Web test default parameters.

TestProject_E2E_2

 

 

 

 

 

 

 

 

 

 

Next, select the “Record” option:

TestProject_E2E_3

 

 

 

 

 

 

 

 

 

 

A new browser tab will open within TestProject’s test recorder.

 

Step 2 – Extract Values by Using GET API command

In our initial step, we will retrieve all of the available asteroids by using GET API command and extracting the first asteroid ID (https://api.nasa.gov/neo/rest/v1/neo/browse?api_key=DEMO_KEY).

In order to do so, let’s create a new step (action type step) within our recorded test. In the step editor, select the “Add step” button (+), then press on “Type” and switch to “Action”:

TestProject_E2E_4

 

 

 

 

 

 

 

 

 

 

 

 

 

 

TestProject_E2E_5

 

 

 

 

 

Now, let’s go to “Actions” and search for GET. Then, you will see the HTTP GET Request action from the addons library:

TestProject_E2E_6

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Select the action. Now, a step will be added with the following available parameters:

TestProject_E2E_7

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Let’s add the relevant paraments needed to get a reply: 

URL  
https://api.nasa.gov/neo/rest/v1/neo/browse

Query
api_key=DEMO_KEY (you can add multiple parameters here separated by ‘&’).

ExpectedStatus
We will enter the code “200”. According to NASA’s documentation, the expected server status in case the request is valid will be 200.

JsonPath
$.near_earth_objects[0].neo_reference_id (based on Jayway JsonPath syntax, we are looking for a first neo_refernce_id, that will be passed as part of the endpoint for our next step).

We will also add a parameter (server_response) for the Response field, in order to store the retrieved first neo_reference_id.
This is how the step is supposed to look like in our step editor:

TestProject_E2E_8

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Let’s click on “Create”. Now, a step will be created and executed successfully. This successful execution will retrieve the response message (displayed below). Note that the specific value that we searched for, by using JayWay syntax, will be stored in the server_response parameter we created (as seen in the image below, it equals 2021277):

TestProject_E2E_9

 

 

 

 

 

 

 

 

 

 

 

Step 3 – Create a Dynamic End Point URL: Extract Information & Analyze Server’s Response

Now, let’s create a dynamic endpoint URL according to the asteroid ID that was retrieved above, and extract from the server response the nasa_jpl_url which represents the URL to a web page that provides full information about the asteroid.

According to NASA’s API documentation, the asteroid ID (aka neo_reference_id) that was retrieved above is part of the endpoint that provides us detailed asteroid information as shown in the following example:  https://api.nasa.gov/neo/rest/v1/neo/2021277?api_key=DEMO_KEY.

In this step, we will create a GET request by using the neo_reference_id value that was stored in the server_response parameter, to assemble the endpoint URL and retrieve a new value called nasa_jpl_url from the response message.

Let’s add a new step (action type step) within our recorded test. Then, search for the action GET. Then, you will see the HTTP GET Request action. Select the action and add the relevant parameters below:

URL 
https://api.nasa.gov/neo/rest/v1/neo/server_response

Query
api_key=DEMO_KEY (you can add multiple parameters here separated by ‘&’).

ExpectedStatus
We will enter the code “200”. According to NASA’s documentation, the expected server status in case the request is valid will be 200.

JsonPath  
$.nasa_jpl_url (based on Jayway JsonPath syntax,  we are looking for a nasa_jpl_url value).

We will also add a parameter (Asteroid_URL) for the Response field, in order to store the retrieved value (nasa_jpl_url).
This is how the step is supposed to look like in our step editor:

TestProject_E2E_10

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Now, let’s run our test steps and view the result.

As seen in the response schema displayed below, we received the specific asteroid id details. Both GET requests (from step 2 and from step 3) are related: the second call (in this step – step 3) is going to a specific endpoint based on the first REST API call (from step 2).

TestProject_E2E_11

 

 

 

 

 

 

 

 

 

 

 

This is the URL that was retrieved from value nasa_jpl_url and stored in Asteroid_URL parameter: http://ssd.jpl.nasa.gov/sbdb.cgi?sstr=2021277

If you will test it in your browser, you will see that the URL link leads to an incorrect web page. The correct URL NASA has been using is based on secured HTTPS (and not based on HTTP that was retrieved in the value nasa_jpl_url). Therefore, let’s add a step to our recorded test that replaces the HTTP with HTTPS and creates a correct URL address.

*It’s important to mention that this is not something that you should have to do in a real-life scenario, but rather instead, you should simply open a bug for your development team. In our case, we already sent an email to the folks at NASA and we are sure they will fix it really soon.

 

Step 4 – Replace the URL with a valid structure

As a workaround for the issue we found in the response schema, let’s replace the URL with a valid structure (adding ‘s’ to the HTTP string). In order to do this, let’s add a new step (action type step) within our recorded test. Then, search for an action called “Replace in string”, select it and add the parameters below:  

SearchPattern: ‘http’.  

Original: string Asteroid_URL parameter.

Literal: ‘https’. 

Result: New parameter called Fixed_Asteroid_URL.

This is how the step is supposed to look like in our step editor:

TestProject_E2E_12

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Now, let’s run this step as part of our test to create a correct URL.

Step 5 – The Power of End-to-End Tests: Navigate to a New URL & Validate ID Labels

In order to navigate to the new URL, let’s add another step (also an action type step) within our recorded test. Then, let’s search for the “Navigate to” action and select it. In the URL parameter select the: Fixed_Asteroid_URL (the URL we retrieved and fixed of step 4 above). Once the step is created it will navigate to the correct asteroid URL:

TestProject_E2E_13

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

To complete this end-to-end functional test scenario, we will create a new step (element action step) in order to validate that the asteroid ID label that is presented in the web page (of the fixed URL) matches the ID that was extracted in the first API call from step 2 above. In this step, we are using TestProject‘s built-in Element Locator to identify the SPK_ID label that appears on the web page, and activating the “Get text” element action step, as it is presented below:

TestProject_E2E_14

 

In the “Get text” step details, we will add a validation by clicking the (+) button.

  • In the “Select Field”, choose: “text”.
  • In the “Select validation type”, choose: “Equals”.
  • In the text box, we will select the server_response parameter which contains the original asteroid ID.

This action will compare the UI label ID with the ID that was retrieved from the API (in step 2 above).

TestProject_E2E_15

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

This is how our end-to-end test will look like now:

TestProject_E2E_16

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Step 6 – Analyze Reports

Now that we’ve created all of the necessary test steps, let’s execute our complete test from the “Project” area within the TestProject dashboard, and then we can analyze the test’s extensive reports (that are automatically created following the completion of the test execution). You can deep dive into each and every one of the test steps to view screenshots, step execution durations and monitor every success & failure of your end-to-end integration test, as seen below:

TestProject_E2E_17

 

Summary

This tutorial covered an entire workflow of end-to-end automated tests, all while using TestProject‘s framework. We started this tutorial by sending basic API calls, and ended it with validating and comparing UI based elements. Basically, we combined REST API and recorded UI steps within the same automated test, and thus achieved a powerful end-to-end test. And all of this happened without a single line of code, which is pretty cool! 

I think it’s important to mention here the fact that recorded tests sure are limited to a certain extent, and there are definitely some circumstances in which you must utilize your coding skills. Thus, I completely relate to those who aren’t so very fond of recorded tests. However, I wrote this tutorial to demonstrate just how easily one can leverage the capabilities of TestProject’s smart test recorder, take an entire testing cycle to a whole new level and create a powerful end-to-end automated test (in this case, while using TestProject’s test recorder that incorporates built-in capabilities as well as custom addons such as REST API). As we’ve seen in this tutorial, end-to-end tests are crucial for any product’s life-cycle and quality, and without it – we might even miss some bugs along the way.

 

What is your opinion on end-to-end tests? How do you scale your API testing? Share your thoughts and experience in the comments below!  💡 

Mark Kardashov

About the author

Mark Kardashov

Test automation enthusiast, entrepreneur, snowboard fan and CEO of TestProject. Has more than 10 years of experience in the testing and test automation field.

Join TestProject Community

Get full access to the world's first cloud-based, open source friendly testing community. Enjoy TestProject's end-to-end test automation Platform, Forum, Blog and Docs - All for FREE.

Join Us Now  

Comments

10 10 comments
  • Avatar
    ngoyal January 4, 2020, 9:51 am

    thanks for the explanation, couple of questions.
    1. Providing the project default URL while creating a api project does not make any sense. Unnecessary it navigates to a url which is of no use for us while running the test case. Thoughts?
    2. How can we save multiple parameters from service response body to be used in the next subsequent api request?

  • Amit Yahav
    Amit Yahav January 5, 2020, 11:32 am

    Hi ngoyal,

    Thanks for leaving a comment! I’ll address your queries here:
    1. You are right, if you are creating an API test case, then navigating to any URL has no meaning. It’s also doesn’t really affect your test, so you can just navigate to any URL. You can’t create a web recording session without specifying the URL. In coded tests, however, you can create a generic test that will not require you for any URL. To create a coded test, you can use our Java or C# SDKs as described here: https://docs.testproject.io/testproject-sdk/overview

    2. You can save the response body inside an input parameter (let’s call it serverResponse). This response is a json string that you can later apply the “Get value from JSON using JsonPath” action on it from the “JSON Operations” addon. Just like the name suggests, you can search for any value in this json string using JsonPath syntax: https://github.com/json-path/JsonPath

  • Avatar
    Martin Jahr February 1, 2020, 2:16 pm

    Thanks for the walkthrough. It needed a moment to recognize that I have to press “+” to add the server_response to the URL, not just typing it.

    I am also not a big fan of the “dummy url” up front. I have a lot of tests that compare different backends, not frontends.

    My question is an extension of the first comment. Suppose the rest interface gives me an array of data records. I would extract the field of interest from each records per jsonPath, and store the array of fields in a result variable.

    Then I would like to instantiate a new test with that array as input parameters. Is that possible?

  • Avatar
    Mark Kardashov February 1, 2020, 5:55 pm

    Hi Martin, you can do in a several ways:

    1.You can add a step with + icon that “call a test” (you should select “Test” in the action type) assuming the test you are calling have input variables you will be able to pass one or many parameters extracted to the inner test. This blog post section will explains it https://blog.testproject.io/2019/09/19/build-data-driven-test-automation-for-web-apps/#nested-tests . You can add as multiple nested tests as part of one main test passing variables (in or out) according to your need
    2. Another way to achieve what you want will be storing the variables in projects parameters (you can create project parameter on the same manner as you do for test parameters just select “Project” radio button) once you do it the variables inside them will be accessible to all tests in your project, so you use them in any other test or update them constantly, (they act as a global variables)

    hope it makes sense 😉

  • Avatar
    Sifat Morshed August 21, 2020, 12:29 am

    Is there any way i can call my App API (like i do in postman) and then check the App UI in the same testcase? I want to send a firebase push notification by postman and then wanna click on that notification from App side..how to do that in testproject

    • David Goichman
      David Goichman August 21, 2020, 8:35 am

      Hello Sifat, yes, that is possible.

      You can perform the API call using any of the actions in the RESTful API client addon from TestProject directly or via postman separately.
      https://addons.testproject.io/restful-api-client

      After the push notification appears in your application, you can perform any click or tap action on that notification, just be sure to increase the adaptive wait time to around ~20 seconds to account for the time it takes for the notification to appear.

    • David Goichman
      David Goichman August 21, 2020, 8:52 am

      To add to my previous comment, both of these actions can be part of the same mobile/web test.
      The first step would be performing the API call, and the second would be the click, you could also perform another validation step in between those two.
      These steps will be performed sequentially when you execute the test.

      • Avatar
        Sifat Morshed September 4, 2020, 10:18 am

        Thanks David, it worked. Here i need to change the jason body a bit. For make it working from App one needs to omit all the spaces and new line from the json body. Also I had to delete the extra spaces from Header Authorization. And format should be text/json..

  • Avatar
    Aleksander Koprivnjak September 14, 2020, 1:47 pm

    Hi,

    I am trying out TestProjects as a solution for my API Testing but I get really weird results.
    While trying to send a request to my internal server I get following error:

    Message
    java.util.concurrent.ExecutionException: io.testproject.java.sdk.v2.exceptions.FailureException: Failed to send the request due to an error
    Caused by io.testproject.java.sdk.v2.exceptions.FailureException: Failed to send the request due to an error

    Any ideas?

    • Ran Tzur
      Ran Tzur October 11, 2020, 6:24 pm

      Hello, the support to query IPs without hostnames have been added to the latest version of the Rest API addon.
      This will solve issues while query IPs without hostnames, like internal servers.

Leave a Reply

popup image

Complete E2E Automation Solution!

Join over 60K users (from Wix, IBM, Payoneer and many more!) using the world's first FREE cloud-based test automation platform, supported by the #1 testing community.
Sign Up Now right arrow
FacebookLinkedInTwitterEmail