How Can You Scale Your REST API Testing?

REST API is a powerful way to run tests, however it’s not always enough. There are some pitfalls that cannot be spotted by REST API testing alone. And here comes the importance of end-to-end test automation.

For example, during the process of writing this article, I tested NASA’s public API and discovered a bug in one of their API schemas. I could not have discovered this bug unless I would have gone through the entire cycle of end-to-end testing. This clearly emphasizes the vast importance that end-to-end testing has in a product’s life-cycle, and we should not underestimate the power of combining API and UI automated testing for better coverage and quality.

TestProject‘s end-to-end test automation framework just released a new open source addon for RESTful API clients. It provides actions to send HTTP/S requests using GET, POST, PUT and DELETE methods and is completely open source. You can see the documentation and source code for it here.

In this article, we will discuss how to create automated REST API tests with TestProject and we will:

  1. Send API commands to the server and validate responses.
  2. Use values from responses as parameters in test steps.
  3. Combine REST API and recorded UI steps within the same automated test to achieve end-to-end testing.
  4. Analyze reports.

To start working you will need to make sure that:

  1. You have a TestProject account, you can get it for FREE here.
  2. You have a TestProject agent installed, watch how to install a TestProject Agent here.
  3. You have installed the REST API Addon, you can find it in the TestProject addons library here.

 

Assembling Our End-to-End Automated Test

We will use NASA’s APIs as an example, not just because we LOVE science but since it is also publicly available 😊. The Near-Earth Object Web Service which is a RESTful web service for near earth Asteroid information, allows users to search for Asteroids based on their closest approach date to Earth.  

Our end-to-end test will contain 6 simple steps:

  1. Step 1 – Open your TestProject account and create a new recorded test.
  2. Step 2 – Retrieve all of the available asteroids by using GET API command and extracting the first asteroid ID (https://api.nasa.gov/neo/rest/v1/neo/browse?api_key=DEMO_KEY).
  3. Step 3 – Create a dynamic endpoint URL according to the first asteroid ID retrieved, analyze the server’s response and extract the value of “$.nasa_jpl_url”, which represents the URL to a web page that provides information about the asteroid.
  4. Step 4 – Replace the URL with a valid structure (adding ‘s’ to the HTTP string) – this is not something that you should do in a real-life scenario. However, we did it to overcome the bug we found in the response schema.
  5. Step 5 – Navigate to the asteroid URL (the correct one we fixed). Then, validate that the asteroid ID label that is presented in the web page matches the ID that was extracted in step 2.
  6. Step 6 – Analyze reports of our recorded test.

Step 1 – Create a New Test

Open your TestProject account and create a new test.
In our example, we will choose a web test (but of course you can also create a mobile test in the same manner and even a coded test):

TestProject_E2E_1

 

 

 

 

 

 

 

 

 

 

 

Then, you will need to provide test details, such as the test name and its description and supply the tested URL (https://api.nasa.gov/api.html#neows-feed).

Since our test will mostly rely on REST API commands, which contains the URL as part of the steps, the initial application URL doesn’t play a major role here, yet should be supplied as part of the Web test default parameters.

TestProject_E2E_2

 

 

 

 

 

 

 

 

 

 

Next, select the “Record” option:

TestProject_E2E_3

 

 

 

 

 

 

 

 

 

 

A new browser tab will open within TestProject’s test recorder.

 

Step 2 – Extract Values by Using GET API command

In our initial step, we will retrieve all of the available asteroids by using GET API command and extracting the first asteroid ID (https://api.nasa.gov/neo/rest/v1/neo/browse?api_key=DEMO_KEY).

In order to do so, let’s create a new step (action type step) within our recorded test. In the step editor, select the “Add step” button (+), then press on “Type” and switch to “Action”:

TestProject_E2E_4

 

 

 

 

 

 

 

 

 

 

 

 

 

 

TestProject_E2E_5

 

 

 

 

 

Now, let’s go to “Actions” and search for GET. Then, you will see the HTTP GET Request action from the addons library:

TestProject_E2E_6

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Select the action. Now, a step will be added with the following available parameters:

TestProject_E2E_7

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Let’s add the relevant paraments needed to get a reply: 

URL  
https://api.nasa.gov/neo/rest/v1/neo/browse

Query
api_key=DEMO_KEY (you can add multiple parameters here separated by ‘&’).

ExpectedStatus
We will enter the code “200”. According to NASA’s documentation, the expected server status in case the request is valid will be 200.

JsonPath
$.near_earth_objects[0].neo_reference_id (based on Jayway JsonPath syntax, we are looking for a first neo_refernce_id, that will be passed as part of the endpoint for our next step).

We will also add a parameter (server_response) for the Response field, in order to store the retrieved first neo_reference_id.
This is how the step is supposed to look like in our step editor:

TestProject_E2E_8

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Let’s click on “Create”. Now, a step will be created and executed successfully. This successful execution will retrieve the response message (displayed below). Note that the specific value that we searched for, by using JayWay syntax, will be stored in the server_response parameter we created (as seen in the image below, it equals 2021277):

TestProject_E2E_9

 

 

 

 

 

 

 

 

 

 

 

Step 3 – Create a Dynamic End Point URL: Extract Information & Analyze Server’s Response

Now, let’s create a dynamic endpoint URL according to the asteroid ID that was retrieved above, and extract from the server response the nasa_jpl_url which represents the URL to a web page that provides full information about the asteroid.

According to NASA’s API documentation, the asteroid ID (aka neo_reference_id) that was retrieved above is part of the endpoint that provides us detailed asteroid information as shown in the following example:  https://api.nasa.gov/neo/rest/v1/neo/2021277?api_key=DEMO_KEY.

In this step, we will create a GET request by using the neo_reference_id value that was stored in the server_response parameter, to assemble the endpoint URL and retrieve a new value called nasa_jpl_url from the response message.

Let’s add a new step (action type step) within our recorded test. Then, search for the action GET. Then, you will see the HTTP GET Request action. Select the action and add the relevant parameters below:

URL 
https://api.nasa.gov/neo/rest/v1/neo/server_response

Query
api_key=DEMO_KEY (you can add multiple parameters here separated by ‘&’).

ExpectedStatus
We will enter the code “200”. According to NASA’s documentation, the expected server status in case the request is valid will be 200.

JsonPath  
$.nasa_jpl_url (based on Jayway JsonPath syntax,  we are looking for a nasa_jpl_url value).

We will also add a parameter (Asteroid_URL) for the Response field, in order to store the retrieved value (nasa_jpl_url).
This is how the step is supposed to look like in our step editor:

TestProject_E2E_10

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Now, let’s run our test steps and view the result.

As seen in the response schema displayed below, we received the specific asteroid id details. Both GET requests (from step 2 and from step 3) are related: the second call (in this step – step 3) is going to a specific endpoint based on the first REST API call (from step 2).

TestProject_E2E_11

 

 

 

 

 

 

 

 

 

 

 

This is the URL that was retrieved from value nasa_jpl_url and stored in Asteroid_URL parameter: http://ssd.jpl.nasa.gov/sbdb.cgi?sstr=2021277

If you will test it in your browser, you will see that the URL link leads to an incorrect web page. The correct URL NASA has been using is based on secured HTTPS (and not based on HTTP that was retrieved in the value nasa_jpl_url). Therefore, let’s add a step to our recorded test that replaces the HTTP with HTTPS and creates a correct URL address.

*It’s important to mention that this is not something that you should have to do in a real-life scenario, but rather instead, you should simply open a bug for your development team. In our case, we already sent an email to the folks at NASA and we are sure they will fix it really soon 😊.

 

Step 4 – Replace the URL with a valid structure

As a workaround for the issue we found in the response schema, let’s replace the URL with a valid structure (adding ‘s’ to the HTTP string). In order to do this, let’s add a new step (action type step) within our recorded test. Then, search for an action called “Replace in string”, select it and add the parameters below:  

SearchPattern: ‘http’.  

Original: string Asteroid_URL parameter.

Literal: ‘https’. 

Result: New parameter called Fixed_Asteroid_URL.

This is how the step is supposed to look like in our step editor:

TestProject_E2E_12

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Now, let’s run this step as part of our test to create a correct URL.

Step 5 – The Power of End-to-End Tests: Navigate to a New URL & Validate ID Labels

In order to navigate to the new URL, let’s add another step (also an action type step) within our recorded test. Then, let’s search for the “Navigate to” action and select it. In the URL parameter select the: Fixed_Asteroid_URL (the URL we retrieved and fixed of step 4 above). Once the step is created it will navigate to the correct asteroid URL:

TestProject_E2E_13

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

To complete this end-to-end functional test scenario, we will create a new step (element action step) in order to validate that the asteroid ID label that is presented in the web page (of the fixed URL) matches the ID that was extracted in the first API call from step 2 above. In this step, we are using TestProject‘s built-in Element Locator to identify the SPK_ID label that appears on the web page, and activating the “Get text” element action step, as it is presented below:

TestProject_E2E_14

 

In the “Get text” step details, we will add a validation by clicking the (+) button.

  • In the “Select Field”, choose: “text”.
  • In the “Select validation type”, choose: “Equals”.
  • In the text box, we will select the server_response parameter which contains the original asteroid ID.

This action will compare the UI label ID with the ID that was retrieved from the API (in step 2 above).

TestProject_E2E_15

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

This is how our end-to-end test will look like now:

TestProject_E2E_16

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Step 6 – Analyze Reports

Now that we’ve created all of the necessary test steps, let’s execute our complete test from the “Project” area within the TestProject dashboard, and then we can analyze the test’s extensive reports (that are automatically created following the completion of the test execution). You can deep dive into each and every one of the test steps to view screenshots, step execution durations and monitor every success & failure of your end-to-end integration test, as seen below:

TestProject_E2E_17

 

Summary

This tutorial covered an entire workflow of end-to-end automated tests, all while using TestProject‘s framework. We started this tutorial by sending basic API calls, and ended it with validating and comparing UI based elements. Basically, we combined REST API and recorded UI steps within the same automated test, and thus achieved a powerful end-to-end test. And all of this happened without a single line of code, which is pretty cool! 

I think it’s important to mention here the fact that recorded tests sure are limited to a certain extent, and there are definitely some circumstances in which you must utilize your coding skills. Thus, I completely relate to those who aren’t so very fond of recorded tests. However, I wrote this tutorial to demonstrate just how easily one can leverage the capabilities of TestProject’s smart test recorder, take an entire testing cycle to a whole new level and create a powerful end-to-end automated test (in this case, while using TestProject’s test recorder that incorporates built-in capabilities as well as custom addons such as REST API). As we’ve seen in this tutorial, end-to-end tests are crucial for any product’s life-cycle and quality, and without it – we might even miss some bugs along the way.

 

What is your opinion on end-to-end tests? How do you scale your API testing? Share your thoughts and experience in the comments below!  💡