Running TestProject Agent in Docker Container

In the previous chapter, we introduced TestProject and TestProject Agent in detail. Now, let’s look at how we can run the TestProject Agent in Docker container 🙌🐳

Introduction to Docker & Docker Desktop
Introduction to TestProject and TestProject Agent in Docker
You’re here → Running TestProject Agent inside Docker Container
- Why Do I Need TestProject Agents Running Inside Docker Containers?
- Setting up TestProject Agent in Docker
- TestProject in Docker Reports and Dashboards
- Conclusion
  - [Webinar Recording] Accelerate Your Test Automation using TestProject & Docker

Why Do I Need TestProject Agents Running Inside Docker Containers?

Containers are a perfect companion to the agile movement. Organizations today need fast paced & efficient software teams, and with that comes the hefty practice of continuous pushing of source code from development to testing & production multiple times. With Docker’s ability to isolate environments with specific dependencies & the flexibility to ship them from or to different environments makes the work more efficient & reliable.

Docker containers can accelerate your test automation platform using “dockerized” TestProject Agents. What makes the docker agent useful is:

It saves a bunch of resources & provides the ability to run tests with just a simple command.
You can execute tests in parallel on multiple agents at the same time (this can be very useful if you wish to install multiple TestProject Agents on the same server/desktop).
You can integrate it easily within your CI/CD flow, without pre-registering the TestProject Agent via the TestProject UI since the registration occurs automatically.

For most cases you’ll need to set up the TP_API_KEY from the web console which is a way of authorizing the docker agent to an account. Optionally, you’ll need a Job id TP_JOB_ID if you want the containers to execute directly instead of waiting for instructions from the web UI.

You can set up the agent in two ways:

Giving signals to execute jobs right from the web UI.
Or providing dedicated tokens beforehand for seamless execution of jobs without the need for manual intervention.

Setting up TestProject Agents in Docker

As seen in the previous chapter, all you need is a TestProject account, that’s it! If you don’t already have one, you can sign up and open your free account here.

The TestProject Agent docker container can be used in two distinct ways, as also described in their DockerHub page:

Permanent Execution Engine – The agent is registered once, and then can be used to execute tests and jobs from the TestProject web application
Ephemeral Instances – In this scenario the agent is started up in order to perform a specific task, and will self terminate upon completion of said task.

The operation mode of the agent is controlled by environment variables that are passed to the container upon creation.

Permanent Execution Engine

Let’s say you want to run multiple jobs & don’t want a fresh container for each test run. Your test capitalizes on a cache previously saved from a test run to continue further execution. Running tests from the start is a bit of an overkill even when using containers.

This is where instances that have saved states come into the picture.

The flexibility to resume or carry on jobs with the ability to save data can become crucial for any testing environment as the testing phase matures.

The Docker agent of TestProject comes with this feature built in. You can easily start images that run for indefinite time & you can save the data with the help of attached volumes in case a running container fails. To get started aforementioned we need the TP_API_KEY to grant access to the project & an TP_AGENT_ALIAS to set a custom name for the agent.

When starting an instance with these variables, the agent is automatically registered to the account. This name can be referred to if you want to view the agent in your account.

In order to run tests in a container we need an API key for authorization & a JOB ID to perform the specific tests. We can generate API keys through the console in the Integrations tab from the top bar.

Click on Integration > API > Create API key as shown below:

Click Create & enter a name. Clicking Next gives us the option to either allow access to all or particular projects. For now we’ll go for unrestricted.

Click “Finish”. Then you will have created an API Key:

Copy the key, because we’ll need it when we’re creating the containers.

Next, it’s time to open a terminal and run the below command:

docker run --name testproject-agent \
-e TP_API_KEY="REPLACE_WITH_YOUR_KEY" \
-e TP_AGENT_ALIAS="My First Agent" \
-v </path/to/host/folder>:/var/testproject/agent \
testproject/agent:latest

In the above CLI, one needs to pass the correct API_KEY. A volume can also be set up to store the images data for the persistence. Once you execute the above command, you will see the below output which depicts successful initialization of the TestProject agent.

By now, you should be able to get a TestProject agent registered. The 172.17.0.2 is the IP address of Docker container running the agent.

Go to Docker Desktop UI > Dashboard > Select testproject-agent

TestProject Agent runs an Alpine based Docker container which you can verify by clicking on the “>_” option as shown above in the UI.

Running Test Jobs using Docker & Setting up Jenkins

One important aspect of test automation is being able to run the tests on a regular cadence. Often teams are interested in how the product quality looks as the code changes and so tests need to be run on some kind of schedule. TestProject provides you with the tools that you need to schedule and run tests by adding a job to your project.

To create a job, navigate to the project you want to add a job for and on the right-hand panel click on the Add a job button.

Click on “Add a Job”. This will bring up the job creation wizard. Enter in a name and description for the job and click next. On the next page you can choose where you want the job to run. You can either run a mobile or web test and you can pick which agent you want to use to run the job. Make sure that you are selecting compatible agents for the kind of testing that you want to do. For example, if you want to run the job as a mobile android test, be sure to pick an agent that will have an android device attached.

For this example, I selected “Web”. Since we’re running on containers, we’ll select headless browser:

Add the test you created earlier to the Job & also copy its ID. You can find the Job ID by clicking on the dots as shown in the below window:

In its current state the Agent is idle & is waiting for a job to run. Let’s bring the container into action by sending it the job signal using TestProject’s Jenkins plugin. The plugin is available for installation from: Manage Jenkins > Manage Plugins.

Using the API_KEY, you can link the plugin to your TestProject account seamlessly. All you need to do is to navigate to Manage Jenkins > Configure System & scroll down to TestProject Global Configuration, enter the API_KEY & you’re good to go.

Create a Freestyle project under the build section add a build step with Run TestProject Job selected:

Ephmeral Instances

In order to have test environments that resemble production environments we can either configure it at the source code level, or have tests run on actual servers with the dependencies like db etc. installed.

The story at the source code level brings unwanted configurations to the code making it hard to maintain with newer updates & bug fixes. However running it on actual hardware is also a burnout as stuff can get overwhelmed pretty quick, & comes with a lot of setup and maintenance overheads. In addition, since resources are shared among tests we can only run them one at a time to ensure they don’t interfere with each other’s executions.

Let’s have a look at Docker Compose which enables us to get the best of both worlds. It makes it easier to replicate the parts we have in environments & create them in the form of containerized solutions. We get the freedom to test in a real physical environment.

The TestProject Agent is responsible for handling the test cases with the given api_key & job_id. The Docker compose file also sets up multiple browsers to run the tests on. After running the test it exits so that any cache left is cleared. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

Create a File with the .yaml extension with the following configuration:

version: "3.1"
services:
  testproject-agent:
    image: testproject/agent:latest
    container_name: testproject-agent3
    depends_on:
      - chrome
      - firefox
    environment:
      TP_API_KEY: <your_api_key>
      TP_AGENT_ALIAS: "Docker Agent"
      TP_JOB_ID: <job_id>
      TP_JOB_PARAMS: '"jobParameters" : { "browsers": [ "chrome", "firefox" ] }'
      CHROME: "chrome:4444"
      FIREFOX: "firefox:4444"
  chrome:
    image: selenium/standalone-chrome
    volumes:
      - /dev/shm:/dev/shm
  firefox:
    image: selenium/standalone-firefox
    volumes:
      - /dev/shm:/dev/shm

You can download YAML file directly from here.

This docker-compose snippet spins up 3 containers:

TestProject Agent
Standalone Selenium with pre-installed Chrome
Standalone Selenium with pre-installed Firefox

The TestProject Agent will automatically detect the Chrome & Firefox containers. By setting the TP_API_KEY & TP_JOB_ID environment variables we instruct the Agent to automatically start executing the requested job on startup. By setting the TP_JOB_PARAMS environment variable we override the execution targets for this job to include both Chrome & Firefox browsers.

In a nutshell, the above configuration starts a container (testproject/agent) that is connected to your TestProject account & with the help of the browser containers runs the jobs.

[Captains-Bay]🚩 >  docker-compose ps
       Name                     Command               State     Ports
-----------------------------------------------------------------------
testproj_chrome_1    /opt/bin/entry_point.sh          Up       4444/tcp
testproj_firefox_1   /opt/bin/entry_point.sh          Up       4444/tcp
testproject-agent3   /bin/sh /opt/testproject/a ...   Up
[Captains-Bay]🚩 >

Now, let’s set up a Docker compose file to run from a Jenkins pipeline. With the Docker TestProject Jenkins plugin, we can run tests directly from the pipeline enabling a seamless CI set up. Here’s a sample Jenkins file that pulls & runs a node.js application, initiates the test on the containers via the plugin & cleans up in the end.

pipeline {
   agent any

   stages {
      stage('Build') {
         steps {
            // Get some code from a GitHub repository
            git 'https://github.com/collabnix/testapp.git'
           
         }

      }
      stage('Run') {
   steps {

      sh "node index.js &"

   }
      }
      stage('Test') {
   steps {
      tpJobRun agentId: 'ph3CXXXXXXXX', jobId: 'o7wS4nXXXXXXX', projectId: 'DzkGyzxkMXXXXXXXX', waitJobFinishSeconds: 180
      
      sh "killall -9 node"

   }
      }
   }
}

TestProject in Docker Reports and Dashboards

Once the job execution is completed, you can click on the “Reports” section in TestProject to see in-depth details of the execution steps, including screenshots, and whether the job successfully passed or failed. You can also download full reports in PDF format to send over to your managers and teammates and easily debug any test failure.

Conclusion

In this 3 chapter Docker tutorial using TestProject, we dived deep into what Docker is, how to setup a Docker environment, where TestProject comes into the picture, and eventually saw that Docker makes it quite simple to start a TestProject Agent with headless Chrome & Firefox browsers using a simple Docker compose file.

On May 17th 2020, I also hosted an online live webinar with over 300 attendees discussing how to accelerate your automation testing using TestProject & Docker:

Happy Testing! 😎🐳