Puppeteer: Web Automation using Headless Chrome Node.js API

Recently, there has been a lot of talk around Puppeteer, and the wide range of features it can offer with very lightweight code! It is a node.js API library that enables control over headless Chrome you can use for web automation. In this beginner’s guide, we will dive into what is Puppeteer, install required dependencies and demonstrate some of its capabilities via code examples. 🎓

Table of Content

What is Puppeteer?
- puppeteer vs. puppeteer-core
Getting Started with Puppeteer
- Installation
- Usage
  - Example Code #1 – Take Screenshot and Save Image
  - Example Code #2 – Save Text to PDF File
  - Example Code #3 – UI Testing of TestProject’s Contact Form
    - [Video] Automated Contact Form Submission using Puppeteer with Chrome & Javascript

What is Puppeteer?

Puppeteer is a Node library that provides a high-level API to control Chrome or Chromium over the DevTools Protocol. It runs headless by default but can be configured to run full (non-headless) Chrome or Chromium.

Puppeteer has been gaining popularity since its inception, due to its growing community and support for headless chrome that is very popular with web scraping and UI automation. You can automate tasks on web pages such as form submission, scrape data from websites, test user interfaces and compatibility with Chrome versions, take screenshots and export web pages as PDF, and more advanced features we will discuss in future articles.

Puppeteer vs puppeteer-core

We have two packages maintained on Github for Puppeteer, namely puppeteer and puppeteer-core. Here is the difference between the two from the official documentation:

“puppeteer is a product for browser automation. When installed, it downloads a version of Chromium, which it then drives using puppeteer-core. Being an end-user product, puppeteer supports a bunch of convenient PUPPETEER_* env variables to tweak its behavior.

puppeteer-core is a library to help drive anything that supports DevTools protocol. puppeteer-core doesn’t download Chromium when installed. Being a library, puppeteer-core is fully driven through its programmatic interface and disregards all the PUPPETEER_* env variables.”

Getting Started

Installation

First of all, we need to install all the required dependencies for puppeteer that include, node package manager (npm) and if it’s not already installed in your machine – Then go ahead and follow these steps here to setup npm.

Now, you can create a directory where we are going to work with our code and install puppeteer. Use the following commands using command prompt:

mkdir testproject

cd testproject

npm i puppeteer

This will take some time as Chrome headless will be downloaded and progress will be shown in the command prompt.

Usage

Now we will learn how to use puppeteer with some code examples!

Example Code #1 – Take Screenshot and Save Image

Let’s start with the first example where we will be navigating to https://blog.testproject.io/, take a screenshot of the homepage and save it as example.png in the same directory.

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://blog.testproject.io/');
  await page.screenshot({path: 'example.png'});

  await browser.close();
})();

Now save this code as example.js and use the below command to execute the code (then you will see that a screenshot is generated as shown below):

node example.js

Screenshot from Code Example #1

Example Code #2 – Save Text to PDF File

Let’s try another example where we will save the whole text on the homepage to a pdf file named example.pdf.

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://blog.testproject.io/');
  await page.screenshot({path: 'example.png'});

  await browser.close();
})();

Now save this code as example.js and use the below command to execute the code (then you will see that a PDF is generated as shown below in screenshot):

node example.js

Screenshot from Code Example #2

Example Code #3 – UI Testing of TestProject’s Contact Submission

This is an example of UI testing using puppeteer where we will be testing the contact form on the TestProject Site.

Let’s start with writing code for this test. The main events for testing this contact form will be an empty form, filled form and result after submitting the form. We will be taking screenshots at each event completion so that we can verify the UI after execution.

Code

Start with creating a project directory and start a command prompt within the same directory. We can use the same commands as we used above to set up a puppeteer project, and then let’s proceed to the UI test code.

In the first line of code, we are just importing the puppeteer library onto our test script:

const puppeteer = require('puppeteer');

Now, after importing the library, we will define some options so that we can see puppeteer executing our script. You have to change the string in executablePath to the chrome.exe location on your laptop:

const chromeOptions =

{   executablePath:"PATH TO YOUR chrome.exe",
    slowMo: 10,
    headless:false
 };

Now we can move onto launching the chrome browser and writing the UI test which can automatically fill the form and take screenshots on each event trigger. Here is the complete code of our test script:

Complete Code

const puppeteer = require('puppeteer');
//Defining chromeOptions for defining path, time to execute and setting headless mode to false
const chromeOptions = {
    executablePath:"C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
    slowMo: 10,
 headless:false,
 args: ['--start-maximized']
 };
(async () => {
  
  const browser = await puppeteer.launch(chromeOptions);
  
  const page = await browser.newPage();
  
  //Navigating to Test Project Contact Form
  await page.goto('https://testproject.io/contact-us/');
  
  //Taking screenshot of initial page UI
  await page.screenshot({path: 'beforefill.png'});
  
  //Waiting for form to load
  await page.waitFor('.wpcf7-form');
  
  //Waiting for field First Name to appear and storing its UI element in name variable and entering test value
  const fname = await page.waitForSelector("#firstName");
  await fname.type("Testing UI Automation");
  
  //Waiting for field Last Name to appear and storing its UI element in name variable and entering test value
  const lname = await page.waitForSelector("#lastName");
  await lname.type("Test last Name");
  
  //Waiting for field Email to appear and storing its UI element in name variable and entering test value
  const email = await page.waitForSelector("#companyEmail");
  await email.type("[email protected]");
  
  //Waiting for dropdown element and clicking on it
  const dropdown = "#select2-subject-container";
  await page.waitForSelector(dropdown);
  await page.click(dropdown);
  
  //Enter on subject
  await page.keyboard.press('Enter');  
  
  
  //Entering Message in the message field
  const message = await page.waitForSelector("#message");
  await message.type("This is a test message created by automated testing of UI using Puppeteer");
  
  //Clicking on Contact us
  await page.keyboard.press('Tab');
  await page.keyboard.press('Enter');  
  
  //Take screenshot of filled form UI
  await page.screenshot({path: 'afterfill.png'});
  
  //Wait for form to be submitted
  await page.waitForSelector(".thank-you__title");
  
  //Take screenshot of after submit form UI
  await page.screenshot({path: 'aftersubmit.png'});
  
  await browser.close();
})();

This is the complete code where we are verifying (both by using screenshots and through the code) that our “contact us” functionality is working as expected, and the user lands on the Thank you page after filling the valid details.

Here you can see a complete video of the whole automated script using Puppeteer:

Feel free to ask any questions in case you got stuck anywhere while playing around with Puppeteer. Share your feedback in the comments below! 😉