Automate Testing Your API with Postman, Newman and Jenkins (Part 2)

This is Part 2 of a four part series on how to automate testing with your APIs. The introductory blog can be found here, the third blog about adding Postman tests can be found here, and the fourth summary blog can be found here.

What is Collection Runner?

In short, Collection Runner in Postman will run your entire collection of endpoints and all of their scripted tests in sequential order.

You can read in detail at https://www.getpostman.com/docs/v6/postman/collection_runs/starting_a_collection_run. You can launch the Collection Runner with the "Runner" button.

This will launch the interface where you select your collection, the environment file to use, and other configurations.

When you run the collection, you will see in real time, the sequence of execution and the results of each test within each endpoint.

To troubleshoot or examine the details of a request or a response, you can click on the urls and drill into the request or response of each test.

The user interface is great but for us to integrate this with Jenkins, we need a way to run this via command line.

What is Newman?

Newman is a Node Package Manager (NPM) package that allows us to run our collections directly from the command line. You'll need NodeJS and NPM already installed. A copy of the NodeJS installable can be downloaded from here.

I am on a Windows 7 machine so I will use Window's Powershell. Here I use the node -v and npm -v command to verify I have Node and NPM installed.

Once you confirm you have Node and NPM installed, install Newman with the following NPM command: $npm install -g newman

Once you have Newman installed, go ahead and export your Collection and the Environment to json files on your local file system. We will be passing these files as arguments to Newman. Click on the Download or Export button in Postman.

Download your Collection.

Download your Environment.

You should have something like this saved.

Now open a command prompt and elevate your privileges so that you can execute the script. Run the following command:

Set-ExecutionPolicy -Scope CurrentUser

Then when it asks you for the Execution Policy, type Unrestricted and hit "enter" on the keyboard. Then Type "Y" to confirm this change and hit "enter" on the keyboard.

Now open a command prompt and run the following command:

newman run [collection_filename.son] -e [environment_filename.json]

-e is the flag for Environment. This should run through your entire Collection of endpoints and their integration tests via the command-line.

You can see the results here. Notice that we have the same failure we saw with Collection Runner?

Summary

What did we cover? We went over what Collection Runner was and how we can use the GUI to execute our entire collection of endpoints. We talked about needing to install NodeJS, NPM, and Newman. We exported/downloaded our Collection and Environment to json files. Finally, we used Newman to run our Collection with our Environment file and saw the same results as when we used Collection Runner.

Next, we will write a script so that we can use this on our Jenkins build server to run Newman.

Continue onto Part 3!