Click here to Skip to main content
14,976,689 members
Articles / DevOps / TFS
Article
Posted 18 Oct 2018

Tagged as

Stats

3.4K views

Testing using Newman on TFS with Docker

Rate me:
Please Sign up or sign in to vote.
5.00/5 (1 vote)
18 Oct 2018CPOL2 min read
Continuous Testing using Newman (Postman) on TFS 2017 with Docker

Introduction

Recently, we wanted to add continuous testing to our existing CI/CD processes. We wanted to ensure that when code was checked in, it was automatically testing and only if passing would the build continue. This would also serve as proof that test cases had been executed against code. Since we use Postman for our micro-service test cases, we decided to integrate its command line version, Newman, into our CI/CD.

Little Background

  • We use the Docker BYO VM and configured it for Docker CLI API calls - how we did that can be found here: Developer Docker Configuration Article
  • We have a continuous integration (CI/CD) on TFS 2017 setup following these instructions - Integration on TFS using Docker Article
  • Instructions on Newman - https://github.com/postmanlabs/newman
  • Instructions on Node/NPM with Enterprise Artifact Repository (EAR)

Export Test Cases

In Postman, we exported the Test Collection as well as supporting environment file.

Image 1

These were then placed in a Test folder in our project folder, along with Certificate file (see CI/CD instructions) and npmrc file (see EAR instructions above):

Image 2

Test Dockerfile

The key to this working is a separate dockerfile used only for testing (and placed in the test folder). This dockerfile does 6 main things:

  1. Creates a separate test image that will NEVER be deployed to EAR
  2. Installs Node
  3. Configures Node to use EAR (copies npmrc file listed above)
  4. Installs Newman from EAR
  5. Installs Test files (json files listed above)
  6. Installs Dev Publish files

Image 3

Update Build Definition

The current build definition now needs to be updated to perform 3 new steps:

Image 4

  1. Build Test Image - References Test dockerfile and gives the image a unique name:
  2. Run Test Image - Executes test image, specifying a container name that we will reference in the next step. 
    Note: Be sure to check the Run in background option.

    Image 5

  3. Run Newman Test - Use Docker execute command against the test container to execute newman test.

    Image 6

Result

When code is checked in, the test will be executed against the test cases. If the test passes, the Newman step is successful, otherwise the Newman step is unsuccessful halting the build process.

Example of Successful Test

Image 7

Example of Test that Fails

Image 8

Note: Having a separate dockerfile that reinstalls the dev publish files is not optimal. Ideally, we should be using Multi-Stage builds to build the test image from the dev image. Or at the very least, reference the dev image during creation of the test images. However, these approaches require use of  multi Stage builds or ARGS in FROM Statement.

History

  • 18th October, 2018: Initial version

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Santosh Vijay Patil
Software Developer (Senior) Cognizant
India India
• Since last two years followed by post-graduation from Chanakya International Institute of Leadership Studies(CIILS), Mumbai University in the field of Leadership, applied those principles in my own organization with outstanding results.
• Leading the team of Salesforce developer at the Capgemini for one of the largest implementation on Force.com platform comprising around 5000 user-base.
• Outstanding written and oral communications skills with comprehensive experience in numerous forms of marketing.
• Enthusiastic to work with intelligent minds of today to remain updated about things which can help him develop new ideas and innovate the older processes to make the world more better place to live.
• Facilitates positive change by improving thinking and the rare ability to electrify an audience yet deliver uncommonly original and useful insights that lead to individuals doing their best work, teams providing superb results and organizations becoming better.

Comments and Discussions

 
-- There are no messages in this forum --