Click here to Skip to main content
15,122,003 members
Articles / DevOps / Testing
Technical Blog
Posted 6 Feb 2017

Tagged as

Stats

18.5K views
5 bookmarked

Unit Testing Using the Command Line

Rate me:
Please Sign up or sign in to vote.
5.00/5 (1 vote)
6 Feb 2017CPOL5 min read
In this post, you are going to learn to run unit tests from the command line.

This is another in my series of blog posts on unit testing. If you are not familiar with unit testing, go back and read these posts.

In this post, you are going to learn to run unit tests from the command line. This allows you to schedule tests using task manager or any other automated scheduling tool. The VSTest.Console.exe is the tool you use as a .NET developer to run your unit tests in your .NET test dll.

Simple VSTest.Console Run

The VSTest.Console.exe is typically installed into the following path: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow. When you run this tool, you need to prefix this path to the .exe, or add this path to the PATH environment variable. If you open a Developer Command Prompt for VS2015 (Figure 1) this path will already be included.

Figure 1: Use the Developer Command Prompt

If you have been creating the project as you have been following along with this series of blog posts, you have a DLL named MyClassesTest.dll located in the \bin\Debug folder of where your project is located on your disk. Open a Developer Command Prompt and navigate to that folder. Type in the following in your command window.

VSTest.Console.exe MyClassesTest.dll

Press the Enter key and you should see something that looks like Figure 2. You may or may not have the warnings depending on if you added the DeploymentItem attributes.

Figure 2: A simple run of the VSTest.Console

View Installed Loggers

Now that you know how to run your unit tests from the command line, you now need to learn to log the results to a file that you can look at later. If you are going to be running your unit tests overnight, you want to come back in the morning to see the results. If they are just sitting in a command window, you could accidentally close that window and you would lose the results. Instead, the VSTest.Console utility has a set of loggers that you can use. Type in the following in the command window.

VSTest.Console.exe /ListLoggers

Press the Enter key to see a screen that looks like Figure 3.

Figure 3: View the log options you can use with VSTest.Console

The ConsoleLogger is the one you are currently looking at in your command window. The TfsLogger is useful if you are using Team Foundation Server as it allows you to send the results to TFS so you can assign work items based on any failed unit tests. The last one is the one that will be useful if you do not have TFS. The TrxLogger creates a .trx file which you can load into Visual Studio and see a list of all of your unit tests. You can then click on each test and see the results, and the outputs for that test.

Using Logger

Let's take a look at using the TrxLogger option when using the VSTestConsole.exe utility. Type in the following into the command window.

VSTest.Console.exe MyClassesTest.dll /Logger:trx

Press the Enter key and the unit testing will run. The results are now stored in a .trx file located in the TestResults folder under your \bin\Debug folder. Each time you run a new .trx file is added with a later date and time add to the file name.

Double click on any of the *.trx files and it will load the results into a Test Results window in Visual Studio as shown in Figure 4. You can double-click on any of the tests to see the output from that test. 

Figure 4: Visual Studio can display all results from a .trx file

Run Specific Test(s)

The VSTest.Console utility allows you to specify single or multiple test method names to run. If you are just testing one or two items, there is no reason to run all the tests in your DLL. Add the /Tests parameter on the command line followed by a comma-delimited list of method names you wish to execute. Type the following into your command window.

VSTest.Console.exe MyClassesTest.dll /Tests:FileNameDoesExist

Press the Enter key and you should see a result that looks like Figure 5.

Figure 5: The /Tests parameter does pattern matching on your method names

Notice that multiple tests where run even though you only specified a single name. This is because the /Tests parameter uses pattern matching on your method names. It will find any method that starts with the name you pass in and run those methods.

You can use a comma-delimited list to specify different sets of methods to run. Type the following into your command window.

VSTest.Console.exe MyClassesTest.dll 
   /Tests:FileNameDoesExist,
    FileNameNullOrEmpty_ThrowsArgumentNullException

Press the Enter key and you should see results that look similar to Figure 6.

Figure 6: You may use a comma-delimited list after the /Tests parameter

Filter Tests to Run based on Attributes

As mentioned in the blog post on attributes, the Priority attribute is not used by the unit test framework. However, when you use the VSTest.Console utility, you are allowed to filter based on the Priority attribute. Type in the following to the command window and you can run just those methods that have the Priority(1) attribute.

VSTest.Console.exe MyClassesTest.dll 
   /TestCaseFilter:"Priority=1" 

The /TestCaseFilter lets you specify attributes and specific names of methods to run. For example, if you want to just run one test with the name of FileNameDoesExist, you type in the following into the command window to run that one test.

VSTest.Console.exe MyClassesTest.dll 
   /Name:"FileNameDoesExist" 

Another attribute you can specify to run is TestCategory. Run the following in the command window to just run those tests marked with [TestCategory(“NoException”)].

VSTest.Console.exe MyClassesTest.dll
   /TestCaseFilter:"TestCategory=NoException"

You are not allowed to use both the /TestCaseFilter parameter and the /Tests parameter together. You must just run one or the other.

Summary

Running unit tests in a batch is made easy with the VSTest.Console.exe utility. This utility allows you to log the output to a file for later review. You can also log to TFS to help you assign work items to unit tests that fail. The VSTest.Console utility also lets you filter the tests to run using either a /Tests parameter, or a /TestCaseFilter parameter.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Paul D. Sheriff
Employed (other) PDS Consulting
United States United States
Paul has been in the IT industry over 34 years. In that time he has successfully assisted hundreds of companies architect software applications to solve their toughest business problems. Paul has been a teacher and mentor through various mediums such as video courses, blogs, articles and speaking engagements at user groups and conferences around the world. Paul has 25 courses in the www.pluralsight.com library (https://www.pluralsight.com/author/paul-sheriff) on topics ranging from JavaScript, Angular, MVC, WPF, XML, jQuery and Bootstrap. Contact Paul at psheriff@pdsa.com.

Comments and Discussions

 
-- There are no messages in this forum --