OpenCover and ReportGenerator Unit Test Coverage in Visual Studio 2013 and 2015

The usual choices for getting code coverage metrics in Visual Studio is to buy the Enterprise version of Visual Studio or buy the dotCover third party tool, both of which are costly, especially if you are developing small applications yourself and want to check the coverage of your unit tests.

There is a free NuGet package for generating code coverage metrics called OpenCover, and together with another free NuGet package, ReportGenerator, can give you code coverage reports for free.

The problem I found, however, was the documentation available to get the tool working was less than adequate for my purposes and it took several hours of faff to get it working for me. To save you time and money, I have documented my experience here.

The following were all referenced in the creation of this document:

• https://www.nuget.org/packages/OpenCover/
• https://www.nuget.org/packages/ReportGenerator/
• https://www.nuget.org/packages/NUnit.ConsoleRunner/
• http://www.allenconway.net/2015/06/using-opencover-and-reportgenerator-to.html
• http://www.nootn.com.au/2014/01/code-coverage-with-opencover-example.html#.VxiNn_krLDc

I have tested that this works in Visual Studio 2013 Professional and Visual Studio 2015 Community Edition, with both MSTest and NUnit.

MSTest unit tests can be run with either mstest.exe or the newer vstest.console.exe. As I primarily use NUnit, I have not done anything other than basic testing to ensure OpenCover works with both these test runners so cannot say how either might work on large projects.

I assume you already have a solution with application project that you are testing with a test project, and where you are using NUnit for that testing that you have already included the NUnit NuGet package.

You will need to open your solution in Visual Studio and get the following packages using Package Manager Console:

REM PM> Install-Package OpenCover
REM PM> Install-Package ReportGenerator

If you are using NUnit rather than MSTest, you will also need this package:

REM PM> Install-Package NUnit.ConsoleRunner

In the solution root folder, you will then need to create a batch file that you will use to produce the code coverage report. I have three batch files shown below, one for use where you are using NUnit, one where you are using MSTest and the mstest.exe test runner, and one for where you are using MSTest and the vstest.console.exe test runner.

You will need to modify the batch file to specify the DLL that contains your unit tests, i.e., change this line as appropriate (note that %~dp0 is a symbol that is replaced automatically with the folder path of the batch file):

SET DllContainingTests=%~dp0WebApp.Tests\bin\Debug\WebApp.Tests.dll

You may also want to change the filter applied to the results to exclude parts of your solution where you are not interested in the code coverage metrics, in my case, I have asked it to include everything (+[*]*), and then exclude my Tests project (-[*.Tests*]*) and all classes ending in Config (-[*]*.*Config), i.e.:

-filter:"+[*]* -[*.Tests*]* -[*]*.*Config"

If you are using MSTest, you may need to change the TestRunnerExe variable to point to the correct version of mstest.exe or vstest.runner.exe. The examples below are pointing to the ones in Visual Studio 2013, but if you are using 2015, you will need to change this to Microsoft Visual Studio 14.0 instead of Microsoft Visual Studio 12.0.

All that then remains is to execute the batch file to produce a report of code coverage that allows you to drill down to see the actual colour coded code being covered, or not.

NUnit Version

@ECHO OFF

REM OpenCover-NUnit.bat

REM Run opencover against NUnit tests in your test project and show report of code coverage

REM Derivative work based on work by: 
REM  Shaun Wilde - https://www.nuget.org/packages/OpenCover/
REM  Daniel Palme - https://www.nuget.org/packages/ReportGenerator/
REM  Charlie Poole - https://www.nuget.org/packages/NUnit.ConsoleRunner/
REM  Allen Conway - 
REM   http://www.allenconway.net/2015/06/using-opencover-and-reportgenerator-to.html
REM  Andrew Newton - 
REM   http://www.nootn.com.au/2014/01/code-coverage-with-opencover-example.html#.VxiNn_krLDc


SET DllContainingTests=%~dp0WebApp.Tests\bin\Debug\WebApp.Tests.dll


REM *** IMPORTANT - Change DllContainingTests variable (above) to point to the DLL 
REM ***             in your solution containing your NUnit tests
REM ***
REM ***             You may also want to change the include/exclude filters (below) 
REM ***             for OpenCover
REM ***
REM ***             This batch file should dbe placed in the root folder of your solution

REM *** Before being able to use this to generate coverage reports you 
REM *** will need the following NuGet packages
REM PM> Install-Package OpenCover
REM PM> Install-Package ReportGenerator
REM PM> Install-Package NUnit.ConsoleRunner
REM

REM NUnit Test Runner (done this way so we dont have to change the code 
REM when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="nunit3-console.exe" SET TestRunnerExe=%%~dpnxa

REM Get OpenCover Executable (done this way so we dont have to change the 
REM code when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="OpenCover.Console.exe" SET OpenCoverExe=%%~dpnxa

REM Get Report Generator (done this way so we dont have to change the code 
REM when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="ReportGenerator.exe" 
    SET ReportGeneratorExe=%%~dpnxa

REM Create a 'GeneratedReports' folder if it does not exist
if not exist "%~dp0GeneratedReports" mkdir "%~dp0GeneratedReports"

REM Run the tests against the targeted output
call :RunOpenCoverUnitTestMetrics

REM Generate the report output based on the test results
if %errorlevel% equ 0 ( 
 call :RunReportGeneratorOutput 
)

REM Launch the report
if %errorlevel% equ 0 ( 
 call :RunLaunchReport 
)
exit /b %errorlevel%

:RunOpenCoverUnitTestMetrics 
REM *** Change the filter to include/exclude parts of the solution you want 
REM *** to check for test coverage
"%OpenCoverExe%" ^
 -target:"%TestRunnerExe%" ^
 -targetargs:"\"%DllContainingTests%\"" ^
 -filter:"+[*]* -[*.Tests*]* -[*]*.*Config" ^
 -mergebyhash ^
 -skipautoprops ^
 -register:user ^
 -output:"%~dp0GeneratedReports\CoverageReport.xml"
exit /b %errorlevel%

:RunReportGeneratorOutput
"%ReportGeneratorExe%" ^
 -reports:"%~dp0\GeneratedReports\CoverageReport.xml" ^
 -targetdir:"%~dp0\GeneratedReports\ReportGenerator Output"
exit /b %errorlevel%

:RunLaunchReport
start "report" "%~dp0\GeneratedReports\ReportGenerator Output\index.htm"
exit /b %errorlevel%

MSTest Version Using mstest.exe

@ECHO OFF

REM OpenCover-MSTest.bat

REM Run opencover against MSTest tests in your test project and show report of code coverage

REM Derivative work based on work by: 
REM  Shaun Wilde - https://www.nuget.org/packages/OpenCover/
REM  Daniel Palme - https://www.nuget.org/packages/ReportGenerator/
REM  Allen Conway - 
REM   http://www.allenconway.net/2015/06/using-opencover-and-reportgenerator-to.html
REM  Andrew Newton - 
REM   http://www.nootn.com.au/2014/01/code-coverage-with-opencover-example.html#.VxiNn_krLDc


SET DllContainingTests=%~dp0WebApp.Tests\bin\Debug\WebApp.Tests.dll


REM *** IMPORTANT - Change DllContainingTests variable (above) to point to the DLL 
REM ***             in your solution containing your NUnit tests
REM ***
REM ***             You may also want to change the include/exclude filters 
REM ***             (below) for OpenCover
REM ***
REM ***             This batch file should dbe placed in the root folder of your solution

REM *** Before being able to use this to generate coverage reports you will 
REM *** need the following NuGet packages
REM PM> Install-Package OpenCover
REM PM> Install-Package ReportGenerator
REM

REM *** MSTest Test Runner (VS2013, will need to change 12.0 to 14.0 for VS2015)
SET TestRunnerExe=%PROGRAMFILES(X86)%\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe

REM Get OpenCover Executable (done this way so we dont have to change 
REM the code when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="OpenCover.Console.exe" SET OpenCoverExe=%%~dpnxa

REM Get Report Generator (done this way so we dont have to change the code 
REM when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="ReportGenerator.exe" 
     SET ReportGeneratorExe=%%~dpnxa

REM Create a 'GeneratedReports' folder if it does not exist
if not exist "%~dp0GeneratedReports" mkdir "%~dp0GeneratedReports"

REM Run the tests against the targeted output
call :RunOpenCoverUnitTestMetrics

REM Generate the report output based on the test results
if %errorlevel% equ 0 ( 
 call :RunReportGeneratorOutput 
)

REM Launch the report
if %errorlevel% equ 0 ( 
 call :RunLaunchReport 
)
exit /b %errorlevel%

:RunOpenCoverUnitTestMetrics 
REM *** Change the filter to include/exclude parts of the solution you want to 
REM *** check for test coverage
"%OpenCoverExe%" ^
 -target:"%TestRunnerExe%" ^
 -targetargs:"/noisolation /testcontainer:\"%DllContainingTests%\"" ^
 -filter:"+[*]* -[*.Tests*]* -[*]*.Global -[*]*.RouteConfig -[*]*.WebApiConfig" ^
 -mergebyhash ^
 -skipautoprops ^
 -register:user ^
 -output:"%~dp0GeneratedReports\CoverageReport.xml"
exit /b %errorlevel%

:RunReportGeneratorOutput
"%ReportGeneratorExe%" ^
 -reports:"%~dp0\GeneratedReports\CoverageReport.xml" ^
 -targetdir:"%~dp0\GeneratedReports\ReportGenerator Output"
exit /b %errorlevel%

:RunLaunchReport
start "report" "%~dp0\GeneratedReports\ReportGenerator Output\index.htm"
exit /b %errorlevel%

MSTest Version Using vstest.console.exe

@ECHO OFF

REM OpenCover-VSTest.bat

REM Run opencover against MSTest tests in your test project and show report of code coverage

REM Derivative work based on work by: 
REM  Shaun Wilde - https://www.nuget.org/packages/OpenCover/
REM  Daniel Palme - https://www.nuget.org/packages/ReportGenerator/
REM  Allen Conway - 
REM   http://www.allenconway.net/2015/06/using-opencover-and-reportgenerator-to.html
REM  Andrew Newton - 
REM   http://www.nootn.com.au/2014/01/code-coverage-with-opencover-example.html#.VxiNn_krLDc


SET DllContainingTests=%~dp0WebApp.Tests\bin\Debug\WebApp.Tests.dll


REM *** IMPORTANT - Change DllContainingTests variable (above) to point to the DLL 
REM ***             in your solution containing your NUnit tests
REM ***
REM ***             You may also want to change the include/exclude filters 
REM ***             (below) for OpenCover
REM ***
REM ***             This batch file should dbe placed in the root folder of your solution

REM *** Before being able to use this to generate coverage reports you will 
REM *** need the following NuGet packages
REM PM> Install-Package OpenCover
REM PM> Install-Package ReportGenerator
REM

REM *** MSTest Test Runner (VS2013, will need to change 12.0 to 14.0 for VS2015)
SET TestRunnerExe=%PROGRAMFILES(X86)%\Microsoft Visual Studio 12.0\Common7\IDE\
CommonExtensions\Microsoft\TestWindow\vstest.console.exe

REM Get OpenCover Executable (done this way so we dont have to change 
REM the code when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="OpenCover.Console.exe" SET OpenCoverExe=%%~dpnxa

REM Get Report Generator (done this way so we dont have to change the code 
REM when the version number changes)
for /R "%~dp0packages" %%a in (*) do if /I "%%~nxa"=="ReportGenerator.exe" 
   SET ReportGeneratorExe=%%~dpnxa

REM Create a 'GeneratedReports' folder if it does not exist
if not exist "%~dp0GeneratedReports" mkdir "%~dp0GeneratedReports"

REM Run the tests against the targeted output
call :RunOpenCoverUnitTestMetrics

REM Generate the report output based on the test results
if %errorlevel% equ 0 ( 
 call :RunReportGeneratorOutput 
)

REM Launch the report
if %errorlevel% equ 0 ( 
 call :RunLaunchReport 
)
exit /b %errorlevel%

:RunOpenCoverUnitTestMetrics 
REM *** Change the filter to include/exclude parts of the solution you want to 
REM *** check for test coverage
"%OpenCoverExe%" ^
 -target:"%TestRunnerExe%" ^
 -targetargs:"\"%DllContainingTests%\"" ^
 -filter:"+[*]* -[*.Tests*]* -[*]*.Global -[*]*.RouteConfig -[*]*.WebApiConfig" ^
 -mergebyhash ^
 -skipautoprops ^
 -register:user ^
 -output:"%~dp0GeneratedReports\CoverageReport.xml"
exit /b %errorlevel%

:RunReportGeneratorOutput
"%ReportGeneratorExe%" ^
 -reports:"%~dp0\GeneratedReports\CoverageReport.xml" ^
 -targetdir:"%~dp0\GeneratedReports\ReportGenerator Output"
exit /b %errorlevel%

:RunLaunchReport
start "report" "%~dp0\GeneratedReports\ReportGenerator Output\index.htm"
exit /b %errorlevel%