Continuous Delivery with TFS / VSTS – Enhancing a CI Build to Help Bake Quality In

In the previous installment of this blog post series on Continuous Delivery with TFS / VSTS, we created a basic CI build. In this post, we enhance the CI build with further configurations that can help bake quality into the application. Just a reminder that I’m using TFS to create my CI build as it’s the lowest common denominator. If you are using VSTS, you can obviously follow along, but do note that screenshots might vary slightly.

Set Branch Policies

Although it’s only marginally related to build, this is probably a good point to set branch policies for the master branch of the ContosoUniversity repository. In the Web Portal for the team project, click on the cog icon at the far right of the blue banner bar:

This will open up the Control panel at the team project administration page. Navigate to the Version Control tab and in the Repositories pane, navigate down to master. In the right pane, select Branch Policies:

The branch policies window contains configuration settings that block poor code from polluting the code base. The big change is that the workflow now changes from being able to commit to the master branch directly to having to use pull requests to make commits. This is a great way of enforcing code reviews and I have more detail on the workflow here. In the screenshot above, I’ve selected all the options, including selecting the ContosoUniveristy.CI build to be run when a pull request is created. This blocks any pull requests that would cause the build to fail. The other options are self explanatory, although enforcing a linked work item can be a nuisance when you are testing. If you are testing on your own, make sure you Allow users to approve their own changes, otherwise this will cause you a problem.

Testing Times

The Contoso University sample application contains MSTest unit tests and we want these to be run after the build to provide early feedback of any failing tests. This is achieved by adding a new build step. On the Build tab in the Web Portal, navigate to the ContosoUniversity.CI build and place it in edit mode. Click on Add build step and from the Add Tasks window, filter on Test and choose Visual Studio Test.

For our simple scenario, there are only three settings that need addressing:

• Test Assembly – We only want unit tests to run and ContosoUniversity contains other tests so changing the default setting to **\*UnitTests*.dll;-:**\obj\** fixes this.
• Platform – Here, we use the $(BuildPlatform) variable defined in the build task.
• Configuration – Here, we use the $(BuildConfiguration) variable defined in the build task.

With the changes saved, queue the build and observe the build report advising that the tests were run and passed:

Code Coverage

In the above screenshot, you’ll notice that there is no code coverage data available. This can be fixed by going back to the Visual Studio Test task and checking the Code Coverage Enabled box. Queueing a new build now gives us that data:

Of slight concern is that the code coverage reported from the build (2.92%) was marginally higher than that reported by analysing code coverage in Visual Studio (2.89%). Whilst the values are the same for all practical purposes, the results suggest that there is something odd going on here that warrants further investigation.

Code Analysis

A further feedback item which is desirable to have in the build report is the results of code analysis. (As a reminder, we configured this in Visual Studio in this post so that the results are available after building locally.) Displaying code analysis results in the build report is straightforward for XAML builds as this is an out-of-the-box setting – see here. I haven’t found this to be quite so easy with the new build system.There’s no setting as with XAML builds but that shouldn’t be a problem since it’s just an MSBuild argument. It feels like the correct argument should be /p:RunCodeAnalysis=Always (as this shouldn’t care how code analysis is configured in Visual Studio). However in my testing, I couldn’t get this to work with any combination of the Visual Studio Build / MSBuild task and release / debug configurations. Next argument I tried was /p:RunCodeAnalysis=True. This worked with either Visual Studio Build or MSBuild task, but to get it to work in a release configuration, you will need to ensure that code analysis has been enabled for the release configuration in Visual Studio (and the change has been committed!). The biggest issue though was that I never managed to get more than 10 code analysis rules displayed in the build report when there were 85 reported in the build output. Perhaps I’m missing something here – if you can shed any light on this, please let me know!

Don’t Ignore the Feedback!

Finally, it may sound obvious, but there’s little point in configuring the build report to contain feedback on the quality of the build if nobody is looking at the reports and is doing something to remedy problems. However, you do it as this needs to be part of your team’s daily routine!

Cheers – Graham

The post Continuous Delivery with TFS / VSTS – Enhancing a CI Build to Help Bake Quality In appeared first on Please Release Me.