Testing Windows 10 Apps

I’m a big supporter of TDD and that’s why I was really looking forward to setup a project for testing Windows 10 apps and see how it works. Unfortunately, testing Windows 10 apps is not easy, especially if you’re using Caliburn.Micro as I am.

Originally, I wanted to play a little with NUnit and MSTest and make a blog post about what’s different in Windows 10, but instead this post will be about what’s working and what’s not working.

Here’s how my perfect setup for TDD would look like:

1. Nunit tests + Resharper test runner
2. The test project can reference Caliburn.Micro so I can test ViewModels
3. Tests run fast
4. I can use a mock library in my tests (I prefer Moq)
5. I can use a dependency injector in my tests project (I prefer Ninject)

To my surprise, no test framework works 100% with UWP apps that use Caliburn.Micro, so I made a project with 3 tests that will help me determine how much I can test in my UWP project.

Here are the tests:

Test 1

This one is a simple assert, it doesn’t reference any UWP app. I used it just to see if the test is detected by the test runners and if it will run.

[TestMethod] //this is a MSTest example, but I also used NUnit
public void FirstTest()
{
   Assert.AreEqual(1, 1);
}

Test 2

This one has a reference to my universal app project. The Calculator class is from that project.

[TestMethod]
public void SecondTest()
{
   Assert.AreEqual(2, MyUniversalApp.Calculator.Add(1, 1));
}

Test 3

This one also has a reference to the universal project and it tests a method from a ViewModel. This will check to see if the testing framework works with Caliburn.Micro.

[TestMethod]
public void ThirdTest()
{
    Assert.AreEqual(1, new MyUniversalApp.MainViewModel().Add(0, 1));
}

Now that I showed you my simple tests, I’ll take MSTest and NUnit and see which work and which won’t work, using different types of projects and test runners. I also tried XUnit but nothing worked with it so I won’t include it in this article. Don’t get me wrong, these test frameworks run on other types of projects (console, Web API, WPF, etc), and maybe they run with other MVVM frameworks for UWP apps (MVVM Light maybe?), but in this article I’m only trying to see if they run with Caliburn.Micro apps, so I won’t judge them for not working with a very specific type of project, especially when they’re in a beta stage.

MSTest

MSTest needs an app container to run your tests, so the only way to do this is by using the Unit Test App project.

I tried to make it work with a class library but Resharper won’t run the tests and the VS test explorer won’t find them, so if you want MSTest, you can only use a Unit Test App. The first test works, and also the second one but only if the project that you’re testing is not using Caliburn.Micro. If you add a reference to a project that has Caliburn.Micro, the MSTest runner will fail with this exception:

The active Test Run was aborted because the execution process exited unexpectedly. To investigate further, enable local crash dumps either at the machine level or for process . For more details, click on this link.
Failed to initialize client proxy: could not connect to test process .

To sum it up, if you use Caliburn.Micro, MSTest won’t work for you, but maybe NUnit will? Let’s see!

NUnit

I was sure that this approach would be the one that will work on all three cases…but unfortunately this is not the case for a few reasons. First, forget about using the Resharper test runner with NUnit because Resharper only supports NUnit 2.6.4 (latest stable release), and NUnit only supports the Windows 10 platform in NUnit 3.0 beta 3. The next step was to make Resharper work with this version of NUnit, but I wasn’t able to do it. Resharper was complaining that some files were missing from NUnit and because both tools are not ready yet, I won’t stress too much about making them work together in this stage, instead I’ll wait for them to have a stable release and then I’ll try to find a workaround if they still won’t work.

Ok, so Resharper test runner doesn’t work, but we still got the NUnit test runner, right? Well, I installed the NUnit 3 Test Adapter and it won’t find my tests. I also added the [TestFixture] attribute to the class (which I know is not mandatory) and it still won’t find my tests.

After a little Google searching, I found a test adapter from testdriven.net which works but only for the first two tests, so you can test projects that use Caliburn.Micro but you can’t test the ViewModels. Another downside is that you can only run one test at a time by right clicking on it and then click on Run Test(s).

If you use the option to run the tests from the context menu of the project, you will get this error or warning and it won’t run any test:

Unknown .NET Framework Version: v5.0

Maybe they will add support for this in the near future, but for the moment this is unacceptable for doing TDD because I need all the tests to run at once, and going through them one by one would take a lot of time.

These are the tools that I used:

• Visual Studio 2015 Enterprise
• Resharper Ultimate 2015.2 EAP 7
• NUnit 3.0 beta 3
• NUnit 3 Test Adapter
• TestDriven.net 3.9.2904 Personal beta 3

I know that Visual Studio 2015 just launched and these tools are barely in beta phase, but it’s hard for me to start working on UWP apps without a testing framework that works. Let’s hope that Resharper will add support for NUnit 3 as soon as possible. I’ll keep following the development of these tools and try to make them work together, so follow my blog if you want to be among the first ones to know when you’ll be able to write unit tests with Caliburn.Micro apps.

I always hated the fact that testing is not easy with Windows Store/Windows Phone apps, I mean look at an Web API project for example, you just make a class library and every tool works, you have Moq, you have Ninject, NUnit works without problems, and you don’t have to do any workarounds to test that code. For me it’s perfect, and I hoped that this was the day that I would say the same about testing UWP apps, but unfortunately it’s not.

If you read this far, you should probably follow me on twitter: Follow @thewindev.

The post Testing Windows 10 apps appeared first on Bogdan Bujdea.