Tests
Test frameworks
.Net has a number of testing frameworks. The major ones:
- NUnit, which has been around since .net 1, originally ported from Java.
- MsTest (Visual Studio test tools) - since Visual Studio 2005/.net 2, the most common.
- XUnit - since 2007, including .net 3.5 and widely used inside Microsoft.
MsTest is the most used. While it may not be the most modern (the core framework is basically unchanged since 2005), it is reliable and easy. Instead of adding richer APIs like NUnit, Microsoft invested into little used things like asp.net unit tests, Coded UI tests, private method accessors and fakes. Both NUnit and XUnit have more "modern" features and are important in open-source, but less so within large companies.
Since Visual Studio 2012, Visual Studio has test adapters, so can easily run NUnit, XUnit and others (before you used to need special runners like Resharper or TestDriven). Originally the adapters were Visual Studio extensions (vsix); now nuget packages are prefered (xunit.runner.visualstudio or NUnit3TestAdapter added to one test project; it automatically applies to the entire solution).
The popularity of Behavior Driven Development has led to some "specification" frameworks like Ruby's Cucumber with its Gherkin language (Scenario/Given/When/Then). Some, like Specflow, are intended used for acceptance tests while retaining *unit for unit tests. More common is simply to name the tests as "Given_x_When_y_Then_z" (with underscores or just capitalization).
- SpecFlow
- Machine.Specifications (MSpec)
- There's a .net version of Fitnesse (integration decision tables defined in html/wiki): FitSharp
For web UI testing, WatIN has been around for ages and is excellent; otherwise use Selenium or standard javascript test tools.
Assertions
You can also use separate "assertion libraries" like Should, Shouldly and FluentAssertions. These fluent styles, like string.Should().Be.Equal("x"), can be quite verbose.
Or you can simply use Assert.AreEqual/ Assert.IsNull / Assert.IsTrue for 95% of tests.
NUnit vs MsTest vs XUnit
For simple unit tests, there's not much difference. For comparison
- NUnit uses [TestFixture] and [Test] attributes.
- MsTest uses [TestClass] and [TestMethod] attributes.
- XUnit uses no class marker, [Fact] attribute or [Theory] (in xunit.extensions for v1 only)
XUnit v2 runs tests in parallel. To run sequentially: [assembly: CollectionBehavior(DisableTestParallelization = true)]
using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using NUnit.Framework;
using Xunit;
namespace UnitTestProject1
{
//XUnit no class attribute
[TestClass] //MsTest
[TestFixture] //NUnit
public class UnitTest1
{
[TestMethod] //MsTest
[Test] //NUnit
[Fact] //XUnit
public void TestMethod1()
{
var result = 1 + 2;
//assert equals
Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual(3, result, "AreEqual optional message");
NUnit.Framework.Assert.AreEqual(3, result, "AreEqual optional message");
NUnit.Framework.Assert.That(result, Is.EqualTo(3)); //alternative form
Xunit.Assert.Equal(3, result); //Equal, not AreEqual, no message
//type
Microsoft.VisualStudio.TestTools.UnitTesting.Assert.IsInstanceOfType(3, typeof(int));
NUnit.Framework.Assert.IsInstanceOf(typeof(int), 3); //note different order
NUnit.Framework.Assert.IsInstanceOf<int>(3); //generic version
NUnit.Framework.Assert.That(3, Is.TypeOf<int>()); //constraint version
Xunit.Assert.IsType<int>(3);
//inconclusive
if (DateTime.Now.DayOfWeek == DayOfWeek.Monday)
Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Inconclusive("May not work on Mondays");
if (DateTime.Now.DayOfWeek == DayOfWeek.Tuesday)
NUnit.Framework.Assert.Inconclusive("May not work on Tuesdays");
//XUnit has no inconclusive
NUnit.Framework.Assert.Ignore("Causes it to be ignored");
}
}
}
Integration tests may need to return Inconclusive rather than Passed or Failed (environmental dependencies aren't present). NUnit and MsTest support Inconclusive, but XUnit doesn't.
Exceptions
MsTest and NUnit have the [ExpectedException(typeof(x))] attribute.
NUnit and XUnit have Assert.Throws<x>
Sometimes a simple try/catch with Assert.Fail() is clearer.
[TestMethod, Microsoft.VisualStudio.TestTools.UnitTesting.ExpectedException(typeof(ArgumentOutOfRangeException))] //MsTest
[Test, NUnit.Framework.ExpectedException(typeof(ArgumentOutOfRangeException))] //NUnit
[Fact] //XUnit has no ExpectedException (see body)
public void ExceptionTest()
{
//throw new InvalidOperationException();
NUnit.Framework.Assert.Throws<ArgumentOutOfRangeException>(() => new DateTime(-1));
Xunit.Assert.Throws<ArgumentOutOfRangeException>(() => new DateTime(-1));
}
Ignoring tests
Removing the Test/TestMethod/Fact attribute is easiest, but sometimes you want to explicitly ignore.
[Microsoft.VisualStudio.TestTools.UnitTesting.Ignore] //MsTest
[NUnit.Framework.Ignore] //NUnit
[Fact(Skip = "Ignore it")] //XUnit
public void BrokenTest()
{
//don't run this
}
See more about the test lifecycle events and parametrized Tests