Testing for Thrown Exceptions in xUnit.net

When writing tests it is sometimes useful to check that the correct exceptions are thrown at the expected time.

When using xUnit.net there are a number of ways to accomplish this.

As an example consider the following simple class:

public class TemperatureSensor
{
    bool _isInitialized;

    public void Initialize()
    {
        // Initialize hardware interface
        _isInitialized = true;
    }

    public int ReadCurrentTemperature()
    {
        if (!_isInitialized)
        {
            throw new InvalidOperationException("Cannot read temperature before initializing.");
        }

        // Read hardware temp
        return 42; // Simulate for demo code purposes
    }        
}

The first test we could write against the preceding class is to check the “happy path”:

[Fact]
public void ReadTemperature()
{
    var sut = new TemperatureSensor();

    sut.Initialize();

    var temperature = sut.ReadCurrentTemperature();

    Assert.Equal(42, temperature);
}

Next a test could be written to check that if the temperature is read before initializing the sensor, an exception of type InvalidOperationException is thrown. To do this the xUnit.net Assert.Throws method can be used. When using this method the generic type parameter indicates the type of expected exception and the method parameter takes an action that should cause this exception to be thrown, for example:

[Fact]
public void ErrorIfReadingBeforeInitialized()
{
    var sut = new TemperatureSensor();

    Assert.Throws<InvalidOperationException>(() => sut.ReadCurrentTemperature());
}

In the preceding test, if an InvalidOperationException is not thrown when the ReadCurrentTemperature method is called the test will fail.

The thrown exception can also be captured in a variable to make further asserts against the exception property values, for example:

[Fact]
public void ErrorIfReadingBeforeInitialized_CaptureExDemo()
{
    var sut = new TemperatureSensor();

    var ex = Assert.Throws<InvalidOperationException>(() => sut.ReadCurrentTemperature());

    Assert.Equal("Cannot read temperature before initializing.", ex.Message);
}

The Assert.Throws method expects the exact type of exception and not derived exceptions. In the case where you want to also allow derived exceptions, the Assert.ThrowsAny method can be used.

Similar exception testing features also exist in MSTest and NUnit frameworks.

To learn more about using exceptions to handle errors in C#, check out my Error Handling in C# with Exceptions Pluralsight course.

SHARE:

MSTest V2

In the (relatively) distant past, MSTest was often used by organizations because it was provided by Microsoft “in the box” with Visual Studio/.NET. Because of this, some organizations trusted MSTest over open source testing frameworks such as NUnit. This was at a time when the .NET open source ecosystem was not as advanced as it is today and before Microsoft began open sourcing some of their own products.

Nowadays MSTest is cross-platform and open source and is known as MSTest V2, and as the documentation states: “is a fully supported, open source and cross-platform implementation of the MSTest test framework with which to write tests targeting .NET Framework, .NET Core and ASP.NET Core on Windows, Linux, and Mac.”.

MSTest V2 provides typical assert functionality such as asserting on the values of: strings, numbers, collections, thrown exceptions, etc. Also like other testing frameworks, MSTest V2 allows the customization of the test execution lifecycle such as the running of additional setup code before each test executes. The framework also allows the creation of data driven tests (a single test method executing  multiple times with different input test data) and the ability to extend the framework with custom asserts and custom test attributes.

You can find out more about MSTest V2 at the GitHub repository, the documentation, or check out my Pluralsight course: Automated Testing with MSTest V2.

SHARE:

Mocking with FeatureToggle

I was asked a question on Twitter so I thought I’d write it up here.

When using the FeatureToggle library you may have some some code that behaves differently if a toggle is enabled.

When writing a test, you can create a mock IFeatureToggle and set it up to be enabled (or not) and then assert the result is as expected.

The following code show a simple console app that has an OptionsConsoleWriter.Generate method that uses a toggle to output a printing feature option:

using static System.Console;
using System.Text;
using FeatureToggle.Toggles;
using FeatureToggle.Core;

namespace ConsoleApp1
{
    public class Printing : SimpleFeatureToggle {}

    public class OptionsConsoleWriter
    {
        public string Generate(IFeatureToggle printingToggle)
        {
            var sb = new StringBuilder();

            sb.AppendLine("Options:");
            sb.AppendLine("(e)xport");
            sb.AppendLine("(s)ave");

            if (printingToggle.FeatureEnabled)
            {
                sb.AppendLine("(p)rinting");
            }

            return sb.ToString();
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            Printing printingToggle = new Printing();

            string options = new OptionsConsoleWriter().Generate(printingToggle);

            Write(options);            

            ReadLine();
        }
    }
}

To write a couple of simple tests for this method, you can use a mocking framework such as Moq to generate a mocked IFeatureToggle and pass it to the Generate method:

using Xunit;
using Moq;
using FeatureToggle.Core;
using ConsoleApp1;

namespace ClassLibrary1.Tests
{
    public class OptionsConsoleWriterTests
    {
        [Fact]
        public void ShouldGeneratePrintingOption()
        {
            var sut = new OptionsConsoleWriter();

            var mockPrintingToggle = new Mock<IFeatureToggle>();
            mockPrintingToggle.SetupGet(x => x.FeatureEnabled)
                              .Returns(true);

            string options = sut.Generate(mockPrintingToggle.Object);

            Assert.Contains("(p)rinting", options);
        }

        [Fact]
        public void ShouldNotGeneratePrintingOption()
        {
            var sut = new OptionsConsoleWriter();

            var mockPrintingToggle = new Mock<IFeatureToggle>();
            mockPrintingToggle.SetupGet(x => x.FeatureEnabled)
                              .Returns(false);

            string options = sut.Generate(mockPrintingToggle.Object);

            Assert.DoesNotContain("(p)rinting", options);
        }
    }
}

SHARE:

Grouping and Filtering Tests in Visual Studio Test Explorer

One way to run automated tests is to use Visual Studio’s Test Explorer. Test Explorer can be found under the Test –> Windows –> Test Explorer menu items.

In this article we’ll look at how to manage the list of tests using grouping and also how to specify custom search filter expressions.

Two test projects in Visual Studio solution

Grouping Tests

There are a number of ways to group tests in Test Explorer, at the highest structural level we can group by project.

To select a group by method, click the drop down arrow as show in the following screenshot:

Selecting a grouping in Visual Studio Test Explorer

With the grouping set to Project, the test list looks as follows:

image

The next structural grouping is Class:

Grouping by test class

The final structural grouping is by Namespace:

Grouping by namespace

There are a number of non-structural groupings.

Group by Duration:

Group by duration

Group by Outcome:

Group by outcome

…and group by Traits:

Grouping by traits

Filtering Tests

Custom filters can also be applied.

For example by file path:

Filtering tests by file path

Other search examples include:

  • Trait:"Smoke Test"
  • Message:"System.Exception"
  • Class1
  • Outcome:"Passed"
  • Outcome:"Failed"

Subsets can also be excluded by prefixing the type of filter with a -. For example to show all tests in Class1 except failed tests: Class:"TestClass1" -Outcome:"Passed".

SHARE:

Getting Started Testing .NET Core Code with xUnit.net

xUnit.net is a testing framework that can be used to write automated tests for .NET (full) framework and also .NET Core.

To get started, first create a .NET Core application, in the following example a .NET Core console app.

Creating a .NET core console project

A testing project can now be added to the solution:

Adding an xUnit test project in Visual Studio 2017

This test project will come pre-configured with the relevant NuGet packages installed to start writing test code, though you may want to update the pre-configured packages to the newest NuGet versions.

The xUnit Test Project template will also create the following default test class:

using System;
using Xunit;

namespace ConsoleCalculator.Tests
{
    public class UnitTest1
    {
        [Fact]
        public void Test1()
        {

        }
    }
}

Notice in the preceding code, the Test1 method is decorated with the [Fact] attribute. This is an xUnit.net attribute that tells a test runner that it should execute the method, treat it as a test, and report on if the test passed or not.

Next add a project reference from the test project to the project that contains the code that is to be tested, this gives the test project access to the production code.

In the production project, the following class can be added:

namespace ConsoleCalculator
{
    public class Calculator
    {
        public int Add(int a, int b)
        {            
            return a + b;
        }
    }
}

Now the test class can be renamed (for example to “CalculatorTests”) and the test method changed to create a test:

using Xunit;

namespace ConsoleCalculator.Tests
{
    public class CalculatorTests
    {
        [Fact]
        public void ShouldAddTwoNumbers()
        {
            Calculator calculator = new Calculator();

            int result = calculator.Add(7, 3);

            Assert.Equal(10, result);
        }
    }
}

In the preceding code, once again the [Fact] attribute is being used, then the thing being tested is created (the Calculator class instance). The next step is to perform some kind of action on the thing being tested, in this example calling the Add method. The final step is to signal to the test runner if the test has passed or not, this is done by using one of the many xUnit.net Assert methods; in the preceding code the Assert.Equal method is being used. The first parameter is the expected value of 10, the second parameter is the actual value produced by the code being tested. So if  result is 10 the test will pass, otherwise it will fail.

One way to execute tests is to use Visual Studio’s Test Explorer which can be found under the Test –> Windows –> Test Explorer menu item. Once the test project is built, the test will show up and can be executed as the following screenshot shows:

Running xUnit tests in Visual Studio Test Explorer

To learn more about how to get started testing .NET Core code check out my Testing .NET Core Code with xUnit.net: Getting Started Pluralsight course or check out the docs.

SHARE:

Testing Automation: The Big Picture

It’s often useful to take a step back and look at the bigger picture, this is true in different aspects of life such as health or wealth or relationships, and is also true of software development.

When it comes to creating automated tests (as with other aspects of software development) dogmatism and absolutist schools of though can exist.

As with all things, the decision to write tests (and how many tests, what type of tests, test coverage aims, etc.) ultimately should boil down to one question: do they add value to what you are doing?

To be clear, I absolutely believe in the creation of automated tests in many cases, however it is good to not be dogmatic. For example if there is a niche market that is ripe for capitalizing on, and time-to-market is the most important thing to capture this market, then an extensive suite of automated tests may slow down getting to that initial release. This of course assumes that the potential market will have some tolerance for software defects. It also depends on what the product is; medical life-critical software is probably going to have a higher quality requirement than a social media app for example. This can be a trade-off however with shorter term delivery speeds being quicker but at the expense of delivery speed in the long term, if you’re overwhelmed fixing production outages you have very little time to add new features/value.

Another aspect to consider is that of risk. What are the risks associated with defects in the software making their way into production? Different features/application areas may also have different risk profiles; for example  the “share on social media” feature may not be deemed as important as a working shopping cart. It’s also important to remember that risk is not just monetary, in the previous example a broken “share on social media” feature may bring the business into disrepute, aka “reputational risk”.

When it comes to the myriad of different types of tests (unit, integration, subcutaneous, etc.) the testing pyramid is an oft-quoted model of how many of each type of tests to have in the test suite. While the testing pyramid may be of great help for someone new to automated testing to help them learn and navigate their initial steps, as experience grows the model may no longer be optimal to some of the projects that are being worked on. Beyond the testing pyramid the different aspects of test types can be considered such as execution speed, breadth/depth, reliability/brittleness etc.

Automated tests also do not exist in and of themselves, they are part of a bigger set of processes/considerations such as pair programming, code reviews, good management, well-understood requirements, good environment management/DevOps, etc.

If you want to take a step back and look at the big picture, or know a developer or manager who wants to understand the trade-offs/benefits check out my Testing Automation: The Big Picture Pluralsight course.

SHARE:

Testing ASP.NET Core Controllers in Isolation with Mock Objects and Moq

In previous posts we saw how to get started testing ASP.NET Core MVC controllers and also how to use the Moq mocking library in .NET Core tests.

If there is code in controllers that needs testing, but the controller has a dependency, for example passed into the constructor, it may not make sense to use the real version of the dependency. In these cases Moq can be used to create a mock version of the dependency and pass it to the controller that needs testing.

As an example suppose we have the following controller code:

public class HomeController : Controller
{
    private readonly ISmsGateway _smsGateway;

    public HomeController(ISmsGateway smsGateway)
    {
        _smsGateway = smsGateway;
    }

    [HttpPost]
    [ValidateAntiForgeryToken]
    public IActionResult Send(SendSmsRequest request)
    {
        if (ModelState.IsValid)
        {
            var sendReceipt = _smsGateway.Send(request.PhoneNumber, request.Message);

            return Ok(sendReceipt);
        }

        return BadRequest();
    }
}

In the preceding code, the controller takes an ISmsGateway dependency as a constructor parameter. This dependency is later used in the the Send() method.

After installing Moq a mock SMS gateway can be created. Once created, Moq’s Setup() method can be used to determine what happens when the controller calls the mocked Send() method as the following code demonstrates:

[Fact]
public void ShouldSendOk()
{
    SendSmsRequest sendSmsRequest = new SendSmsRequest
    {
        PhoneNumber = "42",
        Message = "Hello"
    };

    Guid expectedSendReceipt = Guid.NewGuid();

    var mockSmsGateway = new Mock<ISmsGateway>();
    
    mockSmsGateway.Setup(x => x.Send(sendSmsRequest.PhoneNumber, sendSmsRequest.Message))
                  .Returns(expectedSendReceipt);

    var sut = new HomeController(mockSmsGateway.Object);
    
    IActionResult result = sut.Send(sendSmsRequest);

    var okObjectResult = Assert.IsType<OkObjectResult>(result);

    Assert.Equal(expectedSendReceipt, okObjectResult.Value);
}

We may also want to test that if there is a model binding error, then  no message is sent via the SMS gateway. The follow test code shows the use of the AddModelError() method to simulate an error, and the use of Moq’s Verify() method to check that the gateway’s Send() method was never called:

[Fact]
public void ShouldNotSendWhenModelError()
{
    SendSmsRequest sendSmsRequest = new SendSmsRequest
    {
        PhoneNumber = "42",
        Message = "Hello"
    };

    var mockSmsGateway = new Mock<ISmsGateway>();

    var sut = new HomeController(mockSmsGateway.Object);
    sut.ModelState.AddModelError("Simulated", "Model error");

    sut.Send(sendSmsRequest);

    mockSmsGateway.Verify(x => x.Send(It.IsAny<string>(), It.IsAny<string>()), Times.Never);
}

To learn more about using Moq to create/configure/use mock objects check out my Mocking in .NET Core Unit Tests with Moq: Getting Started Pluralsight course, and to learn how to get started testing ASP.NET Core MVC applications check out my ASP.NET Core MVC Testing Fundamentals Pluralsight course.

SHARE:

Testing ASP.NET Core MVC Controllers: Getting Started

When writing ASP.NET Core MVC web applications, you may want to test that controller actions behave in the expected way, for example that the action returns the correct result type (e.g. a ViewResult) or that the action behaves as expected when the model state is invalid.

To get started writing controller tests, first add a new .NET Core xUnit test project to the solution. This will create the test project along with requried xUnit.net NuGet packages. It will also add a default test class "UnitTest1.cs":

using System;
using Xunit;

namespace WebApplication1.Tests
{
    public class UnitTest1
    {
        [Fact]
        public void Test1()
        {
        }
    }
}

In the preceding code, notice the xUnit.net [Fact] attribute that marks the Test1 method as a test that should be executed by the test runner. One way to run tests in Visual Studio is to use the built-in Test Explorer which can be accessed via the menus: Test->Windows->Test Explorer.

If you build the project you will see the default test shown in the Test Explorer window:

Visual Studio Test Explorer

Adding a Controller Test

First, to get access to the controllers in the ASP.NET Core MVC application, add a reference to the web project from the test project. An instance of a controller can now be created in the test method:

var sut = new WebApplication1.Controllers.HomeController();

We can now call methods (actions) on the controller and verify the results. As a simple example, we can check that the Index method result is a view:

[Fact]
public void Test1()
{
    HomeController sut = new WebApplication1.Controllers.HomeController();

    IActionResult result = sut.Index();

    Assert.IsType<ViewResult>(result);
}

There are many different ways to test the results of controllers, including the ability to simulate model errors or using Moq mock objects as controller constructor dependencies.

The following code shows an excerpt from a controller and a test that examines the view's model that was returned:

public class PersonViewModel
{
    public string Name { get; set; }
}

public IActionResult Person()
{
    PersonViewModel viewModel = new PersonViewModel
    {
        Name = "Amrit"
    };

    return View(viewModel);
}
[Fact]
public void Test2()
{
    HomeController sut = new WebApplication1.Controllers.HomeController();

    IActionResult result = sut.Person();

    ViewResult viewResult = Assert.IsType<ViewResult>(result);

    PersonViewModel model = Assert.IsType<PersonViewModel>(viewResult.Model);

    Assert.Equal("Amrit", model.Name);
}

To learn how to get started testing ASP.NET Core MVC applications check out my ASP.NET Core MVC Testing Fundamentals Pluralsight course.

SHARE:

Mocking in .NET Core Tests with Moq

When writing automated tests it is sometimes useful to isolate the thing(s) being tested from other parts of the system. These ‘other’ parts may still need to be provided, and sometimes the real versions are too hard or cumbersome to use. In these instances “mocked” versions can be created and used.

A mock version of something is an object that can act like the real thing but can be controlled in test code.

Moq (pronounced “mok u” or “mock”) is a library available on NuGet that allows mock objects to be created in test code and it also supports .NET Core.

Moq allows the manipulation of mock objects in many ways, including setting mock methods to return specific values, setting up properties, and matching specific arguments when the thing being tested calls the mock object.

For example, the following code shows a class that requires a constructor dependency to be able to operate:

using System;

namespace Domain
{
    public interface IThingDependency
    {
        string JoinUpper(string a, string b);
        int Meaning { get; }
    }

    // "Real" implementation
    public class ThingDependency : IThingDependency
    {
        public string JoinUpper(string a, string b)
        {
            throw new NotImplementedException();
        }

        public int Meaning => throw new NotImplementedException();
    }

    // Class we want to test in isolation of ThingDependency
    public class ThingBeingTested
    {
        private readonly IThingDependency _thingDependency;

        public string FirstName { get; set; }
        public string LastName { get; set; }

        public ThingBeingTested(IThingDependency thingDependency)
        {
            _thingDependency = thingDependency;
        }

        public string X()
        {
            var fullName = _thingDependency.JoinUpper(FirstName, LastName);

            return $"{fullName} = {_thingDependency.Meaning}";
        }
    }
}

Without a mock object, to write a test we could use the real ThingDependency:

[Fact]
public void TestUsingRealDependency()
{
    var sut = new ThingBeingTested(new ThingDependency());

    // test code
}

To isolate the ThingBeingTested from the rest of the system, Moq can create a mock version of an IThingDependency:

[Fact]
public void TestUsingMockDependency()
{
    // create mock version
    var mockDependency = new Mock<IThingDependency>();

    // set up mock version's method
    mockDependency.Setup(x => x.JoinUpper(It.IsAny<string>(), It.IsAny<string>()))
                  .Returns("A B");

    // set up mock version's property
    mockDependency.Setup(x => x.Meaning)
                  .Returns(42);

    // create thing being tested with a mock dependency
    var sut = new ThingBeingTested(mockDependency.Object);

    var result = sut.X();

    Assert.Equal("A B = 42", result);
}

In the preceding code, the Setup() method is used to tell the mock how to behave when it is called by the ThingBeingTested.

Moq can also be used to test the correct interactions are occurring between the ThingBeingTested and the IThingDependency:

[Fact]
public void TestUsingMockDependencyUsingInteractionVerification()
{
    // create mock version
    var mockDependency = new Mock<IThingDependency>();

    // create thing being tested with a mock dependency
    var sut = new ThingBeingTested(mockDependency.Object)
    {
        FirstName = "Sarah",
        LastName = "Smith"
    };

    sut.X();

    // Assert that the JoinUpper method was called with Sarah Smith
    mockDependency.Verify(x => x.JoinUpper("Sarah", "Smith"), Times.Once);

    // Assert that the Meaning property was accessed once
    mockDependency.Verify(x => x.Meaning, Times.Once);
}

In the preceding code, the Verify method is used to check that the mock JoinUpper method is being called exactly once with the values “Sarah” and “Smith”. The test code is also expecting the method to be called exactly once.

Moq can be used to test in isolation other parts of applications such as ASP.NET Core MVC controllers, where the controller requires a dependency (such as an IFooRepository):

[Fact]
public void ContollerTest()
{            
    var mockDependency = new Mock<IFooRepository>();

    var sut = new HomeController(mockDependency.Object);
    
    // test code
}

To learn more about using Moq to create/configure/use mock objects check out my Mocking in .NET Core Unit Tests with Moq: Getting Started Pluralsight course.

To learn how to get started testing ASP.NET Core MVC applications check out my ASP.NET Core MVC Testing Fundamentals Pluralsight course.

SHARE:

New Pluralsight Course: Testing C# Code in Production with Scientist.NET

My latest Pluralsight course is now available for viewing. It demonstrates how to use the Scientist.NET library to execute candidate code in production alongside the existing production code. This allows the new candidate code to be additionally verified as able to work in production (for example with production data that may be of variable quality) and offers an additional check in addition to automated tests that have been executed in the development/QA environment.

From the course description: “Errors in production code can be costly to fix, more stressful for the development team, and frustrating for the end-user. In this course, Testing C# Code in Production with Scientist.NET, you will get to see how Scientist.NET allows sections of the application to be changed more safely by running both the existing code alongside the new code in production. You'll begin with an introduction to Scientist.NET before installing it and writing your first experiment. Next, you'll learn how to customize the configuration of experiments. Finally, you'll learn how to publish experiment data to SQL Server and analyze experiment results…”

You can check out the new course here.

SHARE: