Mocking with FeatureToggle

I was asked a question on Twitter so I thought I’d write it up here.

When using the FeatureToggle library you may have some some code that behaves differently if a toggle is enabled.

When writing a test, you can create a mock IFeatureToggle and set it up to be enabled (or not) and then assert the result is as expected.

The following code show a simple console app that has an OptionsConsoleWriter.Generate method that uses a toggle to output a printing feature option:

using static System.Console;
using System.Text;
using FeatureToggle.Toggles;
using FeatureToggle.Core;

namespace ConsoleApp1
{
    public class Printing : SimpleFeatureToggle {}

    public class OptionsConsoleWriter
    {
        public string Generate(IFeatureToggle printingToggle)
        {
            var sb = new StringBuilder();

            sb.AppendLine("Options:");
            sb.AppendLine("(e)xport");
            sb.AppendLine("(s)ave");

            if (printingToggle.FeatureEnabled)
            {
                sb.AppendLine("(p)rinting");
            }

            return sb.ToString();
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            Printing printingToggle = new Printing();

            string options = new OptionsConsoleWriter().Generate(printingToggle);

            Write(options);            

            ReadLine();
        }
    }
}

To write a couple of simple tests for this method, you can use a mocking framework such as Moq to generate a mocked IFeatureToggle and pass it to the Generate method:

using Xunit;
using Moq;
using FeatureToggle.Core;
using ConsoleApp1;

namespace ClassLibrary1.Tests
{
    public class OptionsConsoleWriterTests
    {
        [Fact]
        public void ShouldGeneratePrintingOption()
        {
            var sut = new OptionsConsoleWriter();

            var mockPrintingToggle = new Mock<IFeatureToggle>();
            mockPrintingToggle.SetupGet(x => x.FeatureEnabled)
                              .Returns(true);

            string options = sut.Generate(mockPrintingToggle.Object);

            Assert.Contains("(p)rinting", options);
        }

        [Fact]
        public void ShouldNotGeneratePrintingOption()
        {
            var sut = new OptionsConsoleWriter();

            var mockPrintingToggle = new Mock<IFeatureToggle>();
            mockPrintingToggle.SetupGet(x => x.FeatureEnabled)
                              .Returns(false);

            string options = sut.Generate(mockPrintingToggle.Object);

            Assert.DoesNotContain("(p)rinting", options);
        }
    }
}

Grouping and Filtering Tests in Visual Studio Test Explorer

One way to run automated tests is to use Visual Studio’s Test Explorer. Test Explorer can be found under the Test –> Windows –> Test Explorer menu items.

In this article we’ll look at how to manage the list of tests using grouping and also how to specify custom search filter expressions.

Two test projects in Visual Studio solution

Grouping Tests

There are a number of ways to group tests in Test Explorer, at the highest structural level we can group by project.

To select a group by method, click the drop down arrow as show in the following screenshot:

Selecting a grouping in Visual Studio Test Explorer

With the grouping set to Project, the test list looks as follows:

image

The next structural grouping is Class:

Grouping by test class

The final structural grouping is by Namespace:

Grouping by namespace

There are a number of non-structural groupings.

Group by Duration:

Group by duration

Group by Outcome:

Group by outcome

…and group by Traits:

Grouping by traits

Filtering Tests

Custom filters can also be applied.

For example by file path:

Filtering tests by file path

Other search examples include:

  • Trait:"Smoke Test"
  • Message:"System.Exception"
  • Class1
  • Outcome:"Passed"
  • Outcome:"Failed"

Subsets can also be excluded by prefixing the type of filter with a -. For example to show all tests in Class1 except failed tests: Class:"TestClass1" -Outcome:"Passed".

Getting Started Testing .NET Core Code with xUnit.net

xUnit.net is a testing framework that can be used to write automated tests for .NET (full) framework and also .NET Core.

To get started, first create a .NET Core application, in the following example a .NET Core console app.

Creating a .NET core console project

A testing project can now be added to the solution:

Adding an xUnit test project in Visual Studio 2017

This test project will come pre-configured with the relevant NuGet packages installed to start writing test code, though you may want to update the pre-configured packages to the newest NuGet versions.

The xUnit Test Project template will also create the following default test class:

using System;
using Xunit;

namespace ConsoleCalculator.Tests
{
    public class UnitTest1
    {
        [Fact]
        public void Test1()
        {

        }
    }
}

Notice in the preceding code, the Test1 method is decorated with the [Fact] attribute. This is an xUnit.net attribute that tells a test runner that it should execute the method, treat it as a test, and report on if the test passed or not.

Next add a project reference from the test project to the project that contains the code that is to be tested, this gives the test project access to the production code.

In the production project, the following class can be added:

namespace ConsoleCalculator
{
    public class Calculator
    {
        public int Add(int a, int b)
        {            
            return a + b;
        }
    }
}

Now the test class can be renamed (for example to “CalculatorTests”) and the test method changed to create a test:

using Xunit;

namespace ConsoleCalculator.Tests
{
    public class CalculatorTests
    {
        [Fact]
        public void ShouldAddTwoNumbers()
        {
            Calculator calculator = new Calculator();

            int result = calculator.Add(7, 3);

            Assert.Equal(10, result);
        }
    }
}

In the preceding code, once again the [Fact] attribute is being used, then the thing being tested is created (the Calculator class instance). The next step is to perform some kind of action on the thing being tested, in this example calling the Add method. The final step is to signal to the test runner if the test has passed or not, this is done by using one of the many xUnit.net Assert methods; in the preceding code the Assert.Equal method is being used. The first parameter is the expected value of 10, the second parameter is the actual value produced by the code being tested. So if  result is 10 the test will pass, otherwise it will fail.

One way to execute tests is to use Visual Studio’s Test Explorer which can be found under the Test –> Windows –> Test Explorer menu item. Once the test project is built, the test will show up and can be executed as the following screenshot shows:

Running xUnit tests in Visual Studio Test Explorer

To learn more about how to get started testing .NET Core code check out my Testing .NET Core Code with xUnit.net: Getting Started Pluralsight course or check out the docs.

Testing Automation: The Big Picture

It’s often useful to take a step back and look at the bigger picture, this is true in different aspects of life such as health or wealth or relationships, and is also true of software development.

When it comes to creating automated tests (as with other aspects of software development) dogmatism and absolutist schools of though can exist.

As with all things, the decision to write tests (and how many tests, what type of tests, test coverage aims, etc.) ultimately should boil down to one question: do they add value to what you are doing?

To be clear, I absolutely believe in the creation of automated tests in many cases, however it is good to not be dogmatic. For example if there is a niche market that is ripe for capitalizing on, and time-to-market is the most important thing to capture this market, then an extensive suite of automated tests may slow down getting to that initial release. This of course assumes that the potential market will have some tolerance for software defects. It also depends on what the product is; medical life-critical software is probably going to have a higher quality requirement than a social media app for example. This can be a trade-off however with shorter term delivery speeds being quicker but at the expense of delivery speed in the long term, if you’re overwhelmed fixing production outages you have very little time to add new features/value.

Another aspect to consider is that of risk. What are the risks associated with defects in the software making their way into production? Different features/application areas may also have different risk profiles; for example  the “share on social media” feature may not be deemed as important as a working shopping cart. It’s also important to remember that risk is not just monetary, in the previous example a broken “share on social media” feature may bring the business into disrepute, aka “reputational risk”.

When it comes to the myriad of different types of tests (unit, integration, subcutaneous, etc.) the testing pyramid is an oft-quoted model of how many of each type of tests to have in the test suite. While the testing pyramid may be of great help for someone new to automated testing to help them learn and navigate their initial steps, as experience grows the model may no longer be optimal to some of the projects that are being worked on. Beyond the testing pyramid the different aspects of test types can be considered such as execution speed, breadth/depth, reliability/brittleness etc.

Automated tests also do not exist in and of themselves, they are part of a bigger set of processes/considerations such as pair programming, code reviews, good management, well-understood requirements, good environment management/DevOps, etc.

If you want to take a step back and look at the big picture, or know a developer or manager who wants to understand the trade-offs/benefits check out my Testing Automation: The Big Picture Pluralsight course.

Testing ASP.NET Core Controllers in Isolation with Mock Objects and Moq

In previous posts we saw how to get started testing ASP.NET Core MVC controllers and also how to use the Moq mocking library in .NET Core tests.

If there is code in controllers that needs testing, but the controller has a dependency, for example passed into the constructor, it may not make sense to use the real version of the dependency. In these cases Moq can be used to create a mock version of the dependency and pass it to the controller that needs testing.

As an example suppose we have the following controller code:

public class HomeController : Controller
{
    private readonly ISmsGateway _smsGateway;

    public HomeController(ISmsGateway smsGateway)
    {
        _smsGateway = smsGateway;
    }

    [HttpPost]
    [ValidateAntiForgeryToken]
    public IActionResult Send(SendSmsRequest request)
    {
        if (ModelState.IsValid)
        {
            var sendReceipt = _smsGateway.Send(request.PhoneNumber, request.Message);

            return Ok(sendReceipt);
        }

        return BadRequest();
    }
}

In the preceding code, the controller takes an ISmsGateway dependency as a constructor parameter. This dependency is later used in the the Send() method.

After installing Moq a mock SMS gateway can be created. Once created, Moq’s Setup() method can be used to determine what happens when the controller calls the mocked Send() method as the following code demonstrates:

[Fact]
public void ShouldSendOk()
{
    SendSmsRequest sendSmsRequest = new SendSmsRequest
    {
        PhoneNumber = "42",
        Message = "Hello"
    };

    Guid expectedSendReceipt = Guid.NewGuid();

    var mockSmsGateway = new Mock<ISmsGateway>();
    
    mockSmsGateway.Setup(x => x.Send(sendSmsRequest.PhoneNumber, sendSmsRequest.Message))
                  .Returns(expectedSendReceipt);

    var sut = new HomeController(mockSmsGateway.Object);
    
    IActionResult result = sut.Send(sendSmsRequest);

    var okObjectResult = Assert.IsType<OkObjectResult>(result);

    Assert.Equal(expectedSendReceipt, okObjectResult.Value);
}

We may also want to test that if there is a model binding error, then  no message is sent via the SMS gateway. The follow test code shows the use of the AddModelError() method to simulate an error, and the use of Moq’s Verify() method to check that the gateway’s Send() method was never called:

[Fact]
public void ShouldNotSendWhenModelError()
{
    SendSmsRequest sendSmsRequest = new SendSmsRequest
    {
        PhoneNumber = "42",
        Message = "Hello"
    };

    var mockSmsGateway = new Mock<ISmsGateway>();

    var sut = new HomeController(mockSmsGateway.Object);
    sut.ModelState.AddModelError("Simulated", "Model error");

    sut.Send(sendSmsRequest);

    mockSmsGateway.Verify(x => x.Send(It.IsAny<string>(), It.IsAny<string>()), Times.Never);
}

To learn how to get started testing ASP.NET Core MVC applications check out my ASP.NET Core MVC Testing Fundamentals Pluralsight course.

Testing ASP.NET Core MVC Controllers: Getting Started

When writing ASP.NET Core MVC web applications, you may want to test that controller actions behave in the expected way, for example that the action returns the correct result type (e.g. a ViewResult) or that the action behaves as expected when the model state is invalid.

To get started writing controller tests, first add a new .NET Core xUnit test project to the solution. This will create the test project along with requried xUnit.net NuGet packages. It will also add a default test class "UnitTest1.cs":

using System;
using Xunit;

namespace WebApplication1.Tests
{
    public class UnitTest1
    {
        [Fact]
        public void Test1()
        {
        }
    }
}

In the preceding code, notice the xUnit.net [Fact] attribute that marks the Test1 method as a test that should be executed by the test runner. One way to run tests in Visual Studio is to use the built-in Test Explorer which can be accessed via the menus: Test->Windows->Test Explorer.

If you build the project you will see the default test shown in the Test Explorer window:

Visual Studio Test Explorer

Adding a Controller Test

First, to get access to the controllers in the ASP.NET Core MVC application, add a reference to the web project from the test project. An instance of a controller can now be created in the test method:

var sut = new WebApplication1.Controllers.HomeController();

We can now call methods (actions) on the controller and verify the results. As a simple example, we can check that the Index method result is a view:

[Fact]
public void Test1()
{
    HomeController sut = new WebApplication1.Controllers.HomeController();

    IActionResult result = sut.Index();

    Assert.IsType<ViewResult>(result);
}

There are many different ways to test the results of controllers, including the ability to simulate model errors or using Moq mock objects as controller constructor dependencies.

The following code shows an excerpt from a controller and a test that examines the view's model that was returned:

public class PersonViewModel
{
    public string Name { get; set; }
}

public IActionResult Person()
{
    PersonViewModel viewModel = new PersonViewModel
    {
        Name = "Amrit"
    };

    return View(viewModel);
}
[Fact]
public void Test2()
{
    HomeController sut = new WebApplication1.Controllers.HomeController();

    IActionResult result = sut.Person();

    ViewResult viewResult = Assert.IsType<ViewResult>(result);

    PersonViewModel model = Assert.IsType<PersonViewModel>(viewResult.Model);

    Assert.Equal("Amrit", model.Name);
}

To learn how to get started testing ASP.NET Core MVC applications check out my ASP.NET Core MVC Testing Fundamentals Pluralsight course.

Mocking in .NET Core Tests with Moq

When writing automated tests it is sometimes useful to isolate the thing(s) being tested from other parts of the system. These ‘other’ parts may still need to be provided, and sometimes the real versions are too hard or cumbersome to use. In these instances “mocked” versions can be created and used.

A mock version of something is an object that can act like the real thing but can be controlled in test code.

Moq (pronounced “mok u” or “mock”) is a library available on NuGet that allows mock objects to be created in test code and it also supports .NET Core.

Moq allows the manipulation of mock objects in many ways, including setting mock methods to return specific values, setting up properties, and matching specific arguments when the thing being tested calls the mock object.

For example, the following code shows a class that requires a constructor dependency to be able to operate:

using System;

namespace Domain
{
    public interface IThingDependency
    {
        string JoinUpper(string a, string b);
        int Meaning { get; }
    }

    // "Real" implementation
    public class ThingDependency : IThingDependency
    {
        public string JoinUpper(string a, string b)
        {
            throw new NotImplementedException();
        }

        public int Meaning => throw new NotImplementedException();
    }

    // Class we want to test in isolation of ThingDependency
    public class ThingBeingTested
    {
        private readonly IThingDependency _thingDependency;

        public string FirstName { get; set; }
        public string LastName { get; set; }

        public ThingBeingTested(IThingDependency thingDependency)
        {
            _thingDependency = thingDependency;
        }

        public string X()
        {
            var fullName = _thingDependency.JoinUpper(FirstName, LastName);

            return $"{fullName} = {_thingDependency.Meaning}";
        }
    }
}

Without a mock object, to write a test we could use the real ThingDependency:

[Fact]
public void TestUsingRealDependency()
{
    var sut = new ThingBeingTested(new ThingDependency());

    // test code
}

To isolate the ThingBeingTested from the rest of the system, Moq can create a mock version of an IThingDependency:

[Fact]
public void TestUsingMockDependency()
{
    // create mock version
    var mockDependency = new Mock<IThingDependency>();

    // set up mock version's method
    mockDependency.Setup(x => x.JoinUpper(It.IsAny<string>(), It.IsAny<string>()))
                  .Returns("A B");

    // set up mock version's property
    mockDependency.Setup(x => x.Meaning)
                  .Returns(42);

    // create thing being tested with a mock dependency
    var sut = new ThingBeingTested(mockDependency.Object);

    var result = sut.X();

    Assert.Equal("A B = 42", result);
}

In the preceding code, the Setup() method is used to tell the mock how to behave when it is called by the ThingBeingTested.

Moq can also be used to test the correct interactions are occurring between the ThingBeingTested and the IThingDependency:

[Fact]
public void TestUsingMockDependencyUsingInteractionVerification()
{
    // create mock version
    var mockDependency = new Mock<IThingDependency>();

    // create thing being tested with a mock dependency
    var sut = new ThingBeingTested(mockDependency.Object)
    {
        FirstName = "Sarah",
        LastName = "Smith"
    };

    sut.X();

    // Assert that the JoinUpper method was called with Sarah Smith
    mockDependency.Verify(x => x.JoinUpper("Sarah", "Smith"), Times.Once);

    // Assert that the Meaning property was accessed once
    mockDependency.Verify(x => x.Meaning, Times.Once);
}

In the preceding code, the Verify method is used to check that the mock JoinUpper method is being called exactly once with the values “Sarah” and “Smith”. The test code is also expecting the method to be called exactly once.

Moq can be used to test in isolation other parts of applications such as ASP.NET Core MVC controllers, where the controller requires a dependency (such as an IFooRepository):

[Fact]
public void ContollerTest()
{            
    var mockDependency = new Mock<IFooRepository>();

    var sut = new HomeController(mockDependency.Object);
    
    // test code
}

Check out the quickstart for more information on the features of Moq.

To learn how to get started testing ASP.NET Core MVC check out my ASP.NET Core MVC Testing Fundamentals Pluralsight course.

New Pluralsight Course: Testing C# Code in Production with Scientist.NET

My latest Pluralsight course is now available for viewing. It demonstrates how to use the Scientist.NET library to execute candidate code in production alongside the existing production code. This allows the new candidate code to be additionally verified as able to work in production (for example with production data that may be of variable quality) and offers an additional check in addition to automated tests that have been executed in the development/QA environment.

From the course description: “Errors in production code can be costly to fix, more stressful for the development team, and frustrating for the end-user. In this course, Testing C# Code in Production with Scientist.NET, you will get to see how Scientist.NET allows sections of the application to be changed more safely by running both the existing code alongside the new code in production. You'll begin with an introduction to Scientist.NET before installing it and writing your first experiment. Next, you'll learn how to customize the configuration of experiments. Finally, you'll learn how to publish experiment data to SQL Server and analyze experiment results…”

You can check out the new course here.

Refactoring Production Code With Experiments and Scientist.NET

When refactoring a part of an application we can use the existing tests to give a level of confidence that the refactored code still produces the same result, i.e. the existing tests still pass with the new implementations.

A system that has been in production use for some time is likely to have amassed a lot of data that is flowing through the “legacy” code. This means that although the existing tests may still pass, when used in production the new refactored code may produce errors or unexpected results.

It would be helpful as an experiment to run the existing legacy code alongside the new code to see if the results differ, but still continue to use the result of the existing legacy code. Scientist.NET allows exactly this kind of experimentation to take place.

Scientist.NET is a port of the Scientist library for Ruby. It is currently in pre-release and has a NuGet package we can use.

An Example Experiment

Suppose there is an interface as shown in the following code that describes the ability to sum a list if integer values and return the result.

interface ISequenceSummer
{
    int Sum(int numberOfTerms);
}

This interface is currently implemented in the legacy code as follows:

class SequenceSummer : ISequenceSummer
{
    public int Sum(int numberOfTerms)
    {
        var terms = new int[numberOfTerms];


        // generate sequence of terms
        var currentTerm = 0;
        for (int i = 0; i < terms.Length; i++)
        {
            terms[i] = currentTerm;
            currentTerm++;
        }


        // Sum
        int sum = 0;
        for (int i = 0; i < terms.Length; i++)
        {
            sum += terms[i];
        }

        return sum;
    }
}

As part of the refactoring of the legacy code, this implementation is to be replaced with a version that utilizes LINQ as shown in the following code:

class SequenceSummerLinq : ISequenceSummer
{
    public int Sum(int numberOfTerms)
    {
        // generate sequence of terms
        var terms = Enumerable.Range(0, 5).ToArray();
            
        // sum
        return terms.Sum();
    }
}

After installing the Scientist.NET NuGet package, an experiment can be created using the following code:

int result;            

result = Scientist.Science<int>("sequence sum", experiment =>
{
    experiment.Use(() => new SequenceSummer().Sum(5)); // current production method

    experiment.Try("Candidate using LINQ", () => new SequenceSummerLinq().Sum(5)); // new proposed production method

}); // return the control value (result from SequenceSummer)

This code will run the .Use(…) code that contains the existing legacy implementation. It will also run the .Try(…) code that contains the new implementation. Scientist.NET will store both results for reporting on and then return the result from the .Use(…) code for use by the rest of the program. This allows any differences to be reported on but without actually changing the implementation of the production code. At some point in the future, if the results of the legacy code (the control) match that of the new code (the candidate), the refactoring can be completed by removing the old implementation (and the experiment code) and simply calling the new implementation.

To get the results of the experiment, a reporter can be written and configured. The following code shows a custom reporter that simply reports to the Console:

public class ConsoleResultPublisher : IResultPublisher
{
    public Task Publish<T>(Result<T> result)
    {
        Console.ForegroundColor = result.Mismatched ? ConsoleColor.Red : ConsoleColor.Green;

        Console.WriteLine($"Experiment name '{result.ExperimentName}'");
        Console.WriteLine($"Result: {(result.Matched ? "Control value MATCHED candidate value" : "Control value DID NOT MATCH candidate value")}");
        Console.WriteLine($"Control value: {result.Control.Value}");
        Console.WriteLine($"Control execution duration: {result.Control.Duration}");
        foreach (var observation in result.Candidates)
        {
            Console.WriteLine($"Candidate name: {observation.Name}");
            Console.WriteLine($"Candidate value: {observation.Value}");
            Console.WriteLine($"Candidate execution duration: {observation.Duration}");
        }

        if (result.Mismatched)
        {
            // saved mismatched experiments to event log, file, database, etc for alerting/comparison
        }

        Console.ForegroundColor = ConsoleColor.White;

        return Task.FromResult(0);
    }  
}

To plug this in, before the experiment code is executed:

Scientist.ResultPublisher = new ConsoleResultPublisher();

The output of the experiment (and the custom reporter) look like the following screenshot:

screenshot of console application using Scentist.NET with custom reporter

To learn more about Scientist.NET check out my Pluralsight course: Testing C# Code in Production with Scientist.NET.

New Pluralsight Course: Automated Business Readable Web Tests with Selenium and SpecFlow

SpecFlow is a tool that can translate natural language scenarios (e.g. writing in English or other spoken languages) into test code. This can allow business people, users, or other stakeholders to verify that the correct features are being built.

Selenium is a tool that allows test code (multiple programming languages supported) to automated a web browser. This allows the creation of automated UI tests that operate the web application as if an end user where doing it; for example clicking buttons and typing text into input boxes.

My new Pluralsight course shows how to integrate these two tools.

The course is organized into four modules:

  1. Introduction to Business Readable Web Testing
  2. Getting Started with Selenium
  3. Adding Business Readability with SpecFlow
  4. Creating More Maintainable Web Automation

If you’re new to SpecFlow I suggest watching this course first before moving on to Automated Business Readable Web Tests with Selenium and SpecFlow.