New Pluralsight Course - Better .NET Unit Tests with AutoFixture

My new Pluralsight testing course was just released: Better .NET Unit Tests with AutoFixture: Get Started.

AutoFixture is an open source library that allows the simplification of automated .NET tests. It allows for the creation of “anonymous” test data. This is data that is required for the test to execute, but where the exact values are unimportant.

AutoFixture is not dependent upon any specific testing framework, so can be used with xUnit.net, NUnit, MSTest, etc. There are some additional extensions for specific testing frameworks such as xUnit.net that allow further reductions in unit test setup code.

For more info check out the GitHub site or the Pluralsight course.

If you like AutoFixture be sure to give open source thanks to Mark Seemann.

SHARE:

Using AutoFixture To Generate Anonymous Test Data in Web UI Automation With BDDfy and Seleno

I’m currently working on an AutoFixture Pluralsight course and it got me thinking about using it to generate anonymous test data when writing automated UI tests.

The basic premise is that in addition to using AutoFixture to generate unit test data, it can also be used to populate UI elements where the specific data values are unimportant.

If you are unfamiliar with BDDfy and Seleno, they are part of the TestStack project.

BDDfy allows tests to be written and to produce business readable documentation. Seleno allows the automation of web browsers using Selenium and strongly-typed page object models.

Example Scenario

Imagine that we have an (ASP.NET MVC) web site that allows the addition of members of the royal family.

image

If we wanted to test different king/queen names (but didn’t care about the Regnal number) we could start off by defining some strongly typed (Seleno) page object models:

public class HomePage : Page<AddRoyaltyModel>
{
    public CreatedPage CreateRoyalty(AddRoyaltyModel royal)
    {
        Input.Model(royal);

        return Navigate.To<CreatedPage>(By.Id("Create"));
    }
}

public class CreatedPage : Page
{
}

 

(all the examples in this post are fairly quick-and-dirty to demonstrate the AutoFixture involvement)

 

Next we can write some example in BDDfy (using xUnit.net as the test framework):

public class AddRoyaltyTests
{
    private HomePage _home;
    private CreatedPage _confirmationPage;
    public string Name { get; set; }
    public string Number { get; set; }

    [Fact]
    public void ShouldAddRoyaltiesWithDifferentNames()
    {
        this.Given(x => GivenIAmStartingANewRoyalAddition())
            .And("And I have entered <name> as the royalty name")
            .And("And I have entered <number> as the regnal number")
            .When(x => WhenIChooseToAddTheNewRoyal())
            .Then(x => ThenIShouldSeeAConfirmationOfTheNewRoyalHavingBeenAdded())
            .WithExamples(new ExampleTable("name", "number")
                          {
                              {"Richard", "I"},
                              {"Henry", "I"},
                              {"Elizabeth", "I"}
                          })
            .BDDfy();            
    }



    private void GivenIAmStartingANewRoyalAddition()
    {
        _home = Host.Instance.NavigateToInitialPage<HomeController, HomePage>(x => x.Index());
    }        

    private void WhenIChooseToAddTheNewRoyal()
    {
        var royal = new AddRoyaltyModel
        {
            Name = this.Name,
            RegnalRomanNumeral = this.Number
        };

        _confirmationPage = _home.CreateRoyalty(royal);
    }

    private void ThenIShouldSeeAConfirmationOfTheNewRoyalHavingBeenAdded()
    {
        Assert.Equal("Created ok", _confirmationPage.Title);
    }
}

Notice in the preceding code that even though we don’t care about the Regnal number we are still supplying it in the examples. We could just set it manually to a hardcoded value, and in this simple example that might be ok, but if we had a form with many fields then this  will introduce extra work and may make the test less “refactor-safe”.

The following code shows the removal of the Regnal name/number:

public class AddRoyaltyTestsUsingAutoFixture
{
    private HomePage _home;
    private CreatedPage _confirmationPage;
    public string Name { get; set; }

    [Fact]
    public void ShouldAddRoyaltiesWithDifferentNames()
    {
        this.Given(x => GivenIAmStartingANewRoyalAddition())
            .And("And I have entered <name> as the royalty name")
            .When(x => WhenIChooseToAddTheNewRoyal())
            .Then(x => ThenIShouldSeeAConfirmationOfTheNewRoyalHavingBeenAdded())
            .WithExamples(new ExampleTable("name")
                          {
                              "Richard",
                              "Henry",
                              "Elizabeth"
                          })
            .BDDfy();            
    }

    private void GivenIAmStartingANewRoyalAddition()
    {
        _home = Host.Instance.NavigateToInitialPage<HomeController, HomePage>(x => x.Index());
    }        

    private void WhenIChooseToAddTheNewRoyal()
    {
        var fixture = new Fixture();

        // Use AutoFixture to create anonymous data for all properties except
        // name which is set to the BDDfy example value
        var royal = fixture.Build<AddRoyaltyModel>()
            .With(x => x.Name, this.Name)
            .Create();

        _confirmationPage = _home.CreateRoyalty(royal);
    }

    private void ThenIShouldSeeAConfirmationOfTheNewRoyalHavingBeenAdded()
    {
        Assert.Equal("Created ok", _confirmationPage.Title);
    }
}

In this version, AutoFixture’s Build method is being used to automatically generate test data for all the fields, except the royal name which is set to the name(s) specified in the BDDfy examples.

Running this test results in the following automation:

SelenoAutofixture

 

And produces the following HTML BDDfy report:

image

SHARE:

Fixie - A Convention-based .NET Testing Framework

Fixie is a relative newcomer to the .NET testing framework space. It throws away the idea of marking elements of test code with attributes in favour of a conventions based approach.

At a high level, what this means is simply naming things in test projects following a defined default (customized conventions are also supported) convention.

A Fixie test class with a single test inside it would look like the following, notice the lack of attributes and even using statements.

namespace MyApplication.Tests
{
    public class SomeClassTests
    {
        public void ShouldDoSomething()
        {
        }
    }
}

After building the test will now show up in Visual Studio Test Explorer as the following screenshot shows.

Visual Studio Test Explorer showing Fixie Test

Fixie knows this is a test because it matches the default conventions that come out of the box.

Fixie knows that this is a test class because it ends with “Tests” and it knows that the method is a test because it’s a public void method.

If these default conventions are not suitable for your project you can create your own custom conventions to customise the test discovery. There’s also a lot more to custom conventions, such as customising the test execution lifecycle and creating data-driven tests.

To learn more about Fixie, check out the docs or my Introduction to Fixie Pluralsight course.

SHARE:

Arrange Act Assert Comments in Tests

The Arrange, Act, Assert (AAA) pattern is used in tests to help organise and clarify test code. It can also help to spot potential problems in test code if these three phases don’t seem to exist.

The Arrange phase is where the thing we’re testing (the system under test) is put into a known beginning state.

The Act phase is where we perform some action on the thing being tested.

The Assert phase is where we check that the results of the Act phase are as expected.

When first learning to use the AAA pattern, it can be helpful to start with 3 comments:

public void ShouldAddNumbers()
{
    // Arrange
    
    // Act
    
    // Assert

}

 

These comments can help to focus on making sure there are three distinct phases.

While these comments are useful when trying to learn (or teach) AAA, they should not be needed in the final version of the test code.

Test code should ideally be as good as production code. One of the things that qualifies code as “clean” is the absence of useless/pointless comments.

Test code should be easy to read, it should not need the AAA comments to be left in to be able to be understood.

If you are using the comment-first approach to help you get started and learn the AAA approach that’s all well and good. However, once the test is written, these comments should usually be removed before the code is committed to source control.

Once the AAA comments are removed, it should still be clear what the flow of the test is. If it is not then the test code may need some changes to improve the readability.

SHARE:

New Pluralsight Course - Introduction to .NET Testing with NUnit

If you are just dipping your toe in the water when it comes to testing .NET applications it can be a bit confusing. In addition to learning how and what to write test, you also have to learn a testing framework such as MSTest, xUnit.net, NUnit, etc.

My new beginner Pluralsight course helps you to get started with testing in .NET and how to use the NUnit testing framework.

Watch the course at the above link or get to it from my Pluralsight author page.

SHARE:

New Pluralsight Course - Automated ASP.NET MVC Testing: End to End

My new Pluralsight course Automated ASP.NET MVC Testing: End to End was just published.

The course looks at tools and techniques to be able to tests models, controller actions, Razor view rendering logic, and implement automated functional UI browser testing.

The course wraps up by showing how to design and implement a continuous integration build in TeamCity that runs the tests created during the course.

Check it out the full table of contents.

SHARE:

Beyond Simple Asserts with ApprovalTests

In a test, we are often testing (asserting) individual items such as an (int) age is correct or a string matches an expected value.

If we are practicing test-first development we’ll write our asserts first.

Approval tests allow us to go beyond simple asserts.

What if the thing we’re checking is not a simple value, for example that a pie chart image matches the input data? Or what if we want to use our human judgement to decide when something looks correct, something that is hard to codify in one or more basic asserts?

ApprovalTests for .NET can be install via NuGet. Once installed, it gives us a whole new world when it comes to checking the output of code.

For example, say we are developing a class to represent a stickman. We want to be able to tell an instance to raise left arm or raise right leg for example.

Example of Using Approval Tests

So lets start with a test:

[Fact]
[UseReporter(typeof(DiffReporter))]
public void ShouldHaveDefaultPosture()
{
    var sut = new StickMan();

    Approvals.Verify(sut);
}

And an empty StickMan:

public class StickMan
{        
}

Here we’re using xUnit.net (the [Fact] attribute) but you could be using NUnit for example.

The first thing to notice here is there is no traditional Assert method, instead we’re using Approval Tests to verify the state of the system under test (sut).

The other think to notice is the [UseReporter] attribute that tells Approval Tests to use a diff tool to display errors when a test fails.

If we run this test, we’ll get a diff tool opened:

More...

SHARE:

In Process Http Server for Integration Test Faking with Owin, Katana, and WebAPI

Sometimes when integration testing we need an HTTP endpoint. We can set one up in our solution (for example a Web Api project) and make sure it’s running before we execute our integration tests but it can be painful. Also consider when running on a build server, the web site will need to be provisioned, etc.

For isolated integration tests that require some HTTP endpoint we can create an HTTP endpoint from within our test code that doesn’t require a separate web project. This means that when tests run they have no external dependency in the sense of a “real” website being hosted somewhere – even if that somewhere is on the local dev machine. Another advantage of the approach outlined below is that because it’s all in-process it’s also faster than actually going through the network stack.

The System Under Test

We are going to test the following class that calls a service to change a string to uppercase:

using System;
using System.Net;

namespace ClassLibrary
{
    public class ToUpperServiceGateway
    {
        private readonly string _serviceEndpoint;

        public ToUpperServiceGateway(string serviceEndpoint)
        {
            _serviceEndpoint = serviceEndpoint;
        }

        public string MakeUppercase(string lowercase)
        {
            using (var wc = new WebClient())
            {
                return wc.DownloadString(_serviceEndpoint + "/" + lowercase);
            }
        }
    }
}

This code uses WebClient to call a web service. In the integration tests we want to spin up a website that will return a mock value for example.

Writing the Initial Test Code

So an initial test could look somethine like:

using Xunit;

namespace ClassLibrary.IntegrationTests
{
    public class UpperCaseTests
    {
        [Fact]
        public void ShouldGetUppercase()
        {
            var sut = new ToUpperServiceGateway("http://localhost:8084/api/toupper");

            var upper = sut.MakeUppercase("hello");

            Assert.Equal("HELLO", upper);            
        }
    }
}

If we run this we get “System.Net.WebException Unable to connect to the remote server” as we’d expect as there’s no website at the address.

Creating an In-Process Web API Endpoint

First off, in the test project install a few of NuGets: Microsoft.AspNet.WebApi.OwinSelfHost and Microsoft.Owin.Testing.

The first thing is to create a Web API controller, again in the test project, to fake out a real service:

using System.Web.Http;

namespace ClassLibrary.IntegrationTests
{
    public class ToUpperController : ApiController
    {
        public string Get(string lower)
        {            
            return "FAKE"; // fake value
        }
    }
}

Here, regardless of what’s passed in we’re just returning a fake value.

The next task is to create some configuration for Owin setting up our test controller:

using System.Web.Http;
using Owin;

namespace ClassLibrary.IntegrationTests
{
    public class Startup
    {
        public void Configuration(IAppBuilder appBuilder)
        {
            var config = new HttpConfiguration();

            config.Routes.MapHttpRoute("DefaultApi", "api/{controller}/{lower}");

            appBuilder.UseWebApi(config);
        }
    }
}

We can now refactor the test to create an in-memory Web API controller that can respond to our SUT:

using Microsoft.Owin.Hosting;
using Xunit;

namespace ClassLibrary.IntegrationTests
{
    public class UpperCaseTests
    {
        [Fact]
        public void ShouldGetUppercase()
        {
            using (WebApp.Start<Startup>("http://localhost:8084")) // base hosting address
            {
                var sut = new ToUpperServiceGateway("http://localhost:8084/api/toupper"); 
                // api/toupper matches our test controller / route

                var result = sut.MakeUppercase("hello");

                Assert.Equal("\"FAKE\"", result); // extra " because Web API response is not returning text/plain
            }
        }
    }
}

The test now passes, as our SUT connects to our in-memory test controller.

Note: For even simpler code that doesn’t require a test controller we could also use Nancy.

SHARE:

A Feature Based Approach to Organising Test Code in BDDfy and Other Testing Frameworks

We want our test code to be as high quality as possible, this means smaller amounts of code duplication, reasonably easy to find where things are in Visual Studio, etc.

One possible organization structure is to think in terms of the individual features in the application. The approach you take will probably depend on the complexity and size of the test suite, system under test, etc. Because BDDfy is "just code" we can use all the normal techniques of composition and inheritance that we'd use in our production code to get to the right level of code reuse and organization for the application we're testing.

Organisation by Feature

Organising by feature enables a reasonable amount of code reuse between test scenarios and it also helps to think user- or business-first rather than code/implementation first.

So for example, if we’re using BDDfy to test a banking application we might have the following features:

  • Login
  • Logout
  • Move Money
  • Pay Bills
  • View Transactions
  • etc

Each of these features contains a number of scenarios, for example Login would probably contain scenarios for successful login, bad password, locked out account, 2 factor login, etc.

In Visual Studio we could create folders in the test project to represent and organize these features as the following screenshot illustrates.

Visual Studio feature folders

 

So a (cut down) Login BDDfy story class could look like the following code:

namespace Tests.Login
{
    [TestFixture]
    [Story(AsA="As a Customer",
        IWant = "I want to login",
        SoThat = "So that I can manage my accounts and money")]
    public class CustomerLogin
    {
        [Test]
        public void LoginSuccess()
        {
            this.Given(x => GivenIAmOnTheLoginScreen())
                .And(x => x.AndIHaveEnteredMyUsername())
                .And(x => AndIHaveEnteredMyPassword())
                .When( x=> WhenIChooseLogin())
                .Then(x => ThenIShouldBeLoggedIn())
                .BDDfy<CustomerLogin>();
        }

        public void GivenIAmOnTheLoginScreen()
        {
        }

        public void AndIHaveEnteredMyUsername()
        {
        }

        public void AndIHaveEnteredMyPassword()
        {
        }

        public void WhenIChooseLogin()
        {
        }

        public void ThenIShouldBeLoggedIn()
        {
        }
    }
}

Now for arguments sake, say we have a Navigation story class that represents how the user should be able to move around the applications features.

We could reuse the individual given/when/then methods in CustomerLogin but we don’t want our navigations scenarios to be bloated, we want them to represent the essence of the scenario with the right level of detail.

So the first thing we could do is to create a “step aggregation” method in CustomerLogin as follows:

public void GivenIHaveLoggedIn()
{
    GivenIAmOnTheLoginScreen();
    AndIHaveEnteredMyUsername();
    AndIHaveEnteredMyPassword();
    WhenIChooseLogin();
}

This method simply re-uses the existing steps but aggregates them into a method we can call from the navigation tests:

using NUnit.Framework;
using TestStack.BDDfy;
using TestStack.BDDfy.Core;
using Tests.Login;
using TestStack.BDDfy.Scanners.StepScanners.Fluent;

namespace Tests
{
    [TestFixture]
    [Story(AsA="As a Customer",
        IWant = "I want to navigate around the site",
        SoThat = "So that I can get to the features I want to use")]
    public class Navigation
    {
        [Test]
        public void NavigateToMoveMoney()
        {
            var custLogin = new CustomerLogin();

            this.Given(x => custLogin.GivenIHaveLoggedIn())
                .When(x => WhenChooseGotoMoveMoney())
                .Then(x => ThenIShouldBeTakenToTheMainMoveMoneyScreen())
                .BDDfy<Navigation>();
        }

        private void ThenIShouldBeTakenToTheMainMoveMoneyScreen()
        {            
        }

        private void WhenChooseGotoMoveMoney()
        {            
        }
    }
}

So here were are making use of this aggregate method from the CustomerLogin feature class.

If we don’t need to represent the fact that the customer is logged-in in the report and all the tests in the class assume a logged-in user, we could use some (e.g. NUnit) test setup code that logs the user in but doesn’t get reported.

The HTML output of this looks as follows:

BDDfy HTML report

 

While in these examples we have a single story class for the feature (e.g. CustomerLogin) once we start adding scenarios (and steps) this single story class might become too bloated. If this is deemed a problem then we can break it out into sub features/stories or if it’s applicable we could use inheritance to hold common given/when/then steps. The individual story classes relating to the customer login feature would all inherit this base class. We probably however, do not want multiple levels of nested inheritance in our stories as this may make maintenance and discoverability harder.

To see more of what BDDfy can do check out my Building the Right Thing in .NET with TestStack Pluralsight course, head on over to the documentation, or check it out on GitHub

SHARE:

Testing ASP.Net MVC Controllers with FluentMVCTesting

FluentMVCTesting facilitates the writing of tests against MVC controllers in a fluent way.

FluentMVCTesting is available via NuGet: Install-Package TestStack.FluentMVCTesting

There’s a number of things that FluentMVCTesting can help to test such as:

  • A controller action returns the correct view
  • A controller action returns the correct HTTP status
  • A controller action returns an empty result
  • A controller action returns a view if there are model errors
  • A controller action returns a view with the correct model data
  • A controller action should redirect to a Url / Route / Action

It’s also not tied to a specific testing framework, so it can be used with NUnit, xUnit.net, MSTest, etc.

Examples

var sut = new ExampleController();

sut.WithCallTo(x => x.Show()).ShouldRenderView("Orders");

The preceding test code is testing the ExampleController. It is testing that when the Show() action is called then the Orders view is rendered.

The following code checks that an expected HTTP status is returned.

var sut = new ExampleController();

sut.WithCallTo(x => x.MakeAnError())
    .ShouldGiveHttpStatus(HttpStatusCode.InternalServerError);

 

To see more of what FluentMVCTesting can do check out my Building the Right Thing in .NET with TestStack Pluralsight course, check out the documentation, or check it out on GitHub.

SHARE: