Arrange Act Assert Comments in Tests

The Arrange, Act, Assert (AAA) pattern is used in tests to help organise and clarify test code. It can also help to spot potential problems in test code if these three phases don’t seem to exist.

The Arrange phase is where the thing we’re testing (the system under test) is put into a known beginning state.

The Act phase is where we perform some action on the thing being tested.

The Assert phase is where we check that the results of the Act phase are as expected.

When first learning to use the AAA pattern, it can be helpful to start with 3 comments:

public void ShouldAddNumbers()
{
    // Arrange
    
    // Act
    
    // Assert

}

 

These comments can help to focus on making sure there are three distinct phases.

While these comments are useful when trying to learn (or teach) AAA, they should not be needed in the final version of the test code.

Test code should ideally be as good as production code. One of the things that qualifies code as “clean” is the absence of useless/pointless comments.

Test code should be easy to read, it should not need the AAA comments to be left in to be able to be understood.

If you are using the comment-first approach to help you get started and learn the AAA approach that’s all well and good. However, once the test is written, these comments should usually be removed before the code is committed to source control.

Once the AAA comments are removed, it should still be clear what the flow of the test is. If it is not then the test code may need some changes to improve the readability.

New Pluralsight Course - Introduction to .NET Testing with NUnit

If you are just dipping your toe in the water when it comes to testing .NET applications it can be a bit confusing. In addition to learning how and what to write test, you also have to learn a testing framework such as MSTest, xUnit.net, NUnit, etc.

My new beginner Pluralsight course helps you to get started with testing in .NET and how to use the NUnit testing framework.

Watch the course at the above link or get to it from my Pluralsight author page.

New Pluralsight Course - Automated ASP.NET MVC Testing: End to End

My new Pluralsight course Automated ASP.NET MVC Testing: End to End was just published.

The course looks at tools and techniques to be able to tests models, controller actions, Razor view rendering logic, and implement automated functional UI browser testing.

The course wraps up by showing how to design and implement a continuous integration build in TeamCity that runs the tests created during the course.

Check it out the full table of contents.

Beyond Simple Asserts with ApprovalTests

In a test, we are often testing (asserting) individual items such as an (int) age is correct or a string matches an expected value.

If we are practicing test-first development we’ll write our asserts first.

Approval tests allow us to go beyond simple asserts.

What if the thing we’re checking is not a simple value, for example that a pie chart image matches the input data? Or what if we want to use our human judgement to decide when something looks correct, something that is hard to codify in one or more basic asserts?

ApprovalTests for .NET can be install via NuGet. Once installed, it gives us a whole new world when it comes to checking the output of code.

For example, say we are developing a class to represent a stickman. We want to be able to tell an instance to raise left arm or raise right leg for example.

Example of Using Approval Tests

So lets start with a test:

[Fact]
[UseReporter(typeof(DiffReporter))]
public void ShouldHaveDefaultPosture()
{
    var sut = new StickMan();

    Approvals.Verify(sut);
}

And an empty StickMan:

public class StickMan
{        
}

Here we’re using xUnit.net (the [Fact] attribute) but you could be using NUnit for example.

The first thing to notice here is there is no traditional Assert method, instead we’re using Approval Tests to verify the state of the system under test (sut).

The other think to notice is the [UseReporter] attribute that tells Approval Tests to use a diff tool to display errors when a test fails.

If we run this test, we’ll get a diff tool opened:

More...

In Process Http Server for Integration Test Faking with Owin, Katana, and WebAPI

Sometimes when integration testing we need an HTTP endpoint. We can set one up in our solution (for example a Web Api project) and make sure it’s running before we execute our integration tests but it can be painful. Also consider when running on a build server, the web site will need to be provisioned, etc.

For isolated integration tests that require some HTTP endpoint we can create an HTTP endpoint from within our test code that doesn’t require a separate web project. This means that when tests run they have no external dependency in the sense of a “real” website being hosted somewhere – even if that somewhere is on the local dev machine. Another advantage of the approach outlined below is that because it’s all in-process it’s also faster than actually going through the network stack.

The System Under Test

We are going to test the following class that calls a service to change a string to uppercase:

using System;
using System.Net;

namespace ClassLibrary
{
    public class ToUpperServiceGateway
    {
        private readonly string _serviceEndpoint;

        public ToUpperServiceGateway(string serviceEndpoint)
        {
            _serviceEndpoint = serviceEndpoint;
        }

        public string MakeUppercase(string lowercase)
        {
            using (var wc = new WebClient())
            {
                return wc.DownloadString(_serviceEndpoint + "/" + lowercase);
            }
        }
    }
}

This code uses WebClient to call a web service. In the integration tests we want to spin up a website that will return a mock value for example.

Writing the Initial Test Code

So an initial test could look somethine like:

using Xunit;

namespace ClassLibrary.IntegrationTests
{
    public class UpperCaseTests
    {
        [Fact]
        public void ShouldGetUppercase()
        {
            var sut = new ToUpperServiceGateway("http://localhost:8084/api/toupper");

            var upper = sut.MakeUppercase("hello");

            Assert.Equal("HELLO", upper);            
        }
    }
}

If we run this we get “System.Net.WebException Unable to connect to the remote server” as we’d expect as there’s no website at the address.

Creating an In-Process Web API Endpoint

First off, in the test project install a few of NuGets: Microsoft.AspNet.WebApi.OwinSelfHost and Microsoft.Owin.Testing.

The first thing is to create a Web API controller, again in the test project, to fake out a real service:

using System.Web.Http;

namespace ClassLibrary.IntegrationTests
{
    public class ToUpperController : ApiController
    {
        public string Get(string lower)
        {            
            return "FAKE"; // fake value
        }
    }
}

Here, regardless of what’s passed in we’re just returning a fake value.

The next task is to create some configuration for Owin setting up our test controller:

using System.Web.Http;
using Owin;

namespace ClassLibrary.IntegrationTests
{
    public class Startup
    {
        public void Configuration(IAppBuilder appBuilder)
        {
            var config = new HttpConfiguration();

            config.Routes.MapHttpRoute("DefaultApi", "api/{controller}/{lower}");

            appBuilder.UseWebApi(config);
        }
    }
}

We can now refactor the test to create an in-memory Web API controller that can respond to our SUT:

using Microsoft.Owin.Hosting;
using Xunit;

namespace ClassLibrary.IntegrationTests
{
    public class UpperCaseTests
    {
        [Fact]
        public void ShouldGetUppercase()
        {
            using (WebApp.Start<Startup>("http://localhost:8084")) // base hosting address
            {
                var sut = new ToUpperServiceGateway("http://localhost:8084/api/toupper"); 
                // api/toupper matches our test controller / route

                var result = sut.MakeUppercase("hello");

                Assert.Equal("\"FAKE\"", result); // extra " because Web API response is not returning text/plain
            }
        }
    }
}

The test now passes, as our SUT connects to our in-memory test controller.

Note: For even simpler code that doesn’t require a test controller we could also use Nancy.

A Feature Based Approach to Organising Test Code in BDDfy and Other Testing Frameworks

We want our test code to be as high quality as possible, this means smaller amounts of code duplication, reasonably easy to find where things are in Visual Studio, etc.

One possible organization structure is to think in terms of the individual features in the application. The approach you take will probably depend on the complexity and size of the test suite, system under test, etc. Because BDDfy is "just code" we can use all the normal techniques of composition and inheritance that we'd use in our production code to get to the right level of code reuse and organization for the application we're testing.

Organisation by Feature

Organising by feature enables a reasonable amount of code reuse between test scenarios and it also helps to think user- or business-first rather than code/implementation first.

So for example, if we’re using BDDfy to test a banking application we might have the following features:

  • Login
  • Logout
  • Move Money
  • Pay Bills
  • View Transactions
  • etc

Each of these features contains a number of scenarios, for example Login would probably contain scenarios for successful login, bad password, locked out account, 2 factor login, etc.

In Visual Studio we could create folders in the test project to represent and organize these features as the following screenshot illustrates.

Visual Studio feature folders

 

So a (cut down) Login BDDfy story class could look like the following code:

namespace Tests.Login
{
    [TestFixture]
    [Story(AsA="As a Customer",
        IWant = "I want to login",
        SoThat = "So that I can manage my accounts and money")]
    public class CustomerLogin
    {
        [Test]
        public void LoginSuccess()
        {
            this.Given(x => GivenIAmOnTheLoginScreen())
                .And(x => x.AndIHaveEnteredMyUsername())
                .And(x => AndIHaveEnteredMyPassword())
                .When( x=> WhenIChooseLogin())
                .Then(x => ThenIShouldBeLoggedIn())
                .BDDfy<CustomerLogin>();
        }

        public void GivenIAmOnTheLoginScreen()
        {
        }

        public void AndIHaveEnteredMyUsername()
        {
        }

        public void AndIHaveEnteredMyPassword()
        {
        }

        public void WhenIChooseLogin()
        {
        }

        public void ThenIShouldBeLoggedIn()
        {
        }
    }
}

Now for arguments sake, say we have a Navigation story class that represents how the user should be able to move around the applications features.

We could reuse the individual given/when/then methods in CustomerLogin but we don’t want our navigations scenarios to be bloated, we want them to represent the essence of the scenario with the right level of detail.

So the first thing we could do is to create a “step aggregation” method in CustomerLogin as follows:

public void GivenIHaveLoggedIn()
{
    GivenIAmOnTheLoginScreen();
    AndIHaveEnteredMyUsername();
    AndIHaveEnteredMyPassword();
    WhenIChooseLogin();
}

This method simply re-uses the existing steps but aggregates them into a method we can call from the navigation tests:

using NUnit.Framework;
using TestStack.BDDfy;
using TestStack.BDDfy.Core;
using Tests.Login;
using TestStack.BDDfy.Scanners.StepScanners.Fluent;

namespace Tests
{
    [TestFixture]
    [Story(AsA="As a Customer",
        IWant = "I want to navigate around the site",
        SoThat = "So that I can get to the features I want to use")]
    public class Navigation
    {
        [Test]
        public void NavigateToMoveMoney()
        {
            var custLogin = new CustomerLogin();

            this.Given(x => custLogin.GivenIHaveLoggedIn())
                .When(x => WhenChooseGotoMoveMoney())
                .Then(x => ThenIShouldBeTakenToTheMainMoveMoneyScreen())
                .BDDfy<Navigation>();
        }

        private void ThenIShouldBeTakenToTheMainMoveMoneyScreen()
        {            
        }

        private void WhenChooseGotoMoveMoney()
        {            
        }
    }
}

So here were are making use of this aggregate method from the CustomerLogin feature class.

If we don’t need to represent the fact that the customer is logged-in in the report and all the tests in the class assume a logged-in user, we could use some (e.g. NUnit) test setup code that logs the user in but doesn’t get reported.

The HTML output of this looks as follows:

BDDfy HTML report

 

While in these examples we have a single story class for the feature (e.g. CustomerLogin) once we start adding scenarios (and steps) this single story class might become too bloated. If this is deemed a problem then we can break it out into sub features/stories or if it’s applicable we could use inheritance to hold common given/when/then steps. The individual story classes relating to the customer login feature would all inherit this base class. We probably however, do not want multiple levels of nested inheritance in our stories as this may make maintenance and discoverability harder.

To see more of what BDDfy can do check out my Building the Right Thing in .NET with TestStack Pluralsight course, head on over to the documentation, or check it out on GitHub

Testing ASP.Net MVC Controllers with FluentMVCTesting

FluentMVCTesting facilitates the writing of tests against MVC controllers in a fluent way.

FluentMVCTesting is available via NuGet: Install-Package TestStack.FluentMVCTesting

There’s a number of things that FluentMVCTesting can help to test such as:

  • A controller action returns the correct view
  • A controller action returns the correct HTTP status
  • A controller action returns an empty result
  • A controller action returns a view if there are model errors
  • A controller action returns a view with the correct model data
  • A controller action should redirect to a Url / Route / Action

It’s also not tied to a specific testing framework, so it can be used with NUnit, xUnit.net, MSTest, etc.

Examples

var sut = new ExampleController();

sut.WithCallTo(x => x.Show()).ShouldRenderView("Orders");

The preceding test code is testing the ExampleController. It is testing that when the Show() action is called then the Orders view is rendered.

The following code checks that an expected HTTP status is returned.

var sut = new ExampleController();

sut.WithCallTo(x => x.MakeAnError())
    .ShouldGiveHttpStatus(HttpStatusCode.InternalServerError);

 

To see more of what FluentMVCTesting can do check out my Building the Right Thing in .NET with TestStack Pluralsight course, check out the documentation, or check it out on GitHub.

Business-Readable, Living Documentation with BDDfy

BDDfy enables the creation of tests that, in addition to verifying that the system works correctly, also results in business-readable, living documentation.

Business-readable means that the tests are described in natural language (e.g. English) that the business can read, understand, and ensure that the correct features and functionality is being built.

Living documentation means that the report results directly from the passing or failing of the tests. It’s not a word document somewhere on a shared drive or SharePoint site that may or may not actually be in sync with what the system actually does.

The Report

Below is an example of what a typical BDDfy HTML report looks like (there is a “metro” inspired report coming in V4). There’s also the ability to output the test report in markdown.

BDDfy HTML report

Notice that it’s not code-centric output, but rather business-centric.

Here the two test scenarios are grouped into a story, but they don’t have to be.

The underlying test framework that is used to execute the test doesn’t matter – you could use NUnit, xUnit.net, MSTest, etc.

The Test Code

The tests that produce this report could be configured in BDDfy in a couple of ways. There is a reflective style that uses methods that following a specific naming convention. There is also the fluent style.

The test code to produce this report looks like the below (we are using NUnit in this example).

using NUnit.Framework;
using TestStack.BDDfy;
using TestStack.BDDfy.Core;
using TestStack.BDDfy.Scanners.StepScanners.Fluent;

namespace BDDfyDemo
{
    [TestFixture]
    [Story(AsA = "As a customer",
        IWant = "I want my order total to add up correctly",
        SoThat = "I'm not overcharged for my goods")]
    public class OrderTotalCalculatorTests
    {
        [Test]
        public void MultipleOrderedItemsTotals()
        {
            this.Given(x => GivenIHaveAddedItemsToMyCart())
                .When(x => WhenICheckout())
                .Then(x => ThenTheOrderTotalExcludingTaxShouldBeCorrect())
                .BDDfy<OrderTotalCalculatorTests>();
        }

        [Test]
        public void SalesTaxAdded()
        {
            this.Given(x => GivenIHaveAddedItemsToMyCart())
                .When(x => WhenICheckout())
                .Then(x => ThenTheSalesTaxShouldBeCorrect())
                .BDDfy<OrderTotalCalculatorTests>();
        }

        public void GivenIHaveAddedItemsToMyCart()
        {
            // test code
        }

        public void WhenICheckout()
        {
            // test code
        }

        public void ThenTheOrderTotalExcludingTaxShouldBeCorrect()
        {
            // test code
        }

        public void ThenTheSalesTaxShouldBeCorrect()
        {
            // test code
        }

    }
}

Here there are a couple of NUnit tests defined – the methods with the [Test] attribute applied to them.

Within these test methods the BDDfy fluent style is being used to define the different steps in the test scenarios.

The “given” phase sets up the initial context or state of the thing being tested. The “when” phase acts upon the system to produce some change. The “then” phase is typically where we have assert code, it’s in this phase that the resulting state of the system is checked.

Notice how the individual method names appear in the test report in a nice business-readable way.

 

There’s a lot more to BDDfy, such as data-parameterised tests and a whole heap of customisation and configuration options. To see more of what BDDfy can do check out my Building the Right Thing in .NET with TestStack Pluralsight course, head on over to the documentation, or check it out on GitHub.

Beyond the Compiler with ConventionTests

We often have conventions in our code. For example all DTOs/entity/etc. classes should be in a specific namespace, or the name of certain types of class should end with a given word. Another example, we may need to make sure that all classes (or those in a specific namespace) contain all-virtual members.

These kind of things are conventions that the team may agree to but the compiler won’t pick up as build errors or warnings. The code may still be completely valid C# even though it violates a convention.

ConventionTests is a brilliant idea (and it’s ideas like this that keep me loving programming) allows us to write tests to verify these conventions, conventions that the compiler cannot verify.

Installation

Simple installation via NuGet into your test project: PM> Install-Package TestStack.ConventionTests

Simple Usage

Using whatever testing framework you prefer (xUnit.net, NUnit, etc.) create a test method.

these kind of things are conventions that the team may agree to but the compiler won’t pick up

In this method, the first thing to do is to select all the types in the production code that we want to check against a convention.

We can do this with code such as:

var typesToCheck = Types.InAssemblyOf<SomeClass>();   

Here the Types.InAssemblyOf method is being used to find which assembly contains the class called SomeClass. Using this overload, all of the types in the assembly will be selected, however there are other overloads that allow the returning of types in a given namespace.

The next step is to create an instance of one of the out-of-the-box conventions.

var convention = new AllClassesHaveDefaultConstructor();

This convention checks that all the selected types have a default constructor defined.

Now we have the list of types we want to check, and a convention to check them against.

To actually perform the check, we use the Convention.Is method:

Convention.Is(convention, typesToCheck);

If any of the classes selected in typesToCheck don’t have a default constructor then an exception will be thrown (with some useful info in it) and the test will fail.

You can see some of the other out-of-the-box conventions on the GitHub site, you can also write your own custom conventions.

 

ConventionTests is one of the tools in the TestStack suite. To learn more about ConventionTests and the other TestStack tools, checkout out my Building the Right Thing in .NET with TestStack Plurasight course.

Using Page Object Models in UI Test Automation

When using UI automation technologies, changes to the UI can break our tests. For example, if a textbox is being located by its ID in numerous tests, if this ID changes then all these tests will break. They won’t be able to find the ID.

Another problem that can arise with automated UI testing is code duplication. Imagine a system that had an color drop down list and an apply button in multiple places in the application. Each UI test that exercises these color choices must first locate the color choice box, select a color, locate the apply button, and then click it.

The 3 Levels Of User Interface Testing

3 Levels Of User Interface Testing

One way to think of the “evolution” of UI testing is that of: test script, page object model (POM), and logical functional model (LFM).

Test Script

Test scripts are self contained tests. They contain the whole test from starting the application through to the asserts at the end.

Test scripts can be lengthy as they have to include all the UI control finding and interaction code for every single step of the particular test. This can make them hard to read and understand exactly what behaviour is being tested.

Test scripts also don’t share any code with other tests, so code duplication can create additional maintenance effort over time.

Test scripts are an easy way to get started with automated UI testing, especially if you are learning the UI automation framework at the same time.

Pretty quickly, test scripts become unwieldy and so page object models can be introduced.

Page Object Models

Page object models (POM) represent a screen or page or group of controls in the UI that’s being tested.

For example, a screen with a color chooser could be represented by a POM class that exposes the drop down control and buttons as properties.

These properties encapsulate the locating of the UI dropdown and button (perhaps by ID). The test code itself then uses an instance of the POM to interact with the dropdown and button – it doesn’t have to worry about the UI IDs in the test itself.

There’s a couple of levels of abstraction within POMs. The property that represents a UI control can either expose the underlying automation framework object that represents the control, or a simple type such as string or int.

When first getting started with POMs you might want to go with the former approach until you get the idea of using POMs, though there’s not much additional work to implement the latter option.

By returning simple types rather than automation framework types, the actual test code can become more readable; they also help to abstract away the chosen automation framework.

Logical Functional Models

Logical function models (LFM) are an extension to POMs and may not deserve their own distinction in practice, however the distinction helps us to think how we can further enhance the readability of test code and reduce code duplication.

LFMs add methods to POMs. These methods represent logical actions that the end-user can perform on a given page.

The LFM methods can use the properties already defined on the POM.

For example, the logical user function of “I want to choose a color” can be represented as a single method in the POM, for example: ChooseColor(string color);

Within this single method, the existing POM properties representing the dropdown list of colors and button can be used.

Now, test code can simply write: myScreen.ChooseColor(“red”);

The LFM “style” further increases the ability to discern exactly what behaviour a given test is testing.

Building Page Object Models

When building a POM it’s a good idea to evolve it over time, adding properties and logical functional methods as needed by the tests.

The alternative is to model every control and user interaction on the page before starting writing test that use that POM. This increases the up-front cost of writing automated UI tests, you may not end up interacting with every UI control in your tests so it makes sense to evolve POMs in an iterative fashion.

When choosing a UI automation framework, investigate what built-in support it has for POMs/LFMs. It’s possible to hand-code them from scratch, but a little extra help from our chosen framework helps to reduce the cost of building UI automation tests.

 

To see POMs/LFMs in action in both WPF and Web UI automation, check out my Building the Right Thing in .NET with TestStack Pluralsight course.