Testing That Your Public APIs Have Not Changed Unexpectedly with PublicApiGenerator and Approval Tests

We can write automated tests to cover various aspects of the code we write. We can write unit/integration tests that test that the code is producing the expected outcomes. We can use ConventionTests to ensure internal code quality, for example that classes following a specified naming convention and exists in the correct namespace. We may even add the ability to create a business readable tests using tools such as SpecFlow or BDDfy.

Another aspect that we might want to ensure doesn’t change unexpectedly is the public API that our code exposes to callers.

Using PublicApiGenerator to Generate a Report of our Public API

The first step of ensuring our public API hasn’t changed is to be able to capture the public API in a readable way. The PublicApiGenerator NuGet package (from Jake Ginnivan) gives us this ability.

Suppose we have the following class defined:

public class Calculator
{
    public Calculator()
    {
        CurrentValue = 0;
    }

    public int CurrentValue { get; private set; }

    public void Clear()
    {
        CurrentValue = 0;
    }

    public void Add(int number)
    {
        CurrentValue += number;
    }
}

Notice here that this code defines the public API that consumers of the Calculator class can use. It’s this public API that we want to test to ensure it doesn’t change unexpectedly.

We might start with some unit tests as shown in the following code:

public class CalculatorTests
{
    [Fact]
    public void ShouldHaveInitialValue()
    {
        var sut = new Calculator();

        Assert.Equal(0, sut.CurrentValue);
    }

    [Fact]
    public void ShouldAdd()
    {
        var sut = new Calculator();

        sut.Add(1);

        Assert.Equal(1, sut.CurrentValue);
    }
}

These tests help us ensure the code is doing the right thing but do not offer any protection against the public API changing. We can now add a new test that uses PublicApiGenerator to generate a string “report” detailing the public members of our API. The following test code shows this in use:

[Fact]
public void ShouldHaveCorrectPublicApi()
{
    var sut = new Calculator();

    // Get the assembly that we want to generate the public API report for
    Assembly calculatorAssembly = sut.GetType().Assembly;

    // Use PublicApiGenerator to generate the API report
    string apiString = PublicApiGenerator.PublicApiGenerator.GetPublicApi(calculatorAssembly);

    // TODO: assert API has not changed
}

If we debug this test and look at the content of the apiString variable we’d see the following text:

[assembly: System.Runtime.InteropServices.ComVisibleAttribute(false)]
[assembly: System.Runtime.InteropServices.GuidAttribute("c2dc3732-a4a5-4baa-b4df-90f40aad1c6a")]
[assembly: System.Runtime.Versioning.TargetFrameworkAttribute(".NETFramework,Version=v4.5.1", FrameworkDisplayName=".NET Framework 4.5.1")]

namespace Demo
{
    
    public class Calculator
    {
        public Calculator() { }
        public int CurrentValue { get; }
        public void Add(int number) { }
        public void Clear() { }
    }
}

Using Approval Tests to Assert the API Is Correct

Now in our test we have a string that represents the public API. We can combine PublicApiGenerator with the Approval Tests library to check that this API text doesn’t change.

First off we go and install the Approval Tests NuGet Package. We can then modify the test as shown below:

public class CalculatorApiTests
{
    [Fact]
    public void ShouldHaveCorrectPublicApi()
    {
        var sut = new Calculator();

        // Get the assembly that we want to generate the public API report for
        Assembly calculatorAssembly = sut.GetType().Assembly;

        // Use PublicApiGenerator to generate the API report
        string apiString = PublicApiGenerator.PublicApiGenerator.GetPublicApi(calculatorAssembly);

        // Use Approval Tests to verify the API hasn't changed
        Approvals.Verify(apiString);
    }
}

The first time we run this it will fail with a message such as “Failed Approval: Approval File "c:\…\Demo.Tests\CalculatorApiTests.ShouldHaveCorrectPublicApi.approved.txt" Not Found”. It will also generate a file called CalculatorApiTests.ShouldHaveCorrectPublicApi.received.txt. We can rename this to CalculatorApiTests.ShouldHaveCorrectPublicApi.approved.txt, run the test again and it will pass.

If we now modify the public API by changing a method signature (e.g. to public void Clear(int someParam)) and run the test again it will fail with a message such as “Received file c:\...\Demo.Tests\CalculatorApiTests.ShouldHaveCorrectPublicApi.received.txt does not match approved file c:\...\Demo.Tests\CalculatorApiTests.ShouldHaveCorrectPublicApi.approved.txt”.

Modifying the test and adding an Approval Tests reporter attribute ([UseReporter(typeof(DiffReporter))]) and running the test will now gives us a visual diff identifying the changes to the public API as shown in the following screenshot.

Approval Tests Diff Screenshot

To learn more about the features of Approval Tests, check out my Approval Tests for .NET Pluralsight course.

Painless .NET Windows Service Creation with Topshelf

Windows Services allow us to run code in the background, without needing (for example) a console application running continually. Windows Services can run as various system users or specific local/domain users.

To see what Windows Services are installed on your (Win 8+) PC, hit Win+x then hit G. In the Computer Management window that opens, click the Services option under Services and Applications as shown in the following screenshot.

Computer Management window showing installed Windows Services

As the preceding screenshot shows, a Service has a (Display) Name, a Description, a Status (e.g. Running or not), a Startup Type (Manual, Automatic, Disabled, etc), and a Log On As that specifies in which user context the Service executes.

We can create Windows Services to run arbitrary .NET code such as:

  • Self-hosted web server using Owin
  • File system watcher and batch processing (e.g. image file conversion, video encoding)
  • Host remote Akka.NET actor system (such as in my Pluralsight course)
  • Process messages from a message queue as they arrive (e.g. MSMQ)
  • Mail, FTP, etc. servers
  • Integration/gateway, e.g. receive data from external systems

Using Topshelf

Topshelf is an open source project that greatly simplifies the creation of Windows Services.

The overview of creating a Service using Topshelf is:

  1. Create a Console application project in Visual Studio
  2. Add the Topshelf NuGet package
  3. Create a class to represent your service logic
  4. Configure your Service using the Topshelf API
  5. Build your Console application
  6. Execute your Console application passing Topshelf parameters to install/uninstall your Service


So assuming we have a new Console project called “Time” and the following NuGet packages are installed:

<packages>
  <package id="Microsoft.AspNet.WebApi.Client" version="5.2.3" targetFramework="net451" />
  <package id="Microsoft.AspNet.WebApi.Core" version="5.2.3" targetFramework="net451" />
  <package id="Microsoft.AspNet.WebApi.Owin" version="5.2.3" targetFramework="net451" />
  <package id="Microsoft.AspNet.WebApi.OwinSelfHost" version="5.2.3" targetFramework="net451" />
  <package id="Microsoft.Owin" version="2.0.2" targetFramework="net451" />
  <package id="Microsoft.Owin.Host.HttpListener" version="2.0.2" targetFramework="net451" />
  <package id="Microsoft.Owin.Hosting" version="2.0.2" targetFramework="net451" />
  <package id="Newtonsoft.Json" version="6.0.4" targetFramework="net451" />
  <package id="Owin" version="1.0" targetFramework="net451" />
  <package id="Topshelf" version="3.2.0" targetFramework="net451" />
</packages>

We can create a self-hosted OWIN service that tells us the time.

First off we create an API Controller to return the time and then create an OWIN startup configuration class:

public class TimeController : ApiController
{
    [HttpGet]
    public string Now()
    {
        return DateTime.Now.ToLongTimeString();
    }
}

public class Startup
{
    public void Configuration(IAppBuilder appBuilder)
    {
        var config = new HttpConfiguration();

        config.Routes.MapHttpRoute("DefaultApi", "api/{controller}/{time}");
        config.Formatters.Remove(config.Formatters.XmlFormatter);
        config.Formatters.Add(config.Formatters.JsonFormatter);

        appBuilder.UseWebApi(config);
    }
}

Next we create a class that represents our Topshelf Windows Service:

class TimeService
{
    private IDisposable _webServer;

    public void Start()
    {
        // code that runs when the Windows Service starts up
        _webServer = WebApp.Start<Startup>("http://localhost:8084");
    }

    public void Stop()
    {
        // code that runs when the Windows Service stops
        _webServer.Dispose();
    }
}

Now in the Console applications main method we can use Topshelf’s HostFactory to configure what our Windows Service will do (via the TimeService class) and how it will be configured in Windows:

class Program
{
    static void Main(string[] args)
    {
        HostFactory.Run(
            configuration =>
            {
                configuration.Service<TimeService>(
                    service =>
                    {
                        service.ConstructUsing(x => new TimeService());
                        service.WhenStarted(x => x.Start());
                        service.WhenStopped(x => x.Stop());
                    });

                configuration.RunAsLocalSystem();

                configuration.SetServiceName("ASimpleService");
                configuration.SetDisplayName("A Simple Service");
                configuration.SetDescription("Don't Code Tired Demo");
            });
    }
}

Once the solution is built, navigate to the bin/debug directory in a admin command prompt and type:

time.exe install

This will install the Service into Windows as the following screenshots show.

command prompt showing Topshelf service installation


Service installed in Windows


In the command prompt window type:

time.exe start

This will start our new service.

Now in a browser, navigate to http://localhost:8084/api/time/now and you’ll see the current time come back as JSON

screenshot showing browser getting time

To learn more about Topshelf, check out my Pluralsight course: Getting Started Building Windows Services with Topshelf.

Better User Experiences and More Robust Applications Pluralsight Course

My new Pluralsight course was just published which shows how to use the open source Polly library.

Polly is a great library by Michael Wolfenden for configuring the automatic retrying of operations if exceptions occur. Transient errors such as the network being slow/unavailable for a small amount of time can normally result in errors surfacing to the user and/or into log files that need looking at.

By using one of the exception handling strategies that Polly provides, these transient exceptions can automatically be retried, preventing user errors and/or log entries.

There are four strategies:

  • Retry Forever
  • Retry
  • Wait and Retry
  • Circuit Breaker

Polly is configured in a fluent way, for example to use the Wait and Retry strategy to retry 3 times waiting 2, 4, and 6 seconds between retries:

Policy.Handle<some exception>()
      .WaitAndRetry(new[]
          {
            TimeSpan.FromSeconds(2),
            TimeSpan.FromSeconds(4),
            TimeSpan.FromSeconds(6)
          })
      .Execute(some action);

Check out the Polly GitHub repo, the NuGet package or the “Better User Experiences and More Robust Applications with Polly” Pluralsight course.

Improving Test Code Readability and Assert Failure Messages with Shouldly

Shouldly is an open source library that aims to improve the assert phase of tests; it does this in two ways. The first is offering a more “fluent like” syntax that for the most part leverages extension methods and obviates the need to keep remembering which parameter is the expected or actual as with regular Assert.Xxxx(1,2) methods. The second benefit manifests itself when tests fail; Shouldly outputs more readable, easily digestible test failure messages.

Failure Message Examples

The following are three failure messages from tests that don’t use Shouldly and instead use the assert methods bundled with the testing framework (NUnit, xUnit.net, etc):

  • “Expected: 9  But was:  5”
  • “Assert.NotNull() Failure”
  • “Not found: Monday In value:  List<String> ["Tuesday", "Wednesday", "Thursday"]”

In each of the preceding failure messages, there is not much helpful context in the failure message.

Compare the above to the following equivalent Shouldly failure messages:

  • “schedule.TotalHours should be 9 but was 5”
  • “schedule.Title should not be null or empty”
  • “schedule.Days should contain "Monday" but does not”

Notice the additional context in these failure messages. In each case here, Shouldly is telling us the name of the variable in the test code (“schedule”) and the name of the property/field being asserted (e.g. “Total Hours”).

Test Code Readability

For the preceding failure messages, the following test assert code is used (notice the use of the Shouldly extension methods):

  • schedule.TotalHours.ShouldBe(9);
  • schedule.Title.ShouldNotBeNullOrEmpty();
  • schedule.Days.ShouldContain("Monday");

In these examples there is no mistaking an actual value parameter for an expected value parameter and the test code reads more “fluently” as well.

To find out more about Shouldly check out the project on GitHub, install via NuGet, or checkout my Better Unit Test Assertions with Shouldly Pluralsight course.

Clean C# eBook Published

The final version of my free Clean C# eBook has just been published.

Clean C# eBook Cover Image

 

You can download the book for free from http://cleancsharp.com/

The book contains the following chapters:

  • Comments
  • Naming Things
  • Methods
  • Structuring Programs for Readability
  • Errors and Exceptions
  • Visual Formatting
  • Cohesion and Coupling
  • Clean Tests
  • Building On Clean Code
  • New Free eBook: LINQ Succinctly

    My new free Syncfusion eBook is now available.

    x-ebook

    LINQ Succinctly is available from the Syncfusion site along with all the other eBooks in the Succinctly series.

    LINQ Succinctly covers the following areas:

    1. LINQ Fundamentals
    2. Fluent and Query Expression Styles
    3. LINQ Query Operators
    4. LINQ to XML
    5. Interpreted Queries
    6. Parallel LINQ
    7. LINQ Tools and Resources

    Fixie - A Convention-based .NET Testing Framework

    Fixie is a relative newcomer to the .NET testing framework space. It throws away the idea of marking elements of test code with attributes in favour of a conventions based approach.

    At a high level, what this means is simply naming things in test projects following a defined default (customized conventions are also supported) convention.

    A Fixie test class with a single test inside it would look like the following, notice the lack of attributes and even using statements.

    namespace MyApplication.Tests
    {
        public class SomeClassTests
        {
            public void ShouldDoSomething()
            {
            }
        }
    }
    
    

    After building the test will now show up in Visual Studio Test Explorer as the following screenshot shows.

    Visual Studio Test Explorer showing Fixie Test

    Fixie knows this is a test because it matches the default conventions that come out of the box.

    Fixie knows that this is a test class because it ends with “Tests” and it knows that the method is a test because it’s a public void method.

    If these default conventions are not suitable for your project you can create your own custom conventions to customise the test discovery. There’s also a lot more to custom conventions, such as customising the test execution lifecycle and creating data-driven tests.

    To learn more about Fixie, check out the docs or my Introduction to Fixie Pluralsight course.

    Arrange Act Assert Comments in Tests

    The Arrange, Act, Assert (AAA) pattern is used in tests to help organise and clarify test code. It can also help to spot potential problems in test code if these three phases don’t seem to exist.

    The Arrange phase is where the thing we’re testing (the system under test) is put into a known beginning state.

    The Act phase is where we perform some action on the thing being tested.

    The Assert phase is where we check that the results of the Act phase are as expected.

    When first learning to use the AAA pattern, it can be helpful to start with 3 comments:

    public void ShouldAddNumbers()
    {
        // Arrange
        
        // Act
        
        // Assert
    
    }
    

     

    These comments can help to focus on making sure there are three distinct phases.

    While these comments are useful when trying to learn (or teach) AAA, they should not be needed in the final version of the test code.

    Test code should ideally be as good as production code. One of the things that qualifies code as “clean” is the absence of useless/pointless comments.

    Test code should be easy to read, it should not need the AAA comments to be left in to be able to be understood.

    If you are using the comment-first approach to help you get started and learn the AAA approach that’s all well and good. However, once the test is written, these comments should usually be removed before the code is committed to source control.

    Once the AAA comments are removed, it should still be clear what the flow of the test is. If it is not then the test code may need some changes to improve the readability.

    New Pluralsight Course - Introduction to .NET Testing with NUnit

    If you are just dipping your toe in the water when it comes to testing .NET applications it can be a bit confusing. In addition to learning how and what to write test, you also have to learn a testing framework such as MSTest, xUnit.net, NUnit, etc.

    My new beginner Pluralsight course helps you to get started with testing in .NET and how to use the NUnit testing framework.

    Watch the course at the above link or get to it from my Pluralsight author page.