Improving Test Code Readability and Assert Failure Messages with Shouldly

Shouldly is an open source library that aims to improve the assert phase of tests; it does this in two ways. The first is offering a more “fluent like” syntax that for the most part leverages extension methods and obviates the need to keep remembering which parameter is the expected or actual as with regular Assert.Xxxx(1,2) methods. The second benefit manifests itself when tests fail; Shouldly outputs more readable, easily digestible test failure messages.

Failure Message Examples

The following are three failure messages from tests that don’t use Shouldly and instead use the assert methods bundled with the testing framework (NUnit, xUnit.net, etc):

  • “Expected: 9  But was:  5”
  • “Assert.NotNull() Failure”
  • “Not found: Monday In value:  List<String> ["Tuesday", "Wednesday", "Thursday"]”

In each of the preceding failure messages, there is not much helpful context in the failure message.

Compare the above to the following equivalent Shouldly failure messages:

  • “schedule.TotalHours should be 9 but was 5”
  • “schedule.Title should not be null or empty”
  • “schedule.Days should contain "Monday" but does not”

Notice the additional context in these failure messages. In each case here, Shouldly is telling us the name of the variable in the test code (“schedule”) and the name of the property/field being asserted (e.g. “Total Hours”).

Test Code Readability

For the preceding failure messages, the following test assert code is used (notice the use of the Shouldly extension methods):

  • schedule.TotalHours.ShouldBe(9);
  • schedule.Title.ShouldNotBeNullOrEmpty();
  • schedule.Days.ShouldContain("Monday");

In these examples there is no mistaking an actual value parameter for an expected value parameter and the test code reads more “fluently” as well.

To find out more about Shouldly check out the project on GitHub, install via NuGet, or checkout my Better Unit Test Assertions with Shouldly Pluralsight course.

FeatureToggle Version 3 Released

FeatureToggle logo image

Version 3 of my open source feature toggle library is now released to NuGet.

Version 3 introduces some breaking changes and new features.

Breaking Changes

  • The WPF value convertor package has been removed for version 3 as it’s trivial to implement a version in individual applications and reduce the maintenance overhead for the porject
  • Version 3 removes support for Windows Phone 8.0 (you can still use version 2.x if you need this support)

New Features

Platforms Support

In addition to .NET framework 4 full, version 3 also supports:

  • Windows Phone 8.1 Silverlight apps
  • Windows Universal Apps 8.1 (Store and Windows Phone 8.1 WinRT)
  • Windows Store 8.1 apps

Future versions will add support for Windows 10 apps.

New Toggle Types

The EnabledOnOrAfterAssemblyVersionWhereToggleIsDefinedToggle (a little verbose I know) allows a feature to become enabled once a specified assembly version number is reached. The assembly version used is that of the assembly inside which the toggle is defined, rather that the executing assembly. The configuration value for this toggle would be (for example) 0.0.2.1.

Future version will add support for comparing the executing assembly version.

New Decorators

A new type of feature that version 3 introduces is the idea of toggle decorators. These are classes that “wrap” one or more toggles and provide additional features.

The DefaultToEnabledOnErrorDecorator and the DefaultToDisabledOnErrorDecorator allows a default value to provided if there is an error evaluating the wrapped toggle value, for example with a configuration error. These decorators should be used with caution as your application may end up in an unknown state if you forget to configure something properly. In some cases you may want this behaviour rather than the application failing.

Assuming that a feature toggle called MyFeatureToggle has been defined, to use the decorator the following code would be used.

IFeatureToggle printFeatureThatDefaultsToDisabled = new DefaultToDisabledOnErrorDecorator( new MyFeatureToggle() );

bool isPrintEnabled = printFeatureThatDefaultsToDisabled.FeatureEnabled;

These decorators also allow you to specify an optional logging action that gets called if the wrapped toggle throws an exception:

new DefaultToDisabledOnErrorDecorator(wrappedToggle, ex => Console.WriteLine(ex.Message));

 

The CompositeAndDecorator allows multiple toggles to be wrapped, only if all these wrapped toggles are enabled will the decorator return true.

IFeatureToggle andToggle = new CompositeAndDecorator(new AnEnabledFeature(), new ADisabledFeature());

var isEnabled = andToggle.FeatureEnabled; // returns false

 

The final new decorator is the FixedTimeCacheDecorator. This allows a toggle that gets its value from a slow source (such as remote database, http, etc.) to have its value cached for a fixed period of time. To use this you specify the wrapped toggle and a TimeSpan.

IFeatureToggle cachedToggle = new FixedTimeCacheDecorator(cachedToggle, TimeSpan.FromSeconds(10));

 

Check out the release notes on GitHub. If you find any problems or bugs or want to suggest other features, please create an issue on GitHub.

Clean C# eBook Published

The final version of my free Clean C# eBook has just been published.

Clean C# eBook Cover Image

 

You can download the book for free from http://cleancsharp.com/

The book contains the following chapters:

  • Comments
  • Naming Things
  • Methods
  • Structuring Programs for Readability
  • Errors and Exceptions
  • Visual Formatting
  • Cohesion and Coupling
  • Clean Tests
  • Building On Clean Code
  • New Free eBook: LINQ Succinctly

    My new free Syncfusion eBook is now available.

    x-ebook

    LINQ Succinctly is available from the Syncfusion site along with all the other eBooks in the Succinctly series.

    LINQ Succinctly covers the following areas:

    1. LINQ Fundamentals
    2. Fluent and Query Expression Styles
    3. LINQ Query Operators
    4. LINQ to XML
    5. Interpreted Queries
    6. Parallel LINQ
    7. LINQ Tools and Resources

    An Update on FeatureToggle v3 Development

    The next version of my open source feature toggling library is currently in development.

    Version 3 will introduce a number of new features and retire support for some older platforms. For the currently implemented changes (not yet released at the time of writing) see GitHub release notes, and the complete list of v3 issues/features.

    Some notable additions include some new toggle types to enable scenarios such as automatically enabling a feature once the assembly version number meets or exceeds a specific version and decorators to allow a toggle to default to on or off if the underlying toggle errors in some way, such as missing configuration or network error while retrieving the toggle value from a remote source such as SQL Server.

    Once v3 development is complete it will be available via NuGet.

    New Pluralsight Course - Better .NET Unit Tests with AutoFixture

    My new Pluralsight testing course was just released: Better .NET Unit Tests with AutoFixture: Get Started.

    AutoFixture is an open source library that allows the simplification of automated .NET tests. It allows for the creation of “anonymous” test data. This is data that is required for the test to execute, but where the exact values are unimportant.

    AutoFixture is not dependent upon any specific testing framework, so can be used with xUnit.net, NUnit, MSTest, etc. There are some additional extensions for specific testing frameworks such as xUnit.net that allow further reductions in unit test setup code.

    For more info check out the GitHub site or the Pluralsight course.

    If you like AutoFixture be sure to give open source thanks to Mark Seemann.

    Using AutoFixture To Generate Anonymous Test Data in Web UI Automation With BDDfy and Seleno

    I’m currently working on an AutoFixture Pluralsight course and it got me thinking about using it to generate anonymous test data when writing automated UI tests.

    The basic premise is that in addition to using AutoFixture to generate unit test data, it can also be used to populate UI elements where the specific data values are unimportant.

    If you are unfamiliar with BDDfy and Seleno, they are part of the TestStack project.

    BDDfy allows tests to be written and to produce business readable documentation. Seleno allows the automation of web browsers using Selenium and strongly-typed page object models.

    Example Scenario

    Imagine that we have an (ASP.NET MVC) web site that allows the addition of members of the royal family.

    image

    If we wanted to test different king/queen names (but didn’t care about the Regnal number) we could start off by defining some strongly typed (Seleno) page object models:

    public class HomePage : Page<AddRoyaltyModel>
    {
        public CreatedPage CreateRoyalty(AddRoyaltyModel royal)
        {
            Input.Model(royal);
    
            return Navigate.To<CreatedPage>(By.Id("Create"));
        }
    }
    
    public class CreatedPage : Page
    {
    }
    

     

    (all the examples in this post are fairly quick-and-dirty to demonstrate the AutoFixture involvement)

     

    Next we can write some example in BDDfy (using xUnit.net as the test framework):

    public class AddRoyaltyTests
    {
        private HomePage _home;
        private CreatedPage _confirmationPage;
        public string Name { get; set; }
        public string Number { get; set; }
    
        [Fact]
        public void ShouldAddRoyaltiesWithDifferentNames()
        {
            this.Given(x => GivenIAmStartingANewRoyalAddition())
                .And("And I have entered <name> as the royalty name")
                .And("And I have entered <number> as the regnal number")
                .When(x => WhenIChooseToAddTheNewRoyal())
                .Then(x => ThenIShouldSeeAConfirmationOfTheNewRoyalHavingBeenAdded())
                .WithExamples(new ExampleTable("name", "number")
                              {
                                  {"Richard", "I"},
                                  {"Henry", "I"},
                                  {"Elizabeth", "I"}
                              })
                .BDDfy();            
        }
    
    
    
        private void GivenIAmStartingANewRoyalAddition()
        {
            _home = Host.Instance.NavigateToInitialPage<HomeController, HomePage>(x => x.Index());
        }        
    
        private void WhenIChooseToAddTheNewRoyal()
        {
            var royal = new AddRoyaltyModel
            {
                Name = this.Name,
                RegnalRomanNumeral = this.Number
            };
    
            _confirmationPage = _home.CreateRoyalty(royal);
        }
    
        private void ThenIShouldSeeAConfirmationOfTheNewRoyalHavingBeenAdded()
        {
            Assert.Equal("Created ok", _confirmationPage.Title);
        }
    }
    

    Notice in the preceding code that even though we don’t care about the Regnal number we are still supplying it in the examples. We could just set it manually to a hardcoded value, and in this simple example that might be ok, but if we had a form with many fields then this  will introduce extra work and may make the test less “refactor-safe”.

    The following code shows the removal of the Regnal name/number:

    public class AddRoyaltyTestsUsingAutoFixture
    {
        private HomePage _home;
        private CreatedPage _confirmationPage;
        public string Name { get; set; }
    
        [Fact]
        public void ShouldAddRoyaltiesWithDifferentNames()
        {
            this.Given(x => GivenIAmStartingANewRoyalAddition())
                .And("And I have entered <name> as the royalty name")
                .When(x => WhenIChooseToAddTheNewRoyal())
                .Then(x => ThenIShouldSeeAConfirmationOfTheNewRoyalHavingBeenAdded())
                .WithExamples(new ExampleTable("name")
                              {
                                  "Richard",
                                  "Henry",
                                  "Elizabeth"
                              })
                .BDDfy();            
        }
    
        private void GivenIAmStartingANewRoyalAddition()
        {
            _home = Host.Instance.NavigateToInitialPage<HomeController, HomePage>(x => x.Index());
        }        
    
        private void WhenIChooseToAddTheNewRoyal()
        {
            var fixture = new Fixture();
    
            // Use AutoFixture to create anonymous data for all properties except
            // name which is set to the BDDfy example value
            var royal = fixture.Build<AddRoyaltyModel>()
                .With(x => x.Name, this.Name)
                .Create();
    
            _confirmationPage = _home.CreateRoyalty(royal);
        }
    
        private void ThenIShouldSeeAConfirmationOfTheNewRoyalHavingBeenAdded()
        {
            Assert.Equal("Created ok", _confirmationPage.Title);
        }
    }
    

    In this version, AutoFixture’s Build method is being used to automatically generate test data for all the fields, except the royal name which is set to the name(s) specified in the BDDfy examples.

    Running this test results in the following automation:

    SelenoAutofixture

     

    And produces the following HTML BDDfy report:

    image

    Open Source Thanks

    In a previous post (5 Ways to Contribute to Open Source - It’s Not All Code) I mentioned the simplest way to be involved in open source is to simply Tweet about the project or by saying thanks to the creator/contributors. In the post i said “Most open source authors don’t get paid, saying thankyou is not only nice but is good encouragement for the authors to keep the project going”.

    I’d like to propose the hashtag #opensourcethanks – it would be great to see the Twitter search for this hashtag fill up with thankyous :)

    If you’re benefitting from an open source project, take a minute to send a thankyou Tweet.