Hook Execution Order in SpecFlow 2

SpecFlow hooks allow additional code to be executed before and after various stages of the test execution lifecycle, for example running additional setup code before each scenario executes.

If there are multiple of the same type of hook specified, by default the execution order of the hook methods is unspecified. For example the following code has three [BeforeStep] hook methods that could be executed in any order before every step of the scenario executes:

[BeforeStep]
public void BeforeHook1()
{
}

[BeforeStep]
public void BeforeHook2()
{
}

[BeforeStep]
public void BeforeHook3()
{
}

To ensure these hook methods are executed in a specified order, the hook attributes allow an optional order to be specified. When there are multiple of the same hook methods defined, the lowest order values execute before the higher order methods:

[BeforeStep(Order = 100)]
public void BeforeHook1()
{
}

[BeforeStep(Order = 200)]
public void BeforeHook2()
{
}

[BeforeStep(Order = 300)]
public void BeforeHook3()
{
}

The values of the Order property are arbitrary, you may use whatever values you wish, though it is sensible to allow some “wriggle room” for future additional steps by working in increments of 10 or 100 for example.

The following code illustrates another example where the execution order of hooks is important; the database should be reset first before test users are added:

[Binding]
public class Hooks
{
    [BeforeScenario(Order = 100)]
    public void ResetDatabase()
    {
    }

    [BeforeScenario(Order = 200)]
    public void AddTestUsersToDatabase()
    {
    }        
}

To see hook ordering in action, check out my Pluralsight course: Business Readable Automated Tests with SpecFlow 2.0.

New Pluralsight Course: Business Readable Automated Tests with SpecFlow 2.0

My newest Pluralsight course was just published. Business Readable Automated Tests with SpecFlow 2.0 teaches how to create tests that the business can read, understand, and contribute to. These “English-like” tests (other spoken languages are supported) can be executed by writing test code that is associated with the “English-like” steps. Because the tests sit alongside the source code, they can become living (executable) documentation for the system, as opposed to an out-of-date Word document somewhere on the network for example. Check out the course here.

Improving Test Code Readability and Assert Failure Messages with Shouldly

Shouldly is an open source library that aims to improve the assert phase of tests; it does this in two ways. The first is offering a more “fluent like” syntax that for the most part leverages extension methods and obviates the need to keep remembering which parameter is the expected or actual as with regular Assert.Xxxx(1,2) methods. The second benefit manifests itself when tests fail; Shouldly outputs more readable, easily digestible test failure messages.

Failure Message Examples

The following are three failure messages from tests that don’t use Shouldly and instead use the assert methods bundled with the testing framework (NUnit, xUnit.net, etc):

  • “Expected: 9  But was:  5”
  • “Assert.NotNull() Failure”
  • “Not found: Monday In value:  List<String> ["Tuesday", "Wednesday", "Thursday"]”

In each of the preceding failure messages, there is not much helpful context in the failure message.

Compare the above to the following equivalent Shouldly failure messages:

  • “schedule.TotalHours should be 9 but was 5”
  • “schedule.Title should not be null or empty”
  • “schedule.Days should contain "Monday" but does not”

Notice the additional context in these failure messages. In each case here, Shouldly is telling us the name of the variable in the test code (“schedule”) and the name of the property/field being asserted (e.g. “Total Hours”).

Test Code Readability

For the preceding failure messages, the following test assert code is used (notice the use of the Shouldly extension methods):

  • schedule.TotalHours.ShouldBe(9);
  • schedule.Title.ShouldNotBeNullOrEmpty();
  • schedule.Days.ShouldContain("Monday");

In these examples there is no mistaking an actual value parameter for an expected value parameter and the test code reads more “fluently” as well.

To find out more about Shouldly check out the project on GitHub, install via NuGet, or checkout my Better Unit Test Assertions with Shouldly Pluralsight course.

Using Cyclomatic Complexity as an Indicator of Clean Code

Cyclomatic complexity is one way to measure how complicated code is, it measures how complicated the structure of the code is and by extension how likely it may be to attract bugs or additional cost in maintenance/readability.

The calculated value for the cyclomatic complexity indicates how many different paths through the code there are. This means that lower numbers are better than higher numbers.

Clean code is likely to have lower cyclomatic complexity that dirty code. High cyclomatic complexity increases the risk of the presence of defects in the code due to increased difficulty in its testability, readability, and maintainability.

Calculating Cyclomatic Complexity in Visual Studio

To calculate the cyclomatic complexity, go to the the Analyze menu and choose Calculate Code Metrics for Solution (or for a specific project within the solution).

This will open the Code Metrics Results window as seen in the following screenshot.

image

Take the following code (in a project called ClassLibrary1):

namespace ClassLibrary1
{
    public class Class1
    {
    }
}

If we expand the results in the Code Metrics Window we can drill down into classes and right down to individual methods as in the following screenshot.

image

More...

3 Ways to Pass State Between SpecFlow Step Definitions

Sometimes we need one step definition to know about parameter data that was passed to previous steps.

There’s 3 ways main ways to do this, each with various considerations.

Private Field in Binding Class

This is the simplest approach. Simply add a private field to your [Binding] class containing your step definitions.

In a step, just grab the parameter data and store it in this private field.

In later steps, retrieve the data from this private field.

This approach is simple but breaks down when we start to refactor our step definitions into multiple step definition classes; the private field data is not accessible between steps in different binding classes.

ScenarioContext

We can store data for the current executing scenario using the ScenarioContext class.

There's a number of ways to use it, but essentially it’s a dictionary that we can store arbitrary data and retrieve it by a key:

ScenarioContext.Current["age"] = 42;

This approach can be used even if we refactor step definitions out across multiple binding classes. The scenario context is shared between all steps for the lifetime that a scenario is executing.

This approach can start to get a little ugly, we also have to make sure our dictionary keys are correct in all the steps where we set or retrieve data.

By default, it’s also weakly-typed – the dictionary just stores objects so we can end up with casts every time we retrieve data.

Context Injection

The third method (which I wrote about previously) lets us take a dependency injection style approach, and define POCOs to represent the data (in a strongly typed way) that automatically gets injected into each of our binding classes.

Like ScenarioContext, this approach also works across multiple binding classes. Unlike ScenarioContext, we don’t have to worry about dictionary keys or casting.

 

If you’re new to SpecFlow and Gherkin, check out my Pluralsight course:  Automated Acceptance Testing with SpecFlow and Gherkin or if you understand the basics or have been using it for a while, check out  SpecFlow Tips and Tricks course to learn how to create more maintainable SpecFlow test automation solutions.

New Pluralsight Course: SpecFlow Tips and Tricks

My latest Pluralsight course was released today: SpecFlow Tips and Tricks is a short course to help those who are relatively new to SpecFlow and those who have been using it for a while and want to create more maintainable testing solutions.

The course focuses on 3 main areas:

  • Steps and Bindings
  • Hooks and Scoped Bindings
  • Step Parameters and Data Tables

This course is a loose follow-up to my previous Automated Acceptance Testing with SpecFlow and Gherkin course that covers the Gherkin DSL and other fundamental SpecFlow topics.

You can check out the course on the Pluralsight website or on my author page.

SpecFlow Cheat Sheet

SpecFlow is a tool that allows the writing of business-readable tests that can then be automated in code. If you’re new to SpecFlow check out my Pluralsight course or the resources page.

The cheat sheet below shows the basics of different step binding styles, the hooks that can be used to execute additional code, and scoped bindings.

More...

Advanced SpecFlow: Custom Conversion of Step Parameters and Data Tables with [StepArgumentTransformation]

SpecFlow is a tool that allows the writing of business-readable tests that can then be automated in code. If you’re new to SpecFlow check out my Pluralsight course or the resources page to get up to speed before looking at these more advanced topics.

SpecFlow will attempt to perform some conversions of step arguments as specified in the documentation.

We can also tell SpecFlow to use our own custom conversions. Take the feature below:

Feature: SomeFeature

Scenario: Update User Password
    Given The original user was created over 10 days ago
    When I update the users password to "some password"
    Then I should see the following errors
        | ErrorCode         | ErrorDescription          |
        | 442               | Out of date user          |
        | 28                | Changes were not saved    |

From this we generate some basic step definitions:

[Given]
public void Given_The_original_user_was_created_over_DAYS_days_ago(int days)
{
}

[When]
public void When_I_update_the_users_password_to_NEWPASSWORD(string newPassword)
{
}
       
[Then]
public void Then_I_should_see_the_following_errors(Table table)
{            
}

Automatically Parsing 10 Days Ago into a DateTime

In the Given step we get an int specifying how many days ago the user was created. As it’s the Given step we need to put the system into a state where the user creation date, for example held as a DateTime in the user database, is 10 days old. If this is just a single step then we can simply write the code to subtract a number of days from todays date in the step itself. Note: this kind of date based testing where we rely on “now” may prove brittle, imagine the a Given ran just before midnight, but the Then gets executed after midnight, i.e. the next day.

More...