AutoFixture is a library that you can use alongside your testing framework to reduce the amount of boilerplate test code you need to write and thus improve your productivity.

At its core, AutoFixture helps you setup your tests by generating anonymous test data for you. This anonymous test data can be used to fulfil non-important boilerplate test data; this is test data that is required for the test to execute but whose value is unimportant.

Take the follow abbreviated test:

[Fact]
public void ManualCreation()
{
    // arrange

    Customer customer = new Customer()
    {
        CustomerName = "Amrit"
    };

    Order order = new Order(customer)
    {
        Id = 42,
        OrderDate = DateTime.Now,
        Items =
                      {
                          new OrderItem
                          {
                              ProductName = "Rubber ducks",
                              Quantity = 2
                          }
                      }
    };


    // act and assert phases...
}

Suppose the previous test code was only creating an Order (with associated Customer) just to fulfil some dependency and the actual Customer and OrderItems did not matter. In this case we could use AutoFixture to generate them for us.

AutoFixture can be installed via NuGet and once installed allows a Fixture instance to be instantiated. This Fixture object can then be used to generate anonymous test data and greatly simplify the arrange phase, as the following test shows:

[Fact]
public void AutoCreation()
{
    // arrange

    var fixture = new Fixture();

    Order order = fixture.Create<Order>();

    // act and assert phases...
}

If we were to debug this test we’d see the following values:

Notice in the preceding screenshot that AutoFixture has created the object graph for us, including the Customer and 3 OrderItem instances.

There’s a lot more to AutoFixture than just this simple example, for example you can combine with the AutoFixture.Xunit2 package to further reduce code:

[Theory, AutoData]
public void SubtractWhenZeroTest(int aPositiveNumber, Calculator sut)
{
    // Act
    sut.Subtract(aPositiveNumber);

    // Assert
    Assert.True(sut.Value < 0);
}

If you want to learn more about how AutoFixture can improve your productivity check out the docs or start watching for free  my AutoFixture Pluralsight course with a free trial:

SHARE:

Sometimes when you write a test with xUnit.net (or other testing frameworks) you may run into problems if UI technologies are involved. This usually relates to the fact that the test must execute using a specific threading model such as single-threaded apartment (STA).

For example suppose you had a WPF app that you wanted to add tests for.

The XAML looks like:

<Window x:Class="WpfApp1.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:WpfApp1"
        mc:Ignorable="d"
        Title="MainWindow" Height="450" Width="800">
    <Grid>
        <TextBlock FontSize="42" Text="{Binding Path=Greeting}" />
    </Grid>
</Window>

And the simple quick and dirty view model class looks like:

namespace WpfApp1
{
    public class MainWindowViewModel
    {
        public string Greeting { get; set; }
    }
}

And  in the MainWindow constructor we set the data context:

public MainWindow()
{
    InitializeComponent();

    var vm = new MainWindowViewModel { Greeting = "Hi there!" };
    DataContext = vm;
}

(This is a very simple demo code with no change notifications etc.)

If you wanted to write an xUnit.net test that instantiates an instance of MainWindow, such as:

[Fact]
[UseReporter(typeof(DiffReporter))]
public void RenderWithViewModel()
{
    var sut = new MainWindow();
    var vm = new MainWindowViewModel { Greeting = "Good day!" };
    sut.DataContext = vm;

    // Test rendering, e.g. using Approval Tests
    WpfApprovals.Verify(sut);
}

If you run this, the test will fail with: System.InvalidOperationException : The calling thread must be STA, because many UI components require this.

Note: this test is using Approval Tests (e.g. [UseReporter(typeof(DiffReporter))]) to render the UI into an image file for approval, you can learn more about Approval Tests with my Pluralsight course. Approval Tests is no related to the threading model requirements.

To enable this test to run you need to instruct xUnit to run the test using an apartment model process (“STA thread”).

Luckily Andrew Arnott has done all the hard work for us and created some custom xUnit.net attributes that allow us to specify what thread/synchronization context to use for a test.

Once the Xunit.StaFact NuGet package has been installed into the test project you can replace the standard [Fact] attribute with [StaFact]. The test will now execute without error:

using ApprovalTests.Reporters;
using ApprovalTests.Wpf;
using Xunit;

namespace WpfApp1.Tests
{
    public class MainWindowShould
    {
        [StaFact]
        [UseReporter(typeof(DiffReporter))]
        public void RenderWithViewModel()
        {
            var sut = new MainWindow();
            var vm = new MainWindowViewModel { Greeting = "Good day!" };
            sut.DataContext = vm;

            // Test rendering, e.g. using Approval Tests
            WpfApprovals.Verify(sut);
        }
    }
}

There are also a number of other attributes such as [WinFormsFact] for use with Windows Forms apps, check out the entire list of attributes in the docs.

If you use this library make sure to say a thankyou to Andrew on Twitter  :)

Also check out my xUnit.net Pluralsight training course or get started watching with a free trial.

SHARE:

One little-known feature of the xUnit.net testing framework is the ability to write test method names in a specific way and then have them converted to a ‘pretty’ version for example in Visual Studio Test Explorer.

Take the following test method:

using ClassLibrary1;
using Xunit;

namespace XUnitTestProject2
{
    public class CalculatorShould
    {
        [Fact]
        public void Add2PositiveNumbers()
        {
            var sut = new Calculator();

            sut.Add(1);
            sut.Add(1);

            Assert.Equal(2, sut.Value);
        }
    }
}

By default, this will look like the following screenshot in Visual Studio Test Explorer:

The first thing that can be modified to to simplify the test method name display to only display the test method name and not the preceding namespace and class name, for example “XUnitTestProject2.CalculatorShould.Add2PositiveNumbers” becomes more simply “Add2PositiveNumbers” by making a simple configuration change.

Displaying Only Test Method Names in xUnit.net Tests

To control the rendering of method names in xUnit.net, the first thing to do is add a new file called “xunit.runner.json” to the root of the test project and set the Copy To Output Directory property to Copy if newer. This will make this file copy to the output bin directory. Once this is done, if you open the project file you should see something like:

<ItemGroup>
  <None Update="xunit.runner.json">
    <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
  </None>
</ItemGroup>

Next, modify the json file to the following:

{
  "$schema": "https://xunit.net/schema/current/xunit.runner.schema.json",
   "methodDisplay": "method"
}

Notice in the preceding json configuration the methodDisplay has been set to “method”, this will prevent the namespace and class being prepended to the method name in Test Explorer.

Now if you head back to Test Explorer you should see the following:

Enabling Pretty Method Names in xUnit.net

In addition to shortening test method name display we can also make use of xUnit.net’s “pretty method display”.

To enable this feature, modify the json configuration file and add the "methodDisplayOptions": "all" configuration as follows:

{
  "$schema": "https://xunit.net/schema/current/xunit.runner.schema.json",
  "methodDisplay": "method",
  "methodDisplayOptions": "all"
}

Now the previous test can be renamed to “Add_2_positive_numbers” as follows:

[Fact]
public void Add_2_positive_numbers()
{
    var sut = new Calculator();

    sut.Add(1);
    sut.Add(1);

    Assert.Equal(2, sut.Value);
}

In test explorer this test method will show up as “Add 2 positive numbers” as the following screenshot shows:

You can use other items in the test method name, for example you can use the monikers eq, ne, lt, le, gt, ge that get replaced with =, !=, <, <=, >, >= respectively, for example a test name of “Have_a_value_eq_0_when_multiplied_by_zero” would be displayed as “Have a value = 0 when multiplied by zero”. Here the eq has been replaced with =.

You can also use ASCII or Unicode escape sequences, for example the test name “Divide_by_U00BD” gets displayed as “Divide by ½” and the test “Email_address_should_only_contain_a_single_U0040” gets displayed as “Email address should only contain a single @”, or “The_U2211_of_1U002C_2_and_3_should_be_6” becomes “The ∑ of 1, 2 and 3 should be 6”:

You could also combine the "methodDisplay": "classAndMethod" to create something like  and the following:

namespace Given_a_cleared_calculator
{
    public class when_a_number_gt_0_is_added
    {
        [Fact]
        public void then_the_value_should_be_gt_0()
        {
            // etc.
        }

        [Fact]
        public void then_the_value_should_eq_the_one_added()
        {
            // etc.
        }
    }
}

This would produce the following tests in Test Explorer:

If you want to learn more about writing tests with xUnit.net check out my Pluralsight course today.

SHARE:

When the result you want to check is a collection, you can use NUnit to assert that it has the expected number of items or is empty, that all items are unique, that specific items do/not exist, and that items exist that satisfy some condition or predicate.

Asserting on the Number of Items in a Collection with NUnit Asserts

var names = new[] { "Sarah", "Amrit", "Amanda", "Sarah" };

Assert.That(names, Has.Exactly(4).Items); // pass
Assert.That(names, Is.Empty); // fail
Assert.That(names, Is.Not.Empty); // pass

Asserting That All Items in a Collection are Unique with NUnit Asserts

Assert.That(names, Is.Unique); // fail - 2 Sarah items exist

Asserting That An Item Does or Does Not Exist in a Collection with NUnit Asserts

Assert.That(names, Contains.Item("Sarah")); // pass

// Alternative syntax
Assert.That(names, Does.Contain("Sarah")); // pass
Assert.That(names, Does.Not.Contain("Arnold")); // pass

Asserting That An Item Appears a Specified Number Of Times in a Collection with NUnit Asserts

Assert.That(names, Has.Exactly(1).EqualTo("Sarah")); // fail
Assert.That(names, Has.Exactly(2).EqualTo("Sarah")); // pass
Assert.That(names, Has.Exactly(2).EqualTo("Sarah")
                      .And.Exactly(1).EqualTo("Amrit")); // pass

Asserting That All Items In a Collections Satisfy a Predicate/Condition with NUnit Asserts

Assert.That(names, Is.All.Not.Null); // pass
Assert.That(names, Is.All.Contains("a")); // fail lowercase a
Assert.That(names, Is.All.Contains("a").IgnoreCase); // pass
Assert.That(names, Is.All.Matches<string>(name => name.ToUpperInvariant().Contains("A"))); // pass
Assert.That(names, Is.All.Matches<string>(name => name.Length > 4)); // pass

Asserting That Only One Item In a Collection Satisfies a Predicate with NUnit Asserts

Assert.That(names, Has.Exactly(1).Matches<string>(name => name.Contains("mri"))); // pass
Assert.That(names, Has.Exactly(1).Matches<string>(name => name.Contains("ara"))); // fail (2 Sarah items exist)

To learn more about NUnit 3 check out my Introduction to .NET Testing with NUnit 3 Pluralsight course to learn everything you need to know to get started, including asserts, categories, data-driven tests, customizing NUnit, and reducing duplicate test code.

You can start watching with a Pluralsight free trial.

SHARE:

When asserting on equality using the EqualConstraint you may not always get the behaviour you want depending on what objects are being asserted on. This can be influenced by whether or not the objects are value or reference types and if the type implements or overrides methods such as IEquatable<T> or object.Equals overrides.

Asserting on Value Type Equality with NUnit

int a = 42;
int b = 42;

Assert.That(a, Is.EqualTo(b)); // pass - values are same, ints are structs with value semantics
Assert.That(a, Is.SameAs(b)); // fail - a and b do not point to the same object in memory

int c = a;

Assert.That(c, Is.EqualTo(a)); // pass - values are same

Asserting on Reference Type Equality with NUnit

By default, 2 instances of a reference type will not pass an equality assert:

class Person
{
    public string Name { get; set; }
}

Person p1 = new Person { Name = "Sarah" };
Person p2 = new Person { Name = "Sarah" };

Assert.That(p1, Is.EqualTo(p2)); // fail, Person is class with reference semantics

Asserting That Two References Point to the Same Object with NUnit

If you want to assert that 2 object references point to the same object you can use the SameAsConstraint:

Assert.That(p1, Is.SameAs(p2)); // fail, p1 and p2 point to different objects in memory Person p3 = p1; Assert.That(p3, Is.SameAs(p1)); // pass, p3 and p1 point to same object in memory Assert.That(p3, Is.Not.SameAs(p2)); // pass, p3 and p2 point to different objects in memory

Customizing Equality Asserts with NUnit

There are a number of ways to influence how NUnit performs equality assertions including implementing IEquatable<T>:

class Employee : IEquatable<Employee>
{
    public string Name { get; set; }

    public bool Equals(Employee other)
    {
        if (other is null)
        {
            return false;
        }

        return Name == other.Name;
    }
}

Employee e1 = new Employee { Name = "Sarah" };
Employee e2 = new Employee { Name = "Sarah" };

Assert.That(e1, Is.EqualTo(e2)); // pass - IEquatable<Employee>.Equals implementation is used

To learn more about NUnit 3 check out my Introduction to .NET Testing with NUnit 3 Pluralsight course to learn everything you need to know to get started, including asserts, categories, data-driven tests, customizing NUnit, and reducing duplicate test code.

You can start watching with a Pluralsight free trial.

SHARE:

If you are asserting that a value is equal to something and you want to specify some tolerance you can do so.

Specifying a Range for Values with NUnit Asserts (e.g. int)

var i = 42;

Assert.That(i, Is.EqualTo(40)); // fail

Assert.That(i, Is.EqualTo(40).Within(2)); // pass

Assert.That(i, Is.EqualTo(40).Within(1)); // fail "Expected: 40 +/- 1"

Specifying a Range as a Percentage with NUnit Asserts

In addition to specifying a range tolerance as a fixed value you can also specify it as a percentage:

Assert.That(i, Is.EqualTo(40).Within(5).Percent); // pass

Assert.That(i, Is.EqualTo(40).Within(4).Percent); // fail "Expected: 40 +/- 4 Percent"

Specifying a Range for DateTime Objects with NUnit Asserts

When working with DateTimes you can specify the tolerance as a TimeSpan instance:

var newYearsDay2019 = new DateTime(2019, 1, 1);

Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(TimeSpan.FromDays(1))); // pass

Or instead of using a TimeSpan you can use one of the convenience modifiers:

Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(1).Days); // pass

Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(24).Hours); // pass
Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(23).Hours); // fail

var numberOfMinutesInADay = 24 * 60;
Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(numberOfMinutesInADay).Minutes); // pass
Assert.That(newYearsDay2019, Is.EqualTo(new DateTime(2019, 1, 2)).Within(numberOfMinutesInADay - 1).Minutes); // fail "Expected: 2019-01-02 00:00:00 +/- 23:59:00"

// Also Within(n).Seconds .Milliseconds and .Ticks

To learn more about NUnit 3 check out my Introduction to .NET Testing with NUnit 3 Pluralsight course to learn everything you need to know to get started, including asserts, categories, data-driven tests, customizing NUnit, and reducing duplicate test code.

You can start watching with a Pluralsight free trial.

SHARE:

Bogus is a lovely library from Brian Chavez to use in automated tests to automatically generate test data of different kinds.

As an example suppose the following class is involved in a unit test:

public class Review
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string Body { get; set; }
    public int Rating { get; set; }
    public DateTimeOffset Created { get; set; }

    public override string ToString()
    {
        return $"{Id} '{Title}'";
    }
}

In a test, a Review instance may need properties populating with values. This could be done manually, for example to check the ToString() implementation:

[Fact]
public void BeRepresentedAsAString()
{
    var sut = new Review
    {
        Id = 42,
        Title = "blah blah"
    };

    Assert.Equal("42 'blah blah'", sut.ToString());
}

Notice in the preceding test, the actual values and title don’t really matter, only the fact that they’re joined as part of the ToString() call. In this example the values for Id and Title could be considered anonymous variable / values in that we don’t really care about them.

The following test uses the Bogus NuGet package and uses its non-fluent facade syntax:

[Fact]
public void BeRepresentedAsAString_BogusFacadeSyntax()
{
    var faker = new Faker("en"); // default en

    var sut = new Review
    {
        Id = faker.Random.Number(),
        Title = faker.Random.String()
    };

    Assert.Equal($"{sut.Id} '{sut.Title}'", sut.ToString());
}

Bogus also has a powerful fluent syntax to define what a test object will look like. To use the fluent version, a Faker<T> instance is created where T is the test object to be configured and created, for example:

[Fact]
public void BeRepresentedAsAString_BogusFluentSyntax()
{
    var reviewFaker = new Faker<Review>()
        .RuleFor(x => x.Id, f => f.Random.Number(1, 10))
        .RuleFor(x => x.Title, f => f.Lorem.Sentence());

    var sut = reviewFaker.Generate(); 

    Assert.Equal($"{sut.Id} '{sut.Title}'", sut.ToString());
}

The first argument to the RuleFor() methods allows the property of the Review object to be selected and the second argument specifies how the property value should be generated. There is a huge range of test data types supported. In the preceding code the Random API is used as well as the Lorem API.

Some examples of the types of auto generated data include:

• Addresses: ZipCode, City, Country, Latitude, etc.
• Commerce: Department name, ProductName, ProductAdjective, Price, etc.
• Company: CompanyName, CatchPhrase, Bs, etc.
• Date: Past, Soon, Between, etc.
• Finance: Account number, TransactionType, Currency, CreditCardNumber, etc.
• Image URL: Random image, Animals image, Nature image, etc.
• Internet: Email, DomainName, Ipv6, Password, etc.
• Lorem: single word, Words, Sentence, Paragraphs, etc.
• Name: FirstName, LastName, etc.
• Rant: Random user review, etc.
• System: FileName, MimeType, FileExt, etc.

Some of the random generated values are quite entertaining, for example Rant.Review() may produce "My co-worker Fate has one of these. He says it looks tall."; Company.Bs() may produce "transition cross-media users", and Company.CatchPhrase() may produce "Face to face object-oriented focus group".

Bogus configuration is quite powerful and allows fairly complex setup as the following code demonstrates:

[Fact]
public void CalculateAverageRatingWhenMultipleReviews()
{
    int rating = 0;

    var reviewFaker = new Faker<Review>()
        .RuleFor(x => x.Id, f => f.Random.Number(1, 10))
        .RuleFor(x => x.Rating, f => rating++);

    var productFaker = new Faker<Product>()
        .RuleFor(x => x.PricePerUnit, f => f.Finance.Amount())
        .RuleFor(x => x.Description, f => f.WaffleText(3))
        .FinishWith((f, x) =>
            {
                reviewFaker.Generate(3).ForEach(r => x.Reviews.Add(r));
            });

    var sut = productFaker.Generate();

    Assert.Equal(1, sut.AverageRating); // (0 + 1 + 2) / 3
}

The WaffleText() API is provided by one of the extensions to Bogus (WaffleGenerator.Bogus) that produces inane looking waffle text such as the following:

The Quality Of Hypothetical Aesthetic

"The parallel personal hardware cannot explain all the problems in maximizing the efficacy of any fundamental dichotomies of the logical psychic principle. Generally the requirements of unequivocal reciprocal individuality is strictly significant. On the other hand the characteristic organizational change reinforces the weaknesses in the evolution of metaphysical terminology over a given time limit. The objective of the explicit heuristic discordance is to delineate the truly global on-going flexibility or the preliminary qualification limit. A priority should be established based on a combination of functional baseline and inevitability of amelioration The Quality Of Hypothetical Aesthetic"

 - Michael Stringer in The Journal of the Proactive Directive Dichotomy (20174U)

structure plan.

To make the main points more explicit, it is fair to say that;
  * the value of the optical continuous reconstruction is reciprocated by what should be termed the sanctioned major issue.
  * The core drivers poses problems and challenges for both the heuristic non-referent spirituality and any discrete or Philosophical configuration mode.
  * an anticipation of the effects of any interpersonal fragmentation reinforces the weaknesses in the explicit deterministic service. This may be due to a lack of a doctrine of the interpersonal quality..
  * any significant enhancements in the strategic plan probably expresses the strategic personal theme. This trend may dissipate due to the personal milieu.

 firm assumptions about ideal major monologism evinces the universe of attitude.

The Flexible Implicit Aspiration.

Within current constraints on manpower resources, any consideration of the lessons learnt can fully utilize what should be termed the two-phase multi-media program.

For example, the assertion of the importance of the integration of doctrine of the prime remediation with strategic initiatives cannot be shown to be relevant. This is in contrast to the strategic fit.

To learn more about Bogus head over to the documentation.

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

In the (relatively) distant past, MSTest was often used by organizations because it was provided by Microsoft “in the box” with Visual Studio/.NET. Because of this, some organizations trusted MSTest over open source testing frameworks such as NUnit. This was at a time when the .NET open source ecosystem was not as advanced as it is today and before Microsoft began open sourcing some of their own products.

Nowadays MSTest is cross-platform and open source and is known as MSTest V2, and as the documentation states: “is a fully supported, open source and cross-platform implementation of the MSTest test framework with which to write tests targeting .NET Framework, .NET Core and ASP.NET Core on Windows, Linux, and Mac.”.

MSTest V2 provides typical assert functionality such as asserting on the values of: strings, numbers, collections, thrown exceptions, etc. Also like other testing frameworks, MSTest V2 allows the customization of the test execution lifecycle such as the running of additional setup code before each test executes. The framework also allows the creation of data driven tests (a single test method executing  multiple times with different input test data) and the ability to extend the framework with custom asserts and custom test attributes.

You can find out more about MSTest V2 at the GitHub repository, the documentation, or check out my Pluralsight course: Automated Testing with MSTest V2.

You can start watching with a Pluralsight free trial.

SHARE:

xUnit.net is a testing framework that can be used to write automated tests for .NET (full) framework and also .NET Core.

To get started, first create a .NET Core application, in the following example a .NET Core console app.

A testing project can now be added to the solution:

This test project will come pre-configured with the relevant NuGet packages installed to start writing test code, though you may want to update the pre-configured packages to the newest NuGet versions.

The xUnit Test Project template will also create the following default test class:

using System;
using Xunit;

namespace ConsoleCalculator.Tests
{
    public class UnitTest1
    {
        [Fact]
        public void Test1()
        {

        }
    }
}

Notice in the preceding code, the Test1 method is decorated with the [Fact] attribute. This is an xUnit.net attribute that tells a test runner that it should execute the method, treat it as a test, and report on if the test passed or not.

Next add a project reference from the test project to the project that contains the code that is to be tested, this gives the test project access to the production code.

In the production project, the following class can be added:

namespace ConsoleCalculator
{
    public class Calculator
    {
        public int Add(int a, int b)
        {            
            return a + b;
        }
    }
}

Now the test class can be renamed (for example to “CalculatorTests”) and the test method changed to create a test:

using Xunit;

namespace ConsoleCalculator.Tests
{
    public class CalculatorTests
    {
        [Fact]
        public void ShouldAddTwoNumbers()
        {
            Calculator calculator = new Calculator();

            int result = calculator.Add(7, 3);

            Assert.Equal(10, result);
        }
    }
}

In the preceding code, once again the [Fact] attribute is being used, then the thing being tested is created (the Calculator class instance). The next step is to perform some kind of action on the thing being tested, in this example calling the Add method. The final step is to signal to the test runner if the test has passed or not, this is done by using one of the many xUnit.net Assert methods; in the preceding code the Assert.Equal method is being used. The first parameter is the expected value of 10, the second parameter is the actual value produced by the code being tested. So if  result is 10 the test will pass, otherwise it will fail.

One way to execute tests is to use Visual Studio’s Test Explorer which can be found under the Test –> Windows –> Test Explorer menu item. Once the test project is built, the test will show up and can be executed as the following screenshot shows:

To learn more about how to get started testing code with xUnit.net check out my Pluralsight course

You can start watching with a Pluralsight free trial.

SHARE:

It’s often useful to take a step back and look at the bigger picture, this is true in different aspects of life such as health or wealth or relationships, and is also true of software development.

When it comes to creating automated tests (as with other aspects of software development) dogmatism and absolutist schools of though can exist.

As with all things, the decision to write tests (and how many tests, what type of tests, test coverage aims, etc.) ultimately should boil down to one question: do they add value to what you are doing?

To be clear, I absolutely believe in the creation of automated tests in many cases, however it is good to not be dogmatic. For example if there is a niche market that is ripe for capitalizing on, and time-to-market is the most important thing to capture this market, then an extensive suite of automated tests may slow down getting to that initial release. This of course assumes that the potential market will have some tolerance for software defects. It also depends on what the product is; medical life-critical software is probably going to have a higher quality requirement than a social media app for example. This can be a trade-off however with shorter term delivery speeds being quicker but at the expense of delivery speed in the long term, if you’re overwhelmed fixing production outages you have very little time to add new features/value.

Another aspect to consider is that of risk. What are the risks associated with defects in the software making their way into production? Different features/application areas may also have different risk profiles; for example  the “share on social media” feature may not be deemed as important as a working shopping cart. It’s also important to remember that risk is not just monetary, in the previous example a broken “share on social media” feature may bring the business into disrepute, aka “reputational risk”.

When it comes to the myriad of different types of tests (unit, integration, subcutaneous, etc.) the testing pyramid is an oft-quoted model of how many of each type of tests to have in the test suite. While the testing pyramid may be of great help for someone new to automated testing to help them learn and navigate their initial steps, as experience grows the model may no longer be optimal to some of the projects that are being worked on. Beyond the testing pyramid the different aspects of test types can be considered such as execution speed, breadth/depth, reliability/brittleness etc.

Automated tests also do not exist in and of themselves, they are part of a bigger set of processes/considerations such as pair programming, code reviews, good management, well-understood requirements, good environment management/DevOps, etc.

If you want to take a step back and look at the big picture, or know a developer or manager who wants to understand the trade-offs/benefits check out my Testing Automation: The Big Picture Pluralsight course.

You can start watching with a Pluralsight free trial.

SHARE: