Lifelike Test Data Generation with Bogus

Bogus is a lovely library from Brian Chavez to use in automated tests to automatically generate test data of different kinds.

As an example suppose the following class is involved in a unit test:

public class Review
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string Body { get; set; }
    public int Rating { get; set; }
    public DateTimeOffset Created { get; set; }

    public override string ToString()
    {
        return $"{Id} '{Title}'";
    }
}

In a test, a Review instance may need properties populating with values. This could be done manually, for example to check the ToString() implementation:

[Fact]
public void BeRepresentedAsAString()
{
    var sut = new Review
    {
        Id = 42,
        Title = "blah blah"
    };

    Assert.Equal("42 'blah blah'", sut.ToString());
}

Notice in the preceding test, the actual values and title don’t really matter, only the fact that they’re joined as part of the ToString() call. In this example the values for Id and Title could be considered anonymous variable / values in that we don’t really care about them.

The following test uses the Bogus NuGet package and uses its non-fluent facade syntax:

[Fact]
public void BeRepresentedAsAString_BogusFacadeSyntax()
{
    var faker = new Faker("en"); // default en

    var sut = new Review
    {
        Id = faker.Random.Number(),
        Title = faker.Random.String()
    };

    Assert.Equal($"{sut.Id} '{sut.Title}'", sut.ToString());
}

Bogus also has a powerful fluent syntax to define what a test object will look like. To use the fluent version, a Faker<T> instance is created where T is the test object to be configured and created, for example:

[Fact]
public void BeRepresentedAsAString_BogusFluentSyntax()
{
    var reviewFaker = new Faker<Review>()
        .RuleFor(x => x.Id, f => f.Random.Number(1, 10))
        .RuleFor(x => x.Title, f => f.Lorem.Sentence());

    var sut = reviewFaker.Generate(); 

    Assert.Equal($"{sut.Id} '{sut.Title}'", sut.ToString());
}

The first argument to the RuleFor() methods allows the property of the Review object to be selected and the second argument specifies how the property value should be generated. There is a huge range of test data types supported. In the preceding code the Random API is used as well as the Lorem API.

Some examples of the types of auto generated data include:

  • Addresses: ZipCode, City, Country, Latitude, etc.
  • Commerce: Department name, ProductName, ProductAdjective, Price, etc.
  • Company: CompanyName, CatchPhrase, Bs, etc.
  • Date: Past, Soon, Between, etc.
  • Finance: Account number, TransactionType, Currency, CreditCardNumber, etc.
  • Image URL: Random image, Animals image, Nature image, etc.
  • Internet: Email, DomainName, Ipv6, Password, etc.
  • Lorem: single word, Words, Sentence, Paragraphs, etc.
  • Name: FirstName, LastName, etc.
  • Rant: Random user review, etc.
  • System: FileName, MimeType, FileExt, etc.

Some of the random generated values are quite entertaining, for example Rant.Review() may produce "My co-worker Fate has one of these. He says it looks tall."; Company.Bs() may produce "transition cross-media users", and Company.CatchPhrase() may produce "Face to face object-oriented focus group".

Bogus configuration is quite powerful and allows fairly complex setup as the following code demonstrates:

[Fact]
public void CalculateAverageRatingWhenMultipleReviews()
{
    int rating = 0;

    var reviewFaker = new Faker<Review>()
        .RuleFor(x => x.Id, f => f.Random.Number(1, 10))
        .RuleFor(x => x.Rating, f => rating++);

    var productFaker = new Faker<Product>()
        .RuleFor(x => x.PricePerUnit, f => f.Finance.Amount())
        .RuleFor(x => x.Description, f => f.WaffleText(3))
        .FinishWith((f, x) =>
            {
                reviewFaker.Generate(3).ForEach(r => x.Reviews.Add(r));
            });

    var sut = productFaker.Generate();

    Assert.Equal(1, sut.AverageRating); // (0 + 1 + 2) / 3
}

The WaffleText() API is provided by one of the extensions to Bogus (WaffleGenerator.Bogus) that produces inane looking waffle text such as the following:

The Quality Of Hypothetical Aesthetic

"The parallel personal hardware cannot explain all the problems in maximizing the efficacy of any fundamental dichotomies of the logical psychic principle. Generally the requirements of unequivocal reciprocal individuality is strictly significant. On the other hand the characteristic organizational change reinforces the weaknesses in the evolution of metaphysical terminology over a given time limit. The objective of the explicit heuristic discordance is to delineate the truly global on-going flexibility or the preliminary qualification limit. A priority should be established based on a combination of functional baseline and inevitability of amelioration The Quality Of Hypothetical Aesthetic"

 - Michael Stringer in The Journal of the Proactive Directive Dichotomy (20174U)

structure plan.

To make the main points more explicit, it is fair to say that;
  * the value of the optical continuous reconstruction is reciprocated by what should be termed the sanctioned major issue.
  * The core drivers poses problems and challenges for both the heuristic non-referent spirituality and any discrete or Philosophical configuration mode.
  * an anticipation of the effects of any interpersonal fragmentation reinforces the weaknesses in the explicit deterministic service. This may be due to a lack of a doctrine of the interpersonal quality..
  * any significant enhancements in the strategic plan probably expresses the strategic personal theme. This trend may dissipate due to the personal milieu.

 firm assumptions about ideal major monologism evinces the universe of attitude.

The Flexible Implicit Aspiration.

Within current constraints on manpower resources, any consideration of the lessons learnt can fully utilize what should be termed the two-phase multi-media program.

For example, the assertion of the importance of the integration of doctrine of the prime remediation with strategic initiatives cannot be shown to be relevant. This is in contrast to the strategic fit.

To learn more about Bogus head over to the documentation.

Testing for Thrown Exceptions in xUnit.net

When writing tests it is sometimes useful to check that the correct exceptions are thrown at the expected time.

When using xUnit.net there are a number of ways to accomplish this.

As an example consider the following simple class:

public class TemperatureSensor
{
    bool _isInitialized;

    public void Initialize()
    {
        // Initialize hardware interface
        _isInitialized = true;
    }

    public int ReadCurrentTemperature()
    {
        if (!_isInitialized)
        {
            throw new InvalidOperationException("Cannot read temperature before initializing.");
        }

        // Read hardware temp
        return 42; // Simulate for demo code purposes
    }        
}

The first test we could write against the preceding class is to check the “happy path”:

[Fact]
public void ReadTemperature()
{
    var sut = new TemperatureSensor();

    sut.Initialize();

    var temperature = sut.ReadCurrentTemperature();

    Assert.Equal(42, temperature);
}

Next a test could be written to check that if the temperature is read before initializing the sensor, an exception of type InvalidOperationException is thrown. To do this the xUnit.net Assert.Throws method can be used. When using this method the generic type parameter indicates the type of expected exception and the method parameter takes an action that should cause this exception to be thrown, for example:

[Fact]
public void ErrorIfReadingBeforeInitialized()
{
    var sut = new TemperatureSensor();

    Assert.Throws<InvalidOperationException>(() => sut.ReadCurrentTemperature());
}

In the preceding test, if an InvalidOperationException is not thrown when the ReadCurrentTemperature method is called the test will fail.

The thrown exception can also be captured in a variable to make further asserts against the exception property values, for example:

[Fact]
public void ErrorIfReadingBeforeInitialized_CaptureExDemo()
{
    var sut = new TemperatureSensor();

    var ex = Assert.Throws<InvalidOperationException>(() => sut.ReadCurrentTemperature());

    Assert.Equal("Cannot read temperature before initializing.", ex.Message);
}

The Assert.Throws method expects the exact type of exception and not derived exceptions. In the case where you want to also allow derived exceptions, the Assert.ThrowsAny method can be used.

Similar exception testing features also exist in MSTest and NUnit frameworks.

To learn more about using exceptions to handle errors in C#, check out my Error Handling in C# with Exceptions Pluralsight course.

Customizing C# Object Member Display During Debugging

In a previous post I wrote about Customising the Appearance of Debug Information in Visual Studio with the DebuggerDisplay Attribute. In addition to controlling the high level  debugger appearance of an object we can also exert a lot more control over how the object appears in the debugger by using the DebuggerTypeProxy attribute.

For example, suppose we have the following (somewhat arbitrary) class:

class DataTransfer
{
    public string Name { get; set; }
    public string ValueInHex { get; set; }
}

By default, in the debugger it would look like the following:

Default Debugger View

To customize the display of the object members, the DebuggerTypeProxy attribute can be applied.

The first step is to create a class to act as a display proxy. This class takes the original object as part of the constructor and then exposes the custom view via public properties.

For example, suppose that we wanted a decimal display of the hex number that originally is stored in a string property in the original DataTransfer object:

class DataTransferDebugView
{
    private readonly DataTransfer _data;

    public DataTransferDebugView(DataTransfer data)
    {
        _data = data;
    }

    public string NameUpper => _data.Name.ToUpperInvariant();
    public string ValueDecimal
    {
        get
        {
            bool isValidHex = int.TryParse(_data.ValueInHex, System.Globalization.NumberStyles.HexNumber, null, out var value);

            if (isValidHex)
            {
                return value.ToString();
            }

            return "INVALID HEX STRING";
        }
    }
}

Once this view object is defined, it can be selected by decorating the DataTransfer class with the DebuggerTypeProxy attribute as follows:

[DebuggerTypeProxy(typeof(DataTransferDebugView))]
class DataTransfer
{
    public string Name { get; set; }
    public string ValueInHex { get; set; }
}

Now in the debugger, the following can be seen:

Custom debug view showing hex value as a decimal

Also notice in the preceding image, that the original object view is available by expanding the Raw View section.

To learn more about C# attributes and even how to create your own custom ones, check out my C# Attributes: Power and Flexibility for Your Code course at Pluralsight.

MSTest V2

In the (relatively) distant past, MSTest was often used by organizations because it was provided by Microsoft “in the box” with Visual Studio/.NET. Because of this, some organizations trusted MSTest over open source testing frameworks such as NUnit. This was at a time when the .NET open source ecosystem was not as advanced as it is today and before Microsoft began open sourcing some of their own products.

Nowadays MSTest is cross-platform and open source and is known as MSTest V2, and as the documentation states: “is a fully supported, open source and cross-platform implementation of the MSTest test framework with which to write tests targeting .NET Framework, .NET Core and ASP.NET Core on Windows, Linux, and Mac.”.

MSTest V2 provides typical assert functionality such as asserting on the values of: strings, numbers, collections, thrown exceptions, etc. Also like other testing frameworks, MSTest V2 allows the customization of the test execution lifecycle such as the running of additional setup code before each test executes. The framework also allows the creation of data driven tests (a single test method executing  multiple times with different input test data) and the ability to extend the framework with custom asserts and custom test attributes.

You can find out more about MSTest V2 at the GitHub repository, the documentation, or check out my Pluralsight course: Automated Testing with MSTest V2.

Prevent Secrets From Accidentally Being Committed to Source Control in ASP.NET Core Apps

One problem when dealing with developer “secrets” in development is accidentally checking them into source control. These secrets could be connection strings to dev resources, user IDs, product keys, etc.

To help prevent this from accidentally happening, the secrets can be stored outside of the project tree/source control repository. This means that when the code is checked in, there will be no secrets in the repository.

Each developer will have their secrets stored outside of the project code. When the app is run, these secrets can be retrieved at runtime from outside the project structure.

One way to accomplish this in ASP.NET Core  projects is to make use of the Microsoft.Extensions.SecretManager.Tools NuGet package to allow use of the command line tool. (also if you are targeting .NET Core 1.x , install the Microsoft.Extensions.Configuration.UserSecrets NuGet package).

Setting Up User Secrets

After creating a new ASP.NET Core project, add a tools reference to the NuGet package to the project, this will add the following item in the project file:

<DotNetCliToolReference Include="Microsoft.Extensions.SecretManager.Tools" Version="2.0.0" />

Build the project and then right click the project and you will see a new item called “Manage User Secrets” as the following screenshot shows:

Managing user secrets in Visual Studio

Clicking menu item will open a secrets.json file and also add an element named UserSecretsId to the project file. The content of this element is a GUID, the GUID is arbitrary but should be unique for each and every project.

<UserSecretsId>c83d8f04-8dba-4be4-8635-b5364f54e444</UserSecretsId>

User secrets will be stored in the secrets.json file which will be in %APPDATA%\Microsoft\UserSecrets\<user_secrets_id>\secrets.json on Windows or ~/.microsoft/usersecrets/<user_secrets_id>/secrets.json on Linux and macOS. Notice these paths contain the user_secrets_id that matches the GUID in the project file. In this way each project has a separate set of user secrets.

The secrets.json file contains key value pairs.

Managing User Secrets

User secrets can be added by editing the json file or by using the command line (from the project directory).

To list user secrets type: dotnet user-secrets list At the moment his will return “No secrets configured for this application.”

To set (add) a secret: dotnet user-secrets set "Id" "42"

The secrets.json file now contains the following:

{
  "Id": "42"
}

Other dotnet user-secrets  commands include:

  • clear - Deletes all the application secrets
  • list - Lists all the application secrets
  • remove - Removes the specified user secret
  • set - Sets the user secret to the specified value

Accessing User Secrets in Code

To retrieve users secrets, in the startup class, access the item by key, for example:

public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc();

    var secretId = Configuration["Id"]; // returns 42
}

One thing to bear in mind is that secrets are not encrypted in the secrets.json file, as the documentation states: “The Secret Manager tool doesn't encrypt the stored secrets and shouldn't be treated as a trusted store. It's for development purposes only. The keys and values are stored in a JSON configuration file in the user profile directory.” & “You can store and protect Azure test and production secrets with the Azure Key Vault configuration provider.”

There’s a lot more information in the documentation and if you plan to use this tool you should read through it.

Testing Precompiled Azure Functions Overview

Just because serverless allows us to quickly deploy value, it doesn’t mean that testing is now obsolete. (click to Tweet)

If we’re using Azure Functions as our serverless platform we can write our code (for example C#) and test it before deploying to Azure. In this case we’re talking about precompiled Azure Functions as opposed to earlier incarnations of Azure Functions that used .csx script files.

Working with precompiled functions means the code can be developed and tested on a local development machine. The code we write is familiar C# with some additional attributes to integrate the code with the Azure Functions runtime.

Because the code is just regular C#, we can use familiar testing tools such as MSTest, xUnit.net, or NUnit. Using these familiar testing frameworks it’s possible to write tests that operate at different levels of granularity.

One way to categorize these tests are into:

  • Unit tests to check core business logic/value
  • Integration tests to check function run methods are operating correctly
  • End-to-end workflow tests that check multiple functions working together

To enable effective automated testing it may be necessary to write functions in such a way as to make them testable, for example by allowing function run method dependencies to be automatically injected at runtime, whereas at test time mock versions can be supplied for example using a framework such as AzureFunctions.Autofac.

There are other tools that allow us to more easily test functions locally such as the local functions runtime and the Azure storage emulator.

To learn more about using these tools and techniques to test Azure Functions, check out my Pluralsight course Testing Precompiled Azure Functions: Deep Dive.

Stack Overflow Developer Survey 2018 Overview for .NET Developers

The 2018 Stack Overflow Developer Survey was recently released.

This article summarizes some interesting points that .NET developers may find interesting, in additional to some other general items of potential  interest.

.NET Points of Interest

  • C# is the 8th most popular programming language among professional developers at 35.3%.
  • TypeScript is the 12th most popular language among profession developers at 18.3%
  • VB.NET is the 18th most popular programming language at 6.9%.
  • .NET Core is the 4th most popular framework among professional developers at 27.2%.
  • SQL Server is the 2nd most used database among professional developers  at 41.6%.
  • Windows Desktop or Server is the 2nd most developed-for platform among professional developers  at 35.2%.
  • Azure is the 10th most developed-for platform among professional developers at 11.4%.
  • TypeScript is the 4th most loved language at 67%.
  • C# is the 8th most loved language.
  • VB.NET is the 4th most dreaded language
  • .NET Core is the 5th most loved framework, the 8th most dreaded framework, and the 5th most wanted framework.
  • Azure is the 5th most loved database (Tables, CosmosDB, SQL, etc.)
  • SQL Server is the 10th most loved database.
  • Visual Studio Code and Visual Studio are the top two most popular development environments respectively (among all respondents).
  • Professional developers primarily use Windows (49.4%) as their development operating system.
  • F# is associated with the highest salary worldwide, with C# 16th highest.
  • .NET development technologies cluster around C#, Azure, .NET Core, SQL Server etc.

General Points of Interest

  • 52.7% of developers spend 9-12 hours per day on a computer.
  • 37.4% of developers don’t typically exercise.
  • 93.1% of professional developers identify as male.
  • 74.3% of professional developers identify as white or of European descent.
  • 85.9% of professional developers use Agile development methodologies.
  • 88.4% of professional developers use Git for version control.

The survey offers a wealth of additional information and you can find the full set of results over at Stack Overflow.

Automatic Input Blob Binding in Azure Functions from Queue Trigger Message Data

Reading additional blob content when an Azure Function is triggered can be accomplished by using an input blob binding by defining a parameter in the function run method and decorating it with the [Blob] attribute.

For example, suppose you have a number of blobs that need converting in some way. You could initiate a process whereby the list of blob files that need processing are added to a storage queue. Each queue message contains the name of the blob that needs processing. This would allow the conversion function to scale out to convert multiple blobs in parallel.

The following code demonstrates one approach to do this. The code is triggered from a queue message that contains text representing the input bob filename that needs reading, converting, and then outputting to an output blob container.

using System.IO;
using Microsoft.Azure.WebJobs;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;

namespace FunctionApp1
{
    public static class ConvertNameCase
    {
        [FunctionName("ConvertNameCase")]
        public static void Run([QueueTrigger("capitalize-names")]string inputBlobPath)
        {
            string originalName = ReadInputName(inputBlobPath);

            var capitalizedName = originalName.ToUpperInvariant();

            WriteOutputName(inputBlobPath, capitalizedName);
        }
        
        private static string ReadInputName(string blobPath)
        {
            CloudStorageAccount account = CloudStorageAccount.DevelopmentStorageAccount;
            CloudBlobClient blobClient = account.CreateCloudBlobClient();
            CloudBlobContainer container = blobClient.GetContainerReference("names-in");

            var blobReference = container.GetBlockBlobReference(blobPath);

            string originalName = blobReference.DownloadText();

            return originalName;
        }

        private static void WriteOutputName(string blobPath, string capitalizedName)
        {
            CloudStorageAccount account = CloudStorageAccount.DevelopmentStorageAccount;
            CloudBlobClient blobClient = account.CreateCloudBlobClient();
            CloudBlobContainer container = blobClient.GetContainerReference("names-out");

            CloudBlockBlob cloudBlockBlob = container.GetBlockBlobReference(blobPath);
            cloudBlockBlob.UploadText(capitalizedName);            
        }

    }
}

In the preceding code, there is a lot of blob access code (which could be refactored). This function could however be greatly simplified by the use of one of the built-in binding expression tokens. Binding expression tokens can be used in binding expressions and are specified inside a pair of curly braces {…}. The {queueTrigger} binding token will extract the content of the incoming queue message that triggered a function.

For example, the code could be refactored as follows:

using System.IO;
using Microsoft.Azure.WebJobs;

namespace FunctionApp1
{
    public static class ConvertNameCase
    {
        [FunctionName("ConvertNameCase")]
        public static void Run(
        [QueueTrigger("capitalize-names")]string inputBlobPath,
        [Blob("names-in/{queueTrigger}", FileAccess.Read)] string originalName,
        [Blob("names-out/{queueTrigger}")] out string capitalizedName)
        {
                capitalizedName = originalName.ToUpperInvariant();         
        }
}

In the preceding code, the two [Blob] binding paths make use of the {queueTrigger} token. When the function is triggered, the queue message contains the name of the file to be processed. In the two [Blob] binding expressions, the {queueTrigger} token part will automatically be replaced with the text contents of the incoming message. For example if the message contained the text “File1.txt” then the two blob bindings would be set to names-in/File1.txt and names-out/File1.txt respectively. This means the input blob nameBlob string will automatically be read when the function is triggered,

To learn more about creating precompiled Azure Functions in Visual Studio, check out my Writing and Testing Precompiled Azure Functions in Visual Studio 2017 Pluralsight course.

Dynamic Binding in Azure Functions with Imperative Runtime Bindings

When creating precompiled Azure Functions, bindings (such as a blob output bindings) can be declared in the function code, for example the following code defines a blob output binding:

[Blob("todo/{rand-guid}")]

This binding creates a new blob with a random (GUID) name. This style of binding is called declarative binding, the binding details are declared as part of the binding attribute.

In addition to declarative binding, Azure Functions also offers imperative binding. With this style of binding, the details of the binding can be chosen at runtime. These details could be derived from the incoming function trigger data or from an external place such as a configuration value or database item

To create imperative bindings, rather than using a specific binding attribute, a parameter of type IBinder is used. At runtime, a binding can be created (such as a blob binding, queue binding, etc.) using this IBinder. The Bind<T> method of the IBinder can be used with T representing an input/output type that is supported by the binding you intend to use.

The following code shows imperative binding in action. In this example blobs are created and the blob path is derived from the incoming JSON data, namely the category.

public static class CreateToDoItem
{
    [FunctionName("CreateToDoItem")]
    public static async Task<HttpResponseMessage> Run(
        [HttpTrigger(AuthorizationLevel.Function, "post", Route = null)]HttpRequestMessage req,
        IBinder binder,
        TraceWriter log)
    {
        ToDoItem item = await req.Content.ReadAsAsync<ToDoItem>();
        item.Id = Guid.NewGuid().ToString();

        BlobAttribute dynamicBlobBinding = new BlobAttribute(blobPath: $"todo/{item.Category}/{item.Id}");

        using (var writer = binder.Bind<TextWriter>(dynamicBlobBinding))
        {
            writer.Write(JsonConvert.SerializeObject(item));
        }

        return req.CreateResponse(HttpStatusCode.OK, "Added " + item.Description);
    }
}

If the following 2 POSTS are made:

{
    "Description" : "Lift weights",
    "Category" : "Gym"
}
{
    "Description" : "Feed the dog",
    "Category" : "Home"
}

Then 2 blobs will be output with the following paths - note the random filenames and imperatively-bound paths: Gym and Home :

http://127.0.0.1:10000/devstoreaccount1/todo/Gym/5dc4eb72-0ae6-42fc-9a8b-f4bf646dcd28

http://127.0.0.1:10000/devstoreaccount1/todo/Home/530373ef-02bc-4200-a4e7-948448ac081b

Using the Actor Model and Akka.NET for IoT Systems

In additional to cloud-based offerings such as Microsoft’s Azure IoT Suite, it’s also possible to create representations of Internet of Things devices using the Actor Model and Akka.NET. For example, in addition to hosting an Akka.NET actor system in the cloud, you can also have them running on-premises; for example you may have an actor system monitoring an underground railway network for a city. In this case you may already have the infrastructure in place and decide that hosting in the cloud is unnecessary or risky, and deploy to servers geographically close to the underground with some disaster recovery/backup servers located elsewhere.

The Actor Model is a good fit for IoT scenarios due to the inherent concurrency controls, fault-tolerance, and performance/scalability. Akka.NET is a “toolkit and runtime for building highly concurrent, distributed, and fault tolerant event-driven applications on .NET & Mono”[1].

In the Actor Model, the smallest unit of computation is the actor. An actor can receive messages from other actors, perform computations, manage their state, and send messages to other actors.

For IoT scenarios, actors can model the system in a number of ways. For example, for every physical device in the real world there can be an actor instance to represent that device. This means you may have many actor instances, one for each physical device. When a sensor registers a new reading (temperature, proximity, pressure, etc.) this sensor can communicate with the actor responsible for it, the actor receives a message and can update its internal state to represent the latest reading. The actor representing the device can also respond to another type of message to access the last updated value. The device actor may also be responsible for sending messages to the physical device to update it’s configuration (for example) or this functionality may be broken down into its own actor definition depending on the exact requirements. The device actor may communicate with another actor that is purely responsible for handling the network interfacing required to communicate with devices. In this way if the “network actor” crashed and restarts the actors representing the device don’t crash and lose their state.

Groups of “device actors” can created with the supervision hierarchy that the Actor Model provides to protect groups of actors/devices from crashing other groups of actors/devices. These hierarchies can also provide device management semantics such as the registering of a new device that has just come online. Actors can also represent “abstract” concepts such as the creation of a dedicated actor to perform querying of groups of actors.

To learn more about using Akka.NET with IoT scenarios, check out my Representing IoT Systems with the Actor Model and Akka.NET Pluralsight course.