.NET Document Databases with Marten

Document databases are a form of NoSQL database that store items (objects) as single documents rather than potentially splitting the data among multiple relational database tables.

Marten is a .NET library that enables the storing, loading, deleting, and querying of documents. Objects in the application can be stored into the document database and retrieved back as an object from the database. In this approach there is no requirement for the additional “plumbing code” of ORMs.

Marten is not a database itself but rather a library that interacts with the (advanced JSON features) of the open source, cross platform PostgreSQL database.

Once the Marten NuGet package is installed there are a number of steps to storing .NET objects as documents.

First the “document” to be stored is defined. At the most basic level this is a class that has a field or property that represents the document identity as stored in the database. There are a number of ways to define  identity, one of which is to follow the naming convention “Id” as the following class shows:

class Customer
    public int Id { get; set; }
    public string Name { get; set; }
    public List<Address> Addresses { get; set; } = new List<Address>();

internal class Address
    public string Line1 { get; set; }
    public string Line2 { get; set; }
    public string Line3 { get; set; }

Notice in the preceding code that the Address class does not have an Id. This is because the address information will be storing in the overall Customer document, rather than for example in a relational database as rows in a separate Address table.

To work with the database a document store instance is required, this can be created with additional configuration or with the simple code shown below:

var connectionString = "host = localhost; database = CustomerDb; password = YOURPASSWORD; username = postgres";

var store = DocumentStore.For(connectionString);

Working with documents happens by way of a session instance, there are a number of types/configurations of sessions available.

To create a new customer object and store it the following code could be used:

// Store document (& auto-generate Id)
using (var session = store.LightweightSession())
    var customer =  new Customer
        Name = "Arnold",
        Addresses =
            new Address {Line1="Address 1 Line 1", Line2="Address 1 Line 2",Line3="Address 1 Line 3"},
            new Address {Line1="Address 2 Line 1", Line2="Address 2 Line 2",Line3="Address 2 Line 3"}

    // Add customer to session

    // Persist changes to the database

Once the above code executes, the customer will be stored in a PostgreSQL table called “mt_doc_customer”. The Customer object will be serialized to JSON and stored in a special JSONB field in PostgreSQL. The JSON data also contains all the addresses.

Screenshot of pgAdmin showing Marten table

  "Addresses": [
      "Line3": "Address 1 Line 3",
      "Line2": "Address 1 Line 2",
      "Line1": "Address 1 Line 1"
      "Line3": "Address 2 Line 3",
      "Line2": "Address 2 Line 2",
      "Line1": "Address 2 Line 1"
  "Id": 1,
  "Name": "Arnold"

There are a number of ways to retrieve documents and convert them back into .NET objects. One method is to use the Marten LINQ support as the following code shows:

// Retrieve by query
using (var session = store.QuerySession())
    var customer = session.Query<Customer>().Single(x => x.Name == "Arnold");

When executed, this code displays the retrieved customer details as the following screenshot shows:

Console application showing Marten document data

To learn more about the document database features of Marten check out my Pluralsight courses: Getting Started with .NET Document Databases Using Marten and Working with Data and Schemas in Marten.


Sending a Regular SMS with Azure Functions and Twilio

Azure Functions allow the creation of Serverless event driven applications with minimal effort. We can create Azure Functions using C# (or other languages) right inside the Azure Web app.

Functions sit inside a function app and each function can be configured to be executed in response to a number of triggers such as a BlobTrigger, EventHubTrigger, etc. One of the triggers is the Timer trigger that can be configured to execute the function on a specified schedule.

To get get started, once you’ve created an Azure account and logged in, head to http://functions.azure.com/ and this will allow a quickstart experience to develop your function (you may be asked to create a new Function App as a container for your functions so go ahead and follow the prompts to do this).


Click “Create this function” and you’ll be taken to the editor.


In the code section (within the Run method) we can write code that is executed when the function executes.

Clicking on the Integrate tab allows the specification of the schedule in CRON format.


To have the function execute every five minutes we would specify: 0 */5 * * * *

A function can have Outputs, one of which is a Twilio output. At the time of writing this is in an early beta/experimental phase and I couldn’t get it to work correctly. However because we can write C#, we can send an SMS by using the Twilio REST API. You need to sign up for a Twilio trial account and this will give you a Twilio AccountSID and an authorisation token.

Click the Develop tab and add the following code (using your Twilio SID and auth token):

using System;
using Twilio;

public static void Run(TimerInfo myTimer, TraceWriter log)
    log.Info($"C# Timer trigger function executed at: {DateTime.Now}");    

    string accountSid = "YOUR SID HERE";
    string authToken = "YOUR TOKEN HERE";

    var client = new TwilioRestClient(accountSid, authToken);

            "+614xxxxxxxx", // Insert your Twilio from SMS number here
            "+614xxxxxxxx", // Insert your verified (trial) to SMS number here
            "hello from Azure Functions!" + DateTime.Now            

(+614 is the dialling code for Australian mobiles so you should replace this with whatever makes sense for the Twilio account phone numbers you’ve created.)

Click Save and you’ll get some compilation errors because we haven’t yet added the Twilio NuGet package to be used by our code.

Click the View Files link under the code editing window and add a new file (by clicking the plus icon). Add a file called project.json with the following content:


Click Save and in the logs section you should see the package installed:


Now every five minutes you’ll get a text message sent!


Just remember to disable/delete the function or you will continue to get messages sent. You can do this by clicking on the Manage tab and choosing the appropriate option:



New Pluralsight Course: Testing C# Code in Production with Scientist.NET

My latest Pluralsight course is now available for viewing. It demonstrates how to use the Scientist.NET library to execute candidate code in production alongside the existing production code. This allows the new candidate code to be additionally verified as able to work in production (for example with production data that may be of variable quality) and offers an additional check in addition to automated tests that have been executed in the development/QA environment.

From the course description: “Errors in production code can be costly to fix, more stressful for the development team, and frustrating for the end-user. In this course, Testing C# Code in Production with Scientist.NET, you will get to see how Scientist.NET allows sections of the application to be changed more safely by running both the existing code alongside the new code in production. You'll begin with an introduction to Scientist.NET before installing it and writing your first experiment. Next, you'll learn how to customize the configuration of experiments. Finally, you'll learn how to publish experiment data to SQL Server and analyze experiment results…”

You can check out the new course here.


Refactoring Production Code With Experiments and Scientist.NET

When refactoring a part of an application we can use the existing tests to give a level of confidence that the refactored code still produces the same result, i.e. the existing tests still pass with the new implementations.

A system that has been in production use for some time is likely to have amassed a lot of data that is flowing through the “legacy” code. This means that although the existing tests may still pass, when used in production the new refactored code may produce errors or unexpected results.

It would be helpful as an experiment to run the existing legacy code alongside the new code to see if the results differ, but still continue to use the result of the existing legacy code. Scientist.NET allows exactly this kind of experimentation to take place.

Scientist.NET is a port of the Scientist library for Ruby. It is currently in pre-release and has a NuGet package we can use.

An Example Experiment

Suppose there is an interface as shown in the following code that describes the ability to sum a list if integer values and return the result.

interface ISequenceSummer
    int Sum(int numberOfTerms);

This interface is currently implemented in the legacy code as follows:

class SequenceSummer : ISequenceSummer
    public int Sum(int numberOfTerms)
        var terms = new int[numberOfTerms];

        // generate sequence of terms
        var currentTerm = 0;
        for (int i = 0; i < terms.Length; i++)
            terms[i] = currentTerm;

        // Sum
        int sum = 0;
        for (int i = 0; i < terms.Length; i++)
            sum += terms[i];

        return sum;

As part of the refactoring of the legacy code, this implementation is to be replaced with a version that utilizes LINQ as shown in the following code:

class SequenceSummerLinq : ISequenceSummer
    public int Sum(int numberOfTerms)
        // generate sequence of terms
        var terms = Enumerable.Range(0, 5).ToArray();
        // sum
        return terms.Sum();

After installing the Scientist.NET NuGet package, an experiment can be created using the following code:

int result;            

result = Scientist.Science<int>("sequence sum", experiment =>
    experiment.Use(() => new SequenceSummer().Sum(5)); // current production method

    experiment.Try("Candidate using LINQ", () => new SequenceSummerLinq().Sum(5)); // new proposed production method

}); // return the control value (result from SequenceSummer)

This code will run the .Use(…) code that contains the existing legacy implementation. It will also run the .Try(…) code that contains the new implementation. Scientist.NET will store both results for reporting on and then return the result from the .Use(…) code for use by the rest of the program. This allows any differences to be reported on but without actually changing the implementation of the production code. At some point in the future, if the results of the legacy code (the control) match that of the new code (the candidate), the refactoring can be completed by removing the old implementation (and the experiment code) and simply calling the new implementation.

To get the results of the experiment, a reporter can be written and configured. The following code shows a custom reporter that simply reports to the Console:

public class ConsoleResultPublisher : IResultPublisher
    public Task Publish<T>(Result<T> result)
        Console.ForegroundColor = result.Mismatched ? ConsoleColor.Red : ConsoleColor.Green;

        Console.WriteLine($"Experiment name '{result.ExperimentName}'");
        Console.WriteLine($"Result: {(result.Matched ? "Control value MATCHED candidate value" : "Control value DID NOT MATCH candidate value")}");
        Console.WriteLine($"Control value: {result.Control.Value}");
        Console.WriteLine($"Control execution duration: {result.Control.Duration}");
        foreach (var observation in result.Candidates)
            Console.WriteLine($"Candidate name: {observation.Name}");
            Console.WriteLine($"Candidate value: {observation.Value}");
            Console.WriteLine($"Candidate execution duration: {observation.Duration}");

        if (result.Mismatched)
            // saved mismatched experiments to event log, file, database, etc for alerting/comparison

        Console.ForegroundColor = ConsoleColor.White;

        return Task.FromResult(0);

To plug this in, before the experiment code is executed:

Scientist.ResultPublisher = new ConsoleResultPublisher();

The output of the experiment (and the custom reporter) look like the following screenshot:

screenshot of console application using Scentist.NET with custom reporter

To learn more about Scientist.NET check out my Pluralsight course: Testing C# Code in Production with Scientist.NET.


New Pluralsight Course: Working with Nulls in C#

My latest Pluralsight course is now available for viewing. It covers the fundamental aspects of working with nulls in C# code.

From the course description: “Unexpected null values and NullReferenceExceptions can be a constant source of bugs resulting in wasted time and out of hours support callouts. In this course, Working with Nulls in C#, you're going to learn about the different ways that null values pop up in C# code and how to deal with them. First, you're going to learn the fundamentals of why you get null values and the difference between value and reference types. Next you'll learn how you can use the various C# operators to check for and manipulate nulls. Finally, you'll learn how to think about nulls at a higher abstraction level in your object-oriented code. By the end of this course, you'll understand the different types of objects in C#, how to correctly create nullable value types, how to use C# operators to work with nulls with fewer lines of code, and how to implement the Null Object pattern to remove the need to write repetitive null checking code.”

You can check out the course here.


Using Python Expressions from C# Code

Utilizing the dynamic features of C# and the Dynamic Language Runtime (DLR), among other things, allows C# programs to interoperate with other dynamic languages built on top of the DLR.

For example, Python has the ability to chain operators, for example the expression “10 < 15 < 20” returns true: 10 is less than 15 and 15 is less than 20.

To use a chained Python expression from C# code, IronPython can be used.

To get started (in this example a C# console application) first install the IronPython NuGet package. This allows the application to interoperate with Python code.

The following code shows a console application that evaluates a user-entered number “n” against the Python expression “10 < n < 20”:

using IronPython.Hosting;
using Microsoft.Scripting;
using Microsoft.Scripting.Hosting;
using static System.Console;

namespace PythonCSharp
    class Program
        static void Main(string[] args)
            const string pythonExpression = "10 < n < 20";

            int n;

            while (true)
                WriteLine($"Enter a value for n where {pythonExpression}");

                n = int.Parse(ReadLine()); // cast error check omitted

                ScriptEngine pythonEngine = Python.CreateEngine();

                ScriptScope pythonScope = pythonEngine.CreateScope();

                // set the value of the Python token "n" in the expression 10 < n < 20
                pythonScope.SetVariable("n", n);

                ScriptSource pythonScriptSource = pythonEngine.CreateScriptSourceFromString(pythonExpression, SourceCodeKind.Expression);

                // Execute the Python expression and get the return value
                dynamic dynamicResult = pythonScriptSource.Execute(pythonScope);

                WriteLine($"{pythonScriptSource.GetCode().Replace("n", n.ToString())} = {dynamicResult}");


Running the console application allows the user to enter a value for “n” and evaluate it against the Python expression as the following image shows:

Console application calling Python Expression from CSharp

To learn more about how C# makes dynamic possible, some potential use cases, and more detailed on implementing custom types check out my Dynamic C# Fundamentals Pluralsight course.


Persistent Actors in Akka.NET

By default, actors in Akka.NET lose their internal state when they are restarted, for example due to an internal actor error or a system crash.

Akka.Persistence adds additional functionality to enable the storing and restoring of actor state.

There are 2 main persistence concepts: journal storage and snapshot storage.

The Journal

As messages are received by an actor, they may cause some change in the internal actor state. Using Akka Persistence, these messages can be saved in one of a number of configurable journal stores such as SQL Server. When the actor restarts, the messages from the configured journal store are retrieved and “replayed” by the actor. The messages from the journal store will be replayed in the same order as they were originally received by the actor. This in effect rebuilds the actor state one message at a time, each replayed message changing actor state. Once all messages from the journal store have been replayed, the actor state will be equal to what it was before it restarted; it can now process new messages sent to it.


Snapshots are an optional addition on top of the journal store. If an actor has too many messages being replayed from the journal store, and this is causing a performance problem, snapshots can be implemented.

A snapshot, as its name suggests, is a picture of the actor’s state at a given point in time. Snapshots can be created in actor code whenever it is deemed appropriate, for example after every 50 messages have been processed.

Snapshots can improve actor recovery speeds because only those messages stored in the journal since the last snapshot was created need to be replayed. For example, an actor has received 60 messages and a snapshot was created on the 50th message. On restart, Akka.NET notices that a snapshot was created after the 50th message; this snapshot will be loaded into the actor’s state. Now only 10 messages need to be individually replayed (messages 51-60), rather than all 60 messages individually.

To learn more about Akka.Persistence, check out my Akka.NET Persistence Fundamentals Pluralsight course or the Akka.NET documentation.


Creating Your Own Custom Dynamic C# Classes

C# provides pre-supplied dynamic types such as the ExpandoObject. It is also possible to create new dynamic types or add dynamic capabilities to existing custom classes.

One of the core interfaces that enables dynamic behaviour is the IDynamicMetaObjectProvider interface. Whilst this interface can be implemented, an easier way to create a custom dynamic class is to inherit from DynamicObject class.

The DynamicObject base class provides a number of virtual methods that allow an instance of the derived class to respond to dynamic operations such as getting or setting the value of a dynamically added property or invocation of a dynamic method.

The following simple example allows adding of arbitrary dynamic properties at runtime. The class overrides the TrySetMember and TryGetMember virtual methods to provide the behaviour. Notice that these methods have a binder parameter that gives information about the dynamic operation that is being attempted, such as the name of the property (binder.Name). Also note that the methods return a Boolean value to indicate to the runtime whether the dynamic operation succeeded.

using System.Collections.Generic;
using System.Dynamic;
using System.Text;

namespace CustomDynamic
    class MyDynamicClass : DynamicObject
        private readonly Dictionary<string, object> _dynamicProperties = new Dictionary<string, object>(); 

        public override bool TrySetMember(SetMemberBinder binder, object value)
            _dynamicProperties.Add(binder.Name, value);

            // additional error checking code omitted

            return true;

        public override bool TryGetMember(GetMemberBinder binder, out object result)
            return _dynamicProperties.TryGetValue(binder.Name, out result);

        public override string ToString()
            var sb = new StringBuilder();

            foreach (var property in _dynamicProperties)
                sb.AppendLine($"Property '{property.Key}' = '{property.Value}'");

            return sb.ToString();

At runtime, if the instance of the class is declared as dynamic, arbitrary properties can be set as the following simple console application demonstrates:

using System;

namespace CustomDynamic
    class Program
        static void Main(string[] args)
            dynamic d = new MyDynamicClass();

            // Need to declare as dynamic, the following cause compilation errors:
            //      MyDynamicClass d = new MyDynamicClass();
            //      var d = new MyDynamicClass();

            // Dynamically add properties
            d.Name = "Sarah"; // TrySetMember called with binder.Name of "Name"
            d.Age = 42; // TrySetMember called with binder.Name of "Age"

            Console.WriteLine(d.Name); // TryGetMember called
            Console.WriteLine(d.Age); // TryGetMember called


To learn more about how C# makes dynamic possible, some potential use cases, and more detail on implementing custom types check out my Dynamic C# Fundamentals Pluralsight course.


New Pluralsight Course: Dynamic C# Fundamentals

My Pluralsight course “Dynamic C# Fundamentals” is now available.

The course is split into 4 modules:

Module 1 introduces the dynamic features of C# and how it is enabled by the Dynamic Language Runtime (DLR).

Module 2 looks at numerous places that dynamic C# can be used including COM, XAML, MVC, and JSON.

Module 3 digs into more detail and shows how to create custom C# classes that can exhibit dynamic behaviour.

Module 4 shows how the DLR enables C# to interoperate with other dynamic languages such as IronPython.

Full list of topics:

1 Introducing Dynamic C#

  • Why Dynamic C#?
  • Course Outline
  • Introducing the DLR
  • Static and Dynamic Binding
  • Dynamic Binding in Action and RuntimeBinderException
  • Implicit Dynamic Conversions
  • Var and Dynamic
  • Runtime Method Resolution
  • Dynamic and Object Types
  • Limitations of Callable Methods
  • Introducing ExpandoObject
  • Refactoring to Dynamic
  • Dynamically Adding ExpandoObject Behavior

2 Simplifying Code with Dynamic C#

  • Simplifying Reflection Code
  • Reducing Code with Unified Numeric Methods
  • COM Interop Without Interop Assemblies
  • Reducing Plumbing Code with Dynamic JSON
  • Dynamically Populating Excel with Arbitrary JSON Data
  • Dynamic JSON in Web API
  • ExpandoObject and XAML Databinding
  • Dynamic SQL Query Results with Dapper
  • Improving SpecFlow Test Code with Dynamics
  • Dynamic Code Considerations

3 Creating Custom Dynamic Classes

  • Why Custom Dynamic Classes?
  • The IDynamicMetaObjectProvider Interface
  • The DynamicObject Base Class
  • Creating a Dynamic HtmlElement Class
  • Creating the First Test
  • Adding Initial Dynamic Capabilities
  • Dynamic Operation Exceptions
  • Improving the Debugger Experience
  • Overriding ToString
  • Implementing Additional Interfaces
  • Making the Dynamic Object Enumerable
  • Implementing Dynamic Methods
  • Method Invocation Precedence
  • Invoking the Dynamic Object Itself

4 Interoperating with Dynamic Languages

  • Why Dynamic Interop?
  • Getting Started with IronPython
  • Getting Scripting Input from the User
  • Scripting Risks
  • Executing Python Statements
  • Interacting with Python Objects
  • Passing Custom Dynamic Objects to Python

Check out the full course description.