Free eBook C# 7.0: What’s New Quick Start Complete

My new free eBook “C# 7.0: What’s New Quick Start” is now complete and available for download.

C# 7.0: What’s New Quick Start eBook Cover Image

The book covers the following:

  • Literal Digit Separators and Binary Literals
  • Throwing Exceptions in Expressions
  • Local Functions
  • Expression Bodied Accessors, Constructors and Finalizers
  • Out Variables
  • By-Reference Local Variables and Return Values
  • Pattern Matching
  • Switch Statements
  • Tuples

You can download it for free or pay what you think it is worth.

Happy reading!

Creating Versioned APIs with Azure Functions and Proxies

One of the interesting possibilities with the (currently in preview) Azure Function Proxies is the ability to create HTTP APIs that can be versioned and also deployed/managed independently.

For example, suppose there is a API that lives at the root “https://dctdemoapi.azurewebsites.net/api". We could have multiple resources under this root such as customer, products, etc.

So to get the product with an id of 42 we’d construct: “https://dctdemoapi.azurewebsites.net/api/products?id=42”.

If we wanted the ability to version the API we could construct “https://dctdemoapi.azurewebsites.net/api/v1/products?id=42” for version 1 and “https://dctdemoapi.azurewebsites.net/api/v2/products?id=42” for version 2, etc.

Using proxies we can use the format “https://dctdemoapi.azurewebsites.net/api/[VERSION]/[RESOURCE]?[PARAMS]”

Now we can create 2 proxies (for example in an Azure Function called “dctdemoapi”) that forwards the HTTP requests to other Function Apps (in this example dctdemoapiv1 and dctdemoapiv2).

Screenshots of the proxies are as follows:

Azure Function proxy settings for API version 1

Azure Function proxy settings for API version 2

And the respective proxies.json config file:

{
    "proxies": {
        "v1": {
            "matchCondition": {
                "route": "api/v1/{*restOfPath}"
            },
            "backendUri": "https://dctdemoapiv1.azurewebsites.net/api/{restOfPath}"
        },
        "v2": {
            "matchCondition": {
                "route": "api/v2/{*restOfPath}"
            },
            "backendUri": "https://dctdemoapiv2.azurewebsites.net/api/{restOfPath}"
        }
    }
}

Notice in the proxy config the use of the wildcard term “{*restOfPath}” – this will pass the remainder of the path segments to the backend URL, for example “products”, meaning a request to “https://dctdemoapi.azurewebsites.net/api/v1/products?id=42” will be sent to “https://dctdemoapiv1.azurewebsites.net/api/products?id=42”; and “https://dctdemoapi.azurewebsites.net/api/v2/products?id=42” will be sent to “https://dctdemoapiv2.azurewebsites.net/api/products?id=42”.

Now versions of the API can be updated/monitored/managed/etc independently because they are separate Function App instances, but code duplication is a potential problem; common business logic could however be compiled into an assembly and referenced in both Function Apps.

To jump-start your Azure Functions knowledge check out my Azure Function Triggers Quick Start Pluralsight course.

Custom Session Logging in Marten

Marten is a .NET document database library that uses an underlying PostgreSQL database to store objects as JSON. The library has a variety of features that allow the logging of SQL statements issued to the underlying PostgreSQL including previewing LINQ query SQL statements.  One of the other logging features available allows custom logging to be created for individual session operations such as successfully issued database SQL commands, failed commands, and changes that were saved. There are also numerous other logging/extension points that can be utilized such as logging schema change SQL and automatically using a logger for all sessions.

The following code shows a console application that writes two customers and then retrieves them:

using System;
using static System.Console;
using Marten;

namespace MartenFKDemo
{
    class Customer
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public override string ToString() => $"{Id} {Name}";
    }

    class Program
    {
        static void Main(string[] args)
        {
            const string connectionString = "host = localhost; database = OrderDb; password = g7qo84nck22i; username = postgres";

            var store = DocumentStore.For(connectionString);

            WriteLine("Creating new customer");
            using (var session = store.OpenSession())
            {
                session.Store(new Customer {Name = "Amrit"}, new Customer {Name = "Sarah"});

                session.SaveChanges();
            }


            WriteLine("All customers:");
            using (var session = store.QuerySession())
            {
                foreach (Customer customer in session.Query<Customer>())
                {
                    WriteLine(customer);
                }
            }
            
            ReadLine();
        }
    }
}

Running the application results in the following output:

Creating new customer
All customers:
9001 Amrit
9002 Sarah

To create a custom session logger, the IMartenSessionLogger interface can be implemented, a simple version that logs to the console is shown as follows:

class ColorConsoleLogger : IMartenSessionLogger
{
    public void LogSuccess(Npgsql.NpgsqlCommand command)
    {
        ForegroundColor = ConsoleColor.Green;

        WriteLine(command.CommandText); // additional properties (e.g. SQL parameters) are available
    }

    public void LogFailure(Npgsql.NpgsqlCommand command, Exception ex)
    {
        ForegroundColor = ConsoleColor.Red;

        WriteLine(command.CommandText); // additional properties (e.g. SQL parameters) are available
        WriteLine(ex);
    }

    public void RecordSavedChanges(IDocumentSession session, IChangeSet commit)
    {
        ForegroundColor = ConsoleColor.Gray;

        foreach (object insertedItem in commit.Inserted) //  updated/deleted are also available
        {
            WriteLine(insertedItem);
        }
    }
}

To configure individual sessions to use this logger, the sessions Logger property can be set as the following modified code demonstrates:

ResetColor();
WriteLine("Creating new customer");
using (var session = store.OpenSession())
{
    // Set logger for this session
    session.Logger = new ColorConsoleLogger();

    session.Store(new Customer {Name = "Amrit"}, new Customer {Name = "Sarah"});

    session.SaveChanges();
}

ResetColor();
WriteLine("All customers:");
using (var session = store.QuerySession())
{
    // Set logger for this session
    session.Logger = new ColorConsoleLogger();

    foreach (Customer customer in session.Query<Customer>())
    {
        ResetColor();
        WriteLine(customer);
    }
}

This produces the following output:

Creating new customer
select public.mt_upsert_customer(doc := :p0, docDotNetType := :p1, docId := :p2, docVersion := :p3);select public.mt_upsert_customer(doc := :p4, docDotNetType := :p5, docId := :p6, docVersion := :p7);
13001 Amrit
13002 Sarah
All customers:
select d.data, d.id, d.mt_version from public.mt_doc_customer as d
13001 Amrit
13002 Sarah

screenshot of Marten custom logger

To learn more about the document database features of Marten check out my Pluralsight courses: Getting Started with .NET Document Databases Using Marten and Working with Data and Schemas in Marten or the documentation.

Previewing the Generated PostgreSQL SQL for a Query in Marten

Marten is a .NET document database library that uses an underlying PostgreSQL database to store objects as JSON. The library has a variety of features that allow the logging of SQL statements issued to the underlying PostgreSQL database in addition to being able to do things such as get the PostgreSQL query plan for a given LINQ query.

One simple way to get the generated SQL for a Marten LINQ query is to use the ToCommand() extension method.

As an example, suppose we are developing some query code as follows (this code uses the Include method to include the related documents in a single database round-trip):

Customer customer = null;

List<Order> orders = session.Query<Order>()
                            .Include<Customer>(joinOnOrder => joinOnOrder.CustomerId, includedCustomer => customer = includedCustomer)
                            .Where(x => x.CustomerId == 4001).ToList();

If we want to get an idea of what SQL Marten will generate for this LINQ query, we can change the code as shown in the following:

Customer customer = null;

IQueryable<Order> orders = session.Query<Order>()
                                  .Include<Customer>(joinOnOrder => joinOnOrder.CustomerId, includedCustomer => customer = includedCustomer)
                                  .Where(x => x.CustomerId == 4001);

// Get the SQL command that will be issued when the query executes
NpgsqlCommand cmd = orders.ToCommand();

// Output some selected command info
Console.WriteLine(cmd.CommandText);

foreach (NpgsqlParameter parameter in cmd.Parameters)
{
    Console.WriteLine($"Parameter {parameter.ParameterName} = {parameter.Value}");
}

// Ensure included customer variable is populated
List<Order> orderResults = orders.ToList();

Console.WriteLine(customer.Name);
foreach (Order order in orderResults)
{
    Console.WriteLine($" Order {order.Id} for {order.Quantity} items");
}

Running this preceding code  results in the following console output:

select d.data, d.id, d.mt_version, customer_id.data, customer_id.id, customer_id.mt_version from public.mt_doc_order as d INNER JOIN public.mt_doc_customer as customer_id ON d.customer_id = customer_id.id where d.customer_id = :arg0
Parameter arg0 = 4001
Sarah
 Order 3001 for 42 items
 Order 4001 for 477 items
 Order 5001 for 9 items

To learn more about the document database features of Marten check out my Pluralsight courses: Getting Started with .NET Document Databases Using Marten and Working with Data and Schemas in Marten.

Retrieving Raw JSON Data in Web API with Marten

Marten is a .NET document database library that uses an underlying PostgreSQL database to store objects as JSON.

Ordinarily, Marten takes care of retrieving the JSON from the database and deserializing it into an object. We can however instruct Marten to perform a query to retrieve document(s) and not perform the deserialization, instead giving us the JSON as it appears in the underlying PostgreSQL record. If we are exposing documents via a Web API, this feature can be taken advantage of to reduce some processing overhead on the server.

As an example, we could have the following Customer document defined:

public class Customer
{
    public int Id { get; set; }
    public string Name { get; set; }

    // etc.
}

And in a CustomersController we could start with a method as follows to add Customers to the database:

public void Post(Customer customer)
{
    // no validation

    // DocumentStore would normally be created only once in app, e.g. via IOC singleton 
    using (var store = DocumentStore.For(ConnectionString))
    {
        using (var session = store.LightweightSession())
        {
            session.Store(customer);
            session.SaveChanges();
        }
    }
}

To get all Customers, the following method could be written:

// GET api/customers
public IEnumerable<Customer> Get()
{
    // DocumentStore would normally be created only once in app, e.g. via IOC singleton 
    using (var store = DocumentStore.For(ConnectionString))
    {
        using (var session = store.QuerySession())
        {
            return session.Query<Customer>();
        }
    }
}

The preceding method however incurs the additional overhead of Marten deserializing the database JSON into Customer objects, only to be serializing it again back into JSON on the way out of the API.

When creating the LINQ query, the Marten ToJsonArray() method can be added to instruct Marten to simply return the JSON directly from the database.

We can then modify the Get method as follows:

public HttpResponseMessage Get()
{
    // DocumentStore would normally be created only once in app, e.g. via IOC singleton 
    using (var store = DocumentStore.For(ConnectionString))
    {
        using (var session = store.QuerySession())
        {
            string rawJsonFromDb = session.Query<Customer>().ToJsonArray();

            var response = Request.CreateResponse(HttpStatusCode.OK);
            response.Content = new StringContent(rawJsonFromDb, Encoding.UTF8, "application/json");
            return response;
        }
    }
}

We could also write the parameterized Get method and use Marten’s AsJson() method to get the JSON string for the individual Customer document as  in the following code:

public HttpResponseMessage Get(int id)
{
    // DocumentStore would normally be created only once in app, e.g. via IOC singleton 
    using (var store = DocumentStore.For(ConnectionString))
    {
        using (var session = store.LightweightSession())
        {
            var rawJsonFromDb = session.Query<Customer>().Where(x => x.Id == id).AsJson().FirstOrDefault();

            if (string.IsNullOrEmpty(rawJsonFromDb))
            {
                throw new HttpResponseException(HttpStatusCode.NotFound);
            }

            var response = Request.CreateResponse(HttpStatusCode.OK);
            response.Content = new StringContent(rawJsonFromDb, Encoding.UTF8, "application/json");
            return response;
        }
    }
}

To learn more about the document database features of Marten check out my Pluralsight courses: Getting Started with .NET Document Databases Using Marten and Working with Data and Schemas in Marten.

New Pluralsight Course: Working with Data and Schemas in Marten

Marten is a .NET document database library to allows objects to be stored, retrieved, and queried as documents stored as JSON in an underlying PostgreSQL database. This new course is a follow-on from the previous Getting Started with .NET Document Databases Using Marten course, if you’re new to Marten I’d recommend checking out the previous course first before continuing with this new one.

Among other topics, this new course covers how to log/diagnose the SQL that is being issued to PostgreSQL; how to enable offline optimistic concurrency; bulk document inserts; a number of ways to improve query performance; and the customization of database schema objects.

You can check out the new course on the Pluralsight site.

Reducing Database Round Trips With Included Documents in Marten

Marten is a .NET document database library that uses an underlying PostgreSQL database to store objects as JSON.

In the  previous article (Enforcing Referential Integrity Between Documents with Marten and PostgreSQL) we saw how we can enable referential integrity between documents and the example showed a Customer document may have many related Order documents.

When creating LINQ queries with Marten, one feature is the idea of included documents. So for example we might want to get a Customer document and all the Order documents for that customer.

We could write some code as follows to get Customer 4001 and a list of their Orders:

// 2 trips to database, 2 queries
using (var session = store.QuerySession())
{
    Customer customer = session.Query<Customer>()
                               .Single(x => x.Id == 4001);

    Console.WriteLine(customer.Name);

    IEnumerable<Order> orders = session.Query<Order>()
                                       .Where(x => x.CustomerId == 4001);

    foreach (var order in orders)
    {
        Console.WriteLine($" Order {order.Id} for {order.Quantity} items");
    }
}

The preceding code however will result in two round trips to the database which may result in a performance problem:

select d.data, d.id, d.mt_version 
from public.mt_doc_customer as d 
where d.id = 4001 LIMIT 2
select d.data, d.id, d.mt_version
from public.mt_doc_order as d 
where d.customer_id = 4001

 

The .Include() extension method of Marten allows a single round trip to be made to the database that executes a SQL join and returns both the Customer data and Orders.

The following code shows how to accomplish this, notice that we need to declare an “output” variable to hold the “included” customer.

// 1 trip to database, 1 query
using (var session = store.QuerySession())
{                
    Customer customer = null;

    List<Order> orders = 
         session.Query<Order>()
                .Include<Customer>(joinOnOrder => joinOnOrder.CustomerId, includedCustomer => customer = includedCustomer)
                .Where(x => x.CustomerId == 4001)
                .ToList();


    Console.WriteLine(customer.Name);

    foreach (var order in orders)
    {
        Console.WriteLine($" Order {order.Id} for {order.Quantity} items");
    }  
}

The preceding code now only makes a single request to the database with the following SQL:

select d.data, d.id, d.mt_version, customer_id.data, customer_id.id, customer_id.mt_version 
from public.mt_doc_order as d 
INNER JOIN public.mt_doc_customer as customer_id ON d.customer_id = customer_id.id 
where d.customer_id = 4001

To learn more about the document database features of Marten check out my Pluralsight courses: Getting Started with .NET Document Databases Using Marten and Working with Data and Schemas in Marten.

Enforcing Referential Integrity Between Documents with Marten and PostgreSQL

Even though Marten is a library to enable document database storage semantics, because it’s built on top of (the advanced JSON features) of PostgreSQL, and PostgreSQL itself is relational, we can add constraints between document instances just as we would do in relational normalised databases.

For example, the following two simplified classes represent an order that points to a customer document:

class Customer
{
    public int Id { get; set; }
    public string Name { get; set; }
    
    // etc.
}

class Order
{
    public int Id { get; set; }
    public int CustomerId { get; set; }
    public int Quantity { get; set; }
    
    // etc.
}

By default there is no referential integrity enforcing the idea that the CustomerId in the Order class must point to an existing Customer document in the database. This means that the following code succeeds with no errors:

const string connectionString = "host = localhost; database = OrderDb; password = g7qo84nck22i; username = postgres";

var store = DocumentStore.For(connectionString);

using (var session = store.OpenSession())
{
    var order = new Order
    {
        Quantity=42,
        CustomerId = 34567 // non-existent
    };

    session.Store(order);

    session.SaveChanges(); // no error
}

The preceding code will result in an order document being stored with an invalid CustomerId.

Marten can be configured to enforce this by either adding the [ForeignKey(typeof(Customer))] attribute to the CustomerId property or by configuring the document store as the following code shows:

var store = DocumentStore.For(configure =>
{
    configure.Connection(connectionString);
    configure.Schema.For<Order>().ForeignKey<Customer>(x => x.CustomerId);
});

Now running the code again will result in an exception: insert or update on table "mt_doc_order" violates foreign key constraint "mt_doc_order_customer_id_fkey”.

If the code is modified to create a valid customer in the same session, Marten will generate a customer Id that can then be used in the order. When SaveChanges() is called, Marten will save the dependent customer document first and then the order document that points to it. The following code executes successfully:

using (var session = store.OpenSession())
{
    var customer = new Customer {Name = "Sarah"};

    session.Store(customer);
    
    // customer now has an Id auto-generated 

    var order = new Order
    {
        Quantity = 42,
        CustomerId = customer.Id // valid existing customer ID
    };

    session.Store(order);

    session.SaveChanges();
}

To learn more about the document database features of Marten check out my Pluralsight courses: Getting Started with .NET Document Databases Using Marten and Working with Data and Schemas in Marten.

.NET Document Databases with Marten

Document databases are a form of NoSQL database that store items (objects) as single documents rather than potentially splitting the data among multiple relational database tables.

Marten is a .NET library that enables the storing, loading, deleting, and querying of documents. Objects in the application can be stored into the document database and retrieved back as an object from the database. In this approach there is no requirement for the additional “plumbing code” of ORMs.

Marten is not a database itself but rather a library that interacts with the (advanced JSON features) of the open source, cross platform PostgreSQL database.

Once the Marten NuGet package is installed there are a number of steps to storing .NET objects as documents.

First the “document” to be stored is defined. At the most basic level this is a class that has a field or property that represents the document identity as stored in the database. There are a number of ways to define  identity, one of which is to follow the naming convention “Id” as the following class shows:

class Customer
{
    public int Id { get; set; }
    public string Name { get; set; }
    public List<Address> Addresses { get; set; } = new List<Address>();
}

internal class Address
{
    public string Line1 { get; set; }
    public string Line2 { get; set; }
    public string Line3 { get; set; }
}

Notice in the preceding code that the Address class does not have an Id. This is because the address information will be storing in the overall Customer document, rather than for example in a relational database as rows in a separate Address table.

To work with the database a document store instance is required, this can be created with additional configuration or with the simple code shown below:

var connectionString = "host = localhost; database = CustomerDb; password = YOURPASSWORD; username = postgres";

var store = DocumentStore.For(connectionString);

Working with documents happens by way of a session instance, there are a number of types/configurations of sessions available.

To create a new customer object and store it the following code could be used:

// Store document (& auto-generate Id)
using (var session = store.LightweightSession())
{
    var customer =  new Customer
    {
        Name = "Arnold",
        Addresses =
        {
            new Address {Line1="Address 1 Line 1", Line2="Address 1 Line 2",Line3="Address 1 Line 3"},
            new Address {Line1="Address 2 Line 1", Line2="Address 2 Line 2",Line3="Address 2 Line 3"}
        }
    };

    // Add customer to session
    session.Store(customer);

    // Persist changes to the database
    session.SaveChanges();
}

Once the above code executes, the customer will be stored in a PostgreSQL table called “mt_doc_customer”. The Customer object will be serialized to JSON and stored in a special JSONB field in PostgreSQL. The JSON data also contains all the addresses.

Screenshot of pgAdmin showing Marten table

{
  "Addresses": [
    {
      "Line3": "Address 1 Line 3",
      "Line2": "Address 1 Line 2",
      "Line1": "Address 1 Line 1"
    },
    {
      "Line3": "Address 2 Line 3",
      "Line2": "Address 2 Line 2",
      "Line1": "Address 2 Line 1"
    }
  ],
  "Id": 1,
  "Name": "Arnold"
}

There are a number of ways to retrieve documents and convert them back into .NET objects. One method is to use the Marten LINQ support as the following code shows:

// Retrieve by query
using (var session = store.QuerySession())
{
    var customer = session.Query<Customer>().Single(x => x.Name == "Arnold");
                
    Console.WriteLine(customer.Id);
    Console.WriteLine(customer.Name);
}

When executed, this code displays the retrieved customer details as the following screenshot shows:

Console application showing Marten document data

To learn more about the document database features of Marten check out my Pluralsight courses: Getting Started with .NET Document Databases Using Marten and Working with Data and Schemas in Marten.