(This post refers to Azure Functions v2)

There are different ways to access both the request data and also request metadata when a HTTP-triggered Azure Function is executed.

Getting Query String Data in Azure Functions

Suppose we have the following class (e.g. in table storage):

public class PhotoMetadata
{
    public string PartitionKey { get; set; }
    public string RowKey { get; set; }
    public string FileName { get; set; }
    public string Keywords { get; set; }
}

We could write an Azure Function triggered by a HTTP GET that returns an item from a database by a querystring parameter called “id”:

[FunctionName("GetPhotoMetadata")]
public static IActionResult GetPhotoMetadata(
    [HttpTrigger(AuthorizationLevel.Function, "get")] HttpRequest req,
    ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    string id = req.Query["id"];

    if (string.IsNullOrWhiteSpace(id))
    {
        return new NotFoundResult();
    }

    PhotoMetadata metadata = LoadFromDatabase(id);

    return new OkObjectResult(metadata);
}

In the preceding code, to access querystring parameters use req.Query and specify the key you are looking for, in this example “id”.

If there is no value in the incoming request, id will be null and we return a NotFoundResult (404).

Getting HTTP POST JSON Request Data in Azure Functions

When it comes to accessing POSTed data, there are a number of options.

Manually Convert JSON Request Strings

The first option is to take control of the process at a lower level and read the posted data from the request body and parse the JSON into a dynamic C# object. [If you’re not familiar with dynamic C# check out my Dynamic C# Fundamentals Pluralsight course]

First we define a model that will represent the posted data (we don’t want to use the PhotoMetadata class as we don’t want clients specifying partition and row keys):

public class PhotoMetadataAdditionRequest
{
    public string FileName { get; set; }
    public string Keywords { get; set; }
}

Next we can write a function that will parse this incoming data:

[FunctionName("AddPhotoMetadata")]
public static async Task<IActionResult> AddPhotoMetadata(
    [HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequest req,
    ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");


    log.LogInformation("You can get additional information about the request such as:");
    log.LogInformation($" length : {req.ContentLength}");
    log.LogInformation($" type   : {req.ContentType}");
    log.LogInformation($" https  : {req.IsHttps}");
    log.LogInformation($" host   : {req.Host}");


    // read the contents of the posted data into a string
    string requestBody = await new StreamReader(req.Body).ReadToEndAsync();

    // use Json.NET to deserialize the posted JSON into a C# dynamic object
    dynamic data = JsonConvert.DeserializeObject(requestBody);

    // data validation omitted for demo purposes

    // extract data from the dynamic object into strongly typed object
    PhotoMetadata metadata = new PhotoMetadata
    {
        FileName = data.fileName, // notice the camel case (lowercase f)
        PartitionKey = "landscapes",
        RowKey = Guid.NewGuid().ToString(),
        Keywords = data.keywords // notice the camel case (lowercase k)
    };

    SaveToDatabase(metadata);

    return new OkObjectResult(metadata.RowKey);
}

Notice in the preceding code that you can also access information about the request such as req.ContentLength. Also note the lowercase f and k in data.fileName and data.keywords.

We can post the following JSON to the function:

{
    "fileName": "IMG0382435.jpg",
    "keywords": "landscape, sky, sunset"
}

Automatically Bind to Strongly Types POCOs in Azure HTTP Functions

You can also let the runtime auto-convert the POSTed JSON into a specified C# type:

[FunctionName("AddPhotoMetadata")]
public static IActionResult AddPhotoMetadata(
    [HttpTrigger(AuthorizationLevel.Function, "post")] PhotoMetadataAdditionRequest metadataAdditionRequest,
    ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    log.LogInformation($" FileName : {metadataAdditionRequest.FileName}");
    log.LogInformation($" Keywords : { metadataAdditionRequest.Keywords}");

    PhotoMetadata metadata = new PhotoMetadata
    {
        FileName = metadataAdditionRequest.FileName,
        PartitionKey = "landscapes",
        RowKey = Guid.NewGuid().ToString(),
        Keywords = metadataAdditionRequest.Keywords
    };

    SaveToDatabase(metadata);

    return new OkObjectResult(metadata.RowKey);
}

In the preceding code, instead of binding to a HttpRequest object,  we bind to the PhotoMetadataAdditionRequest. Behind the scenes the JSON will be automatically deserialized into a PhotoMetadataAdditionRequest object.

Note that if you have malformed JSON you may get errors. For example if the “fileName” item in the JSON was misspelt as “file” then the FileName property of the PhotoMetadata would end up being set to null but the function body would still execute. If you had an int in the POCO but the POSTed JSON had a string (e.g. “hello”) instead of a number, then the runtime cannot bind a “hello” to an int – in this case your function body code will not even execute and you get an error from the runtime such as: “System.Private.CoreLib: Exception while executing function: AddPhotoMetadata. Microsoft.Azure.WebJobs.Host: Exception binding parameter 'metadataAdditionRequest'. System.Private.CoreLib: Input string was not in a correct format.” (and a 500 will status be returned to the client).

If you were handling things at a lower level (e.g. with the dynamic approach) you could perhaps provide a default value, do some extra logging, etc.

Accessing HTTP Request Metadata When Auto-binding to POCOs

If you want to do automatic binding and also want to get request metadata, you can simply add an extra parameter of type HttpRequest:

[FunctionName("AddPhotoMetadata")]
public static IActionResult AddPhotoMetadata(
    [HttpTrigger(AuthorizationLevel.Function, "post")] PhotoMetadataAdditionRequest metadataAdditionRequest,
    HttpRequest req,
    ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    log.LogInformation("You can get additional information about the request such as:");
    log.LogInformation($" length : {req.ContentLength}");
    log.LogInformation($" type   : {req.ContentType}");
    log.LogInformation($" https  : {req.IsHttps}");
    log.LogInformation($" host   : {req.Host}");

    log.LogInformation($" FileName : {metadataAdditionRequest.FileName}");
    log.LogInformation($" Keywords : { metadataAdditionRequest.Keywords}");

    PhotoMetadata metadata = new PhotoMetadata
    {
        FileName = metadataAdditionRequest.FileName,
        PartitionKey = "landscapes",
        RowKey = Guid.NewGuid().ToString(),
        Keywords = metadataAdditionRequest.Keywords
    };

    SaveToDatabase(metadata);

    return new OkObjectResult(metadata.RowKey);
}

Posting Form Data to Azure Functions

In addition to POSTing JSON content to an Azure Function, you can also POST form data and access the HttpRequest.Form property:

[FunctionName("AddPhotoMetadata")]
public static IActionResult AddPhotoMetadata(
    [HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequest req,
    ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    log.LogInformation("You can get additional information about the request such as:");
    log.LogInformation($" length : {req.ContentLength}");
    log.LogInformation($" type   : {req.ContentType}");
    log.LogInformation($" https  : {req.IsHttps}");
    log.LogInformation($" host   : {req.Host}");

    PhotoMetadata metadata = new PhotoMetadata
    {
        FileName = req.Form["fileName"], // access form data
        PartitionKey = "landscapes",
        RowKey = Guid.NewGuid().ToString(),
        Keywords = req.Form["keywords"]  // access form data
    };

    SaveToDatabase(metadata);

    return new OkObjectResult(metadata.RowKey);
}

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

The Azure SignalR Service is a serverless offering from Microsoft to facilitate real-time communications without having to manage the infrastructure yourself.

SignalR itself has been around for a while, now the hosted/serverless version makes it even easier to consume.

There are also Azure Functions bindings available that make it easy to integrate SignalR with Azure Functions and end clients.

This means that any Azure Function with any trigger type (e.g. Azure Cosmos DB changes, queue messages, HTTP requests,  blob triggers, etc.) can push out a notification to clients via Azure SignalR Service.

In this article we’ll build a simple example that simulates the “someone in Austin just bought a Surface Laptop” kind of messages that you see on some shopping websites.

Creating an Azure SignalR Service Instance

After logging into the Azure Portal, create a new SignalR Service instance (search for “SignalR Service”) – you can currently choose a free pricing tier.

Give the new instance a name, in this example the  instance was called “dctdemosorderplaced”. Once you’ve filled out the info for the new instance (such as resource group and  location) hit the create button and wait for Azure to create the new instance.

Once the instance has been created, open it and head into the settings and change the service mode to Serverless and save the changes, as the following screenshot shows:

You will need to copy the connection string to be used in the function app later, you can get this from the Keys tab – copy the Primary connection string as shown in the following screenshot:

Creating the Azure Functions App

In Visual Studio (or Code) create a new Azure Functions project.

Open the local.settings.json file and make the following changes:

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet",
    "AzureSignalRConnectionString": "PASTE YOUR SIGNALR SERVICE CONNECTION STRING HERE"
  },
  "Host": {
    "LocalHttpPort": 7071,
    "CORS": "http://localhost:3872",
    "CORSCredentials": true
  }
}

In the preceding code notice 2 things:

1. The AzureSignalRConnectionString setting: paste your connection string here
2. The CORS entries to configure the local environment (http://localhost:3872 is whatever local IIS Express port you will run the test website from)

Add a couple of classes to represent an incoming order and an order stored in table storage (we could have also use Cosmos DB or another storage mechanism – it isn’t really relevant to the SignalR demo):

public class OrderPlacement
{
    public string CustomerName { get; set; }
    public string Product { get; set; }
}


// Simplified for demo purposes
public class Order
{
    public string PartitionKey { get; set; }
    public string RowKey { get; set; }
    public string CustomerName { get; set; }
    public string Product { get; set; }
}

We can now add a function that will allow a new order to be placed:

public static class PlaceOrder
{
    [FunctionName("PlaceOrder")]
    public static async Task<IActionResult> Run(
        [HttpTrigger(AuthorizationLevel.Function, "post")] OrderPlacement orderPlacement,
        [Table("Orders")] IAsyncCollector<Order> orders, // could use cosmos db etc.
        [Queue("new-order-notifications")] IAsyncCollector<OrderPlacement> notifications,
        ILogger log)
    {
        await orders.AddAsync(new Order
        {
            PartitionKey = "US",
            RowKey = Guid.NewGuid().ToString(),
            CustomerName = orderPlacement.CustomerName,
            Product = orderPlacement.Product
        });

        await notifications.AddAsync(orderPlacement);

        return new OkResult();
    }
}

The preceding function allows a new OrderPlacement to be HTTP POSTed to the function. The function saves it in table storage and also adds it to a storage queue.

Creating The Azure Functions with the SignalR Bindings

The first SignalR-related function is the function that a client calls to get itself wired-up to the the SignalR service in the cloud:

[FunctionName("negotiate")]
public static SignalRConnectionInfo GetOrderNotificationsSignalRInfo(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequest req,
    [SignalRConnectionInfo(HubName = "notifications")] SignalRConnectionInfo connectionInfo)
{
    return connectionInfo;
}

This function will be automatically called from the SignalR client code as we’ll see later in this article. Notice the function returns a SignalRConnectionInfo object to the client. This data contains the SignalR URL and an access token which the client can use. Also notice that the SignalRConnectionInfo binding allows the SignalR hub name to be specified, in this case “notifications”.

Now the client can negotiate a connection to the Azure SignalR Service via our Function App.

Now the client can get itself wired-up, we need to push some data to the client:

[FunctionName("PlacedOrderNotification")]
public static async Task Run(
    [QueueTrigger("new-order-notifications")] OrderPlacement orderPlacement,
    [SignalR(HubName = "notifications")] IAsyncCollector<SignalRMessage> signalRMessages,
    ILogger log)
{
    log.LogInformation($"Sending notification for {orderPlacement.CustomerName}");

    await signalRMessages.AddAsync(
        new SignalRMessage
        {
            Target = "productOrdered",
            Arguments = new[] { orderPlacement }
        });
}

The preceding function is triggered from the queue (“new-order-notifications”) that the initial HTTP function wrote to, but you can send SignalR messages from any triggered function.

Notice the [SignalR(HubName = "notifications")] binding specifies the same hub name as in the negotiate function.

To send notifications, you can simply add one or more SignalRMessage messages to the IAsyncCollector<SignalRMessage> In this function, the arguments contain the OrderPlacement object. The client will then be ale to access the customer name and product and display it on the website as a notification.

Creating an Azure SignalR Service JavaScript Client

Create a new empty ASP.NET Core project and add a default.html file under wwwroot. This example does not require any server-side code (controllers, etc.) in the demo web site, so a simple static HTML file is sufficient.

In the startup.cs configure default and static files:

public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
    if (env.IsDevelopment())
    {
        app.UseDeveloperExceptionPage();
    }

    app.UseDefaultFiles();
    app.UseStaticFiles();
}

Add the following content to the default.html file:

<html>

<head>
     <!--Adapted (aka hacked) from: https://azure-samples.github.io/signalr-service-quickstart-serverless-chat/demo/chat-v2/ -->

    <title>SignalR Demo</title>
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@4.1.3/dist/css/bootstrap.min.css">

    <style>
        .slide-fade-enter-active, .slide-fade-leave-active {
            transition: all 1s ease;
        }

        .slide-fade-enter, .slide-fade-leave-to {
            height: 0px;
            overflow-y: hidden;
            opacity: 0;
        }
    </style>
</head>

<body>
    <p>&nbsp;</p>
    <div id="app" class="container">
        <h3>Recent orders</h3>

        <div class="row" v-if="!ready">
            <div class="col-sm">
                <div>Loading...</div>
            </div>
        </div>
        <div v-if="ready">
            <transition-group name="slide-fade" tag="div">
                <div class="row" v-for="order in orders" v-bind:key="order.id">
                    <div class="col-sm">
                        <hr />
                        <div>
                            <div style="display: inline-block; padding-left: 12px;">
                                <div>
                                    <span class="text-info small"><strong>{{ order.CustomerName }}</strong> just ordered a</span>
                                </div>
                                <div>
                                    {{ order.Product }}
                                </div>
                            </div>
                        </div>
                    </div>
                </div>
            </transition-group>
        </div>
    </div>

    <script src="https://cdn.jsdelivr.net/npm/vue@2.5.17/dist/vue.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/@aspnet/signalr@1.1.2/dist/browser/signalr.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/axios@0.18.0/dist/axios.min.js"></script>

    <script>
        const data = {
          orders: [],
          ready: false
        };

        const app = new Vue({
          el: '#app',
          data: data,
          methods: {
          }
        });

        const connection = new signalR.HubConnectionBuilder()
          .withUrl('http://localhost:7071/api')
          .configureLogging(signalR.LogLevel.Information)
          .build();

        connection.on('productOrdered', productOrdered);
        connection.onclose(() => console.log('disconnected'));

        console.log('connecting...');
        connection.start()
            .then(() => data.ready = true)
            .catch(console.error);

        let counter = 0;

        function productOrdered(orderPlacement) {
        orderPlacement.id = counter++; // vue transitions need an id
        data.orders.unshift(orderPlacement);
    }
    </script>
</body>

</html>

The .withUrl('http://localhost:7071/api') line is the root of your locally running Azure Function development runtime environment.

When this page loads, the negotiate function will be called in the Function App and will return the Azure SignalR Service connection details so the client can connect to the hub in the cloud and send/receive messages (in this example we are just receiving messages in the JavaScript client).

Running the Demo Application

Right-click the solution in Visual Studio and select “Set Startup Projects…” and choose Multiple startup projects and set the Function App project to Start and the ASP.NET core project to Start without debugging. This just means you can hit F5 and both projects will run.

Once both projects are running you should be able to post some JSON to the PlaceOrder function HTTP endpoint (for example using Postman) containing the CustomerName and Product:

{
    "CustomerName" : "Amrit",
    "Product" : "Surface Book"
}

This will save the order to table storage and add a message to the queue.

The queue-triggered function will pick up this message an send a notification to the client and you should then see “Amrit just ordered a Surface Book” appear on the web site.

Note that this demo application allows anonymous/unauthenticated clients to access the functions and the SignalR Service, in a real-work production app you should ensure you secure the entire system appropriately – see docs for more info.

You can find this entire sample app on GitHub.

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

(This post refers to Azure Functions v2)

One way to test Azure Functions that use Event Grid triggers is to run the Function App locally and then get Azure in the cloud to invoke the function running on the local machine. As an example, suppose you want to use Event Grid to improve the reliability and responsiveness of Blob Storage processing. To do this the documentation suggests the use of ngrok. Now when a blob is added to a container in the cloud, the locally running function on the dev machine will be invoked via ngrok.

There is a somewhat simpler solution that allows you to invoke the Event Grid triggered function locally.

This approach bypasses Event Grid completely, so it is not a substitute for proper end-to-end testing, it’s more a development-time testing & debugging tool.

Manually Running Non HTTP-Triggered Azure Functions

You can manually trigger a non HTTP-triggered function (such as a timer triggered or Event Grid triggered function) via a special HTTP endpoint.

The endpoint is of the format: {host}/admin/functions/{function name}

For example, take the following function (which was also used in the post Improving Azure Functions Blob Trigger Performance and Reliability - Part 3: Using Event Grid to Respond to New Blobs):

public static class ProcessFoodBlobsEventGrid
{
    private static readonly string[] _meats = { "steak", "chicken", "venison" };

    [FunctionName("ProcessFoodBlobsEventGrid")]
    public static void Run(
     [EventGridTrigger]EventGridEvent blobCreatedEvent,
     [Blob("{data.url}")] string foods, // assumes small blob size so using string not stream
     [Blob("{data.url}.vegetarian")] out string vegetarian,
     [Blob("{data.url}.nonvegetarian")] out string nonVegetarian,
     ILogger log)
    {
        log.LogInformation("Processing a blob created event");

        StorageBlobCreatedEventData createdEvent = ((JObject)blobCreatedEvent.Data).ToObject<StorageBlobCreatedEventData>();

        log.LogInformation($"Blob: {createdEvent.Url}");
        log.LogInformation($"Api operation: {createdEvent.Api}");

        vegetarian = null;
        nonVegetarian = null;

        string[] foodLines = foods.Split(new[] { "\r\n", "\n" }, StringSplitOptions.RemoveEmptyEntries);


        foreach (var food in foodLines)
        {
            var isMeat = _meats.Contains(food);

            if (isMeat)
            {
                nonVegetarian += food + Environment.NewLine;
            }
            else
            {
                vegetarian += food + Environment.NewLine;
            }
        }
    }
}

The preceding function when running locally in development would have the special URL: http://localhost:7071/admin/functions/ProcessFoodBlobsEventGrid

If you had a timer-triggered function called HerdCats that you wanted to manually invoke (so you didn’t have to wait for the next timed invocation) the special URL would be:http://localhost:7071/admin/functions/HerdCats

Note: when running locally in development you do not have to authenticate. If you wanted to manually invoke a deployed function in Azure, you need to provide an x-functions-key header that contains the function master key.

Manually Invoking an Event Grid Triggered Azure Function

When using the special URL to invoke a function, you can also provide data to be passed to the function. The type of data passed will depend on the trigger type of the function that you are invoking.

To provide data to the function, a JSON payload can be posted to the special URL. The data that is passed to the function is contained in a JSON property called “input”:

{
    "input": "trigger data goes here"
}

If the Event Grid triggered function will be invoked by a new blob event, the contents of this input property must match the event schema for an Azure Blob Storage event.

An example of event JSON (taken from the Microsoft documentation):

[{
  "topic": "/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/xstoretestaccount",
  "subject": "/blobServices/default/containers/testcontainer/blobs/testfile.txt",
  "eventType": "Microsoft.Storage.BlobCreated",
  "eventTime": "2017-06-26T18:41:00.9584103Z",
  "id": "831e1650-001e-001b-66ab-eeb76e069631",
  "data": {
    "api": "PutBlockList",
    "clientRequestId": "6d79dbfb-0e37-4fc4-981f-442c9ca65760",
    "requestId": "831e1650-001e-001b-66ab-eeb76e000000",
    "eTag": "0x8D4BCC2E4835CD0",
    "contentType": "text/plain",
    "contentLength": 524288,
    "blobType": "BlockBlob",
    "url": "https://example.blob.core.windows.net/testcontainer/testfile.txt",
    "sequencer": "00000000000004420000000000028963",
    "storageDiagnostics": {
      "batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
    }
  },
  "dataVersion": "",
  "metadataVersion": "1"
}]

When testing the function outlined earlier, the first thing to do is ensure that there is a blob in the local blob container that will be read by the function by way of the blob input binding: [Blob("{data.url}")] string foods.

For example, in the Storage Emulator a blob called in.txt can be uploaded to the food-in container.

Now the new blob event data JSON needs to be modified, specifically the data.url property needs to contain the URL to the local blob: http://127.0.0.1:10000/devstoreaccount1/food-in/in.txt

A modified version with updated data.url would be as follows:

[{
  "topic": "/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/xstoretestaccount",
  "subject": "/blobServices/default/containers/testcontainer/blobs/testfile.txt",
  "eventType": "Microsoft.Storage.BlobCreated",
  "eventTime": "2017-06-26T18:41:00.9584103Z",
  "id": "831e1650-001e-001b-66ab-eeb76e069631",
  "data": {
    "api": "PutBlockList",
    "clientRequestId": "6d79dbfb-0e37-4fc4-981f-442c9ca65760",
    "requestId": "831e1650-001e-001b-66ab-eeb76e000000",
    "eTag": "0x8D4BCC2E4835CD0",
    "contentType": "text/plain",
    "contentLength": 524288,
    "blobType": "BlockBlob",
    "url": "http://127.0.0.1:10000/devstoreaccount1/food-in/in.txt",
    "sequencer": "00000000000004420000000000028963",
    "storageDiagnostics": {
      "batchId": "b68529f3-68cd-4744-baa4-3c0498ec19f0"
    }
  },
  "dataVersion": "",
  "metadataVersion": "1"
}]

The next step is to remove the surrounding [], and replace the “ with ‘. Then paste the resulting JSON into the input property:

{
    "input": "
  {
    'topic': '/subscriptions/{subscription-id}/resourceGroups/Storage/providers/Microsoft.Storage/storageAccounts/xstoretestaccount',
    'subject': '/blobServices/default/containers/oc2d2817345i200097container/blobs/oc2d2817345i20002296blob',
    'eventType': 'Microsoft.Storage.BlobCreated',
    'eventTime': '2017-06-26T18:41:00.9584103Z',
    'id': '831e1650-001e-001b-66ab-eeb76e069631',
    'data': {
      'api': 'PutBlockList',
      'clientRequestId': '6d79dbfb-0e37-4fc4-981f-442c9ca65760',
      'requestId': '831e1650-001e-001b-66ab-eeb76e000000',
      'eTag': '0x8D4BCC2E4835CD0',
      'contentType': 'application/octet-stream',
      'contentLength': 524288,
      'blobType': 'BlockBlob',
      'url': 'http://127.0.0.1:10000/devstoreaccount1/food-in/in.txt',
      'sequencer': '00000000000004420000000000028963',
      'storageDiagnostics': {
        'batchId': 'b68529f3-68cd-4744-baa4-3c0498ec19f0'
      }
    },
    'dataVersion': '',
    'metadataVersion': '1'
  }
"
}

Now this JSON can be POSTed to the special URL: in the case of the example in this post the URL would be: http://localhost:7071/admin/functions/ProcessFoodBlobsEventGrid

The following screenshot shows posting using Postman:

Posting will cause the Event Grid triggered function to be invoked and the JSON contained inside the input property will be passed to the trigger input EventGridEvent blobCreatedEvent object. The function will execute and read in the blob called “in.txt”.

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

(This article refers to Azure Functions v2)

Out of the box, Azure Functions comes with a range of triggers, input bindings, and output bindings to work with blobs, queues, HTTP, etc.

You can also create your own input and/or output bindings.

Overview of Custom Azure Function Bindings

The general process to create a custom binding is:

1. Create a class library (e.g. .NET Standard)
2. Create a class that implements IAsyncCollector<T>
3. Implement the AddAsync method from the interface and add code to perform some output
4. Create a custom C# attribute (check out my course if you’ve never created custom attributes before ) to represent the binding attribute that will be used in functions
5. Create a class that implements IExtensionConfigProvider that wires up the new binding
6. In the Function App, create a startup class to register your custom extension

An Example Scenario – Integrating with Pushover

Pushover is a service (with accompanying phone app) that lets you send notifications to your phone.There is a 7 day free trial to start with and the simplest way to get started is to search for the Pushover app in the app store on your phone. Once registered you can follow the instructions to set up your application/device that push notifications can be sent to. At the end of this process, ultimately you will end up with a user key and and application API token key. Both of these are needed when calling the Pushover API.

In the following example, a custom Azure Function output binding will be created that can be used from any Azure Function to send notifications. The output could be anything however, for example you could replace the Pushover API call with a Twitter call, LinkedIn call, etc.

The following example creates an output binding but you can also create bindings that get input and passes it to a function.

Creating a POCO to Represent a Notification Message

In the new class library project, add a class:

namespace PushoverBindingExtensions
{
    public class PushoverNotification
    {
        public string Title { get; set; }
        public string Message { get; set; }
    }
}

Implementing an IAsyncCollector

The next step is to create a class that implements IAsyncCollector<T> where T is the “data” that we want to pass to the output binding; in our case its the custom POCO class we just created but this could be a primitive type such as a string.

using Microsoft.Azure.WebJobs;
using System.Collections.Generic;
using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;

namespace PushoverBindingExtensions
{
    internal class PushoverNotificationAsyncCollector : IAsyncCollector<PushoverNotification>
    {
        private static readonly HttpClient _httpClient = new HttpClient();

        private PushoverAttribute _pushoverAttribute;

        public PushoverNotificationAsyncCollector(PushoverAttribute attribute)
        {
            _pushoverAttribute = attribute;
        }

        public async Task AddAsync(PushoverNotification notification, CancellationToken cancellationToken = default(CancellationToken))
        {
            await SendNotification(notification);
        }

        public Task FlushAsync(CancellationToken cancellationToken = default(CancellationToken))
        {
            return Task.CompletedTask;
        }

        private async Task SendNotification(PushoverNotification notification)
        {
            var parameters = new Dictionary<string, string>
                {
                    { "token", _pushoverAttribute.AppToken },
                    { "user", _pushoverAttribute.UserKey },
                    { "title", notification.Title },
                    { "message", notification.Message }
                };

            var response = await _httpClient.PostAsync("https://api.pushover.net/1/messages.json", new FormUrlEncodedContent(parameters));
            response.EnsureSuccessStatusCode();
        }
    }
}

The key point in the preceding code is the implemented AddAsync method. This method gets called by your function when you use the binding, in this example it’s calling into the SendNotification method that talks to the Pushover API.

Notice that the constructor takes an instance of a PushoverAttribtue which we’ll define next.

Defining a Custom Binding Attribute for Azure Functions

The following code defines a .NET attribute that will be used to decorate parameters in function run methods:

using Microsoft.Azure.WebJobs.Description;
using System;

namespace PushoverBindingExtensions
{
    [Binding]
    [AttributeUsage(AttributeTargets.Parameter | AttributeTargets.ReturnValue)]
    public class PushoverAttribute : Attribute
    {
        public PushoverAttribute(string appToken, string userKey)
        {
            AppToken = appToken;
            UserKey = userKey;
        }

        [AutoResolve]        
        public string AppToken { get; set; }

        [AutoResolve]       
        public string UserKey { get; set; }
    }
}

Note in the preceding code that the [Binding] attribute has been applied and the attribute has been limited to use on parameters and return values. (You can learn more about how to create custom attributes in my Pluralsight course).

This attribute definition has a couple of string properties to represent the Pushover user and app tokens. Notice these properties have been decorated with the [AutoResolve] attribute. This enables binding expressions for the properties and allows them to be read from AppSettings with %% syntax.

Creating the Azure Function Binding Custom Extension

The next step is to create a class that implements the IExtensionConfigProvider interface. This class will define the rules that are applicable to the custom binding:

using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Description;
using Microsoft.Azure.WebJobs.Host.Config;

namespace PushoverBindingExtensions
{
    [Extension("PushoverExtensions")]
    public class PushoverExtensions : IExtensionConfigProvider
    {
        public void Initialize(ExtensionConfigContext context)
        {
            var rule = context.AddBindingRule<PushoverAttribute>();            

            rule.BindToCollector<PushoverNotification>(BuildCollector);            
        }

        private IAsyncCollector<PushoverNotification> BuildCollector(PushoverAttribute attribute)
        {            
            return new PushoverNotificationAsyncCollector(attribute);
        }
    }
}

The preceding class is decorated with the [Extension] attribute to mark the class as an extension and the Initialize method is where the binding rules are defined. In this method we can add binding rules and also custom convertors (for example if we wanted to be able to bind to IAsyncCollector<string> and this be automatically converted to IAsyncCollector<PushoverNotification>). In this example we’re not defining any such convertors.

Once the binding rule has been added (for the PushoverAttribute we created earlier) it can be configured as an output binding by calling the BindToCollector method. This method is used to create an instance of the PushoverNotificationAsyncCollector we created earlier, in this example this is done by calling the BuildCollector method that returns a IAsyncCollector<PushoverNotification> instance.

So hopefully your still with me; at this point we have the custom binding created in the class library project, we can now actually use it in our functions.

Using a Custom Binding in an Azure Function

In the function app project, add a reference to the class library project containing the custom binding.

We can now use the custom binding just as we would any of the pre-supplied ones as the following function code demonstrates:

[FunctionName("SendPushoverNotification")]
public static async Task<IActionResult> Run(
    [HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequest req,
    [Pushover("%appkey%", "%userkey%")] IAsyncCollector<PushoverNotification> notifications,
    ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    // validation omitted for demo purposes
    string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
    dynamic data = JsonConvert.DeserializeObject(requestBody);

    await notifications.AddAsync(new PushoverNotification { Title = data.Title, Message = data.Message });

    return new OkResult();
}

The preceding function code just happens to have an HTTP trigger but the custom binding can be used in functions with other triggers as well.

Notice the [Pushover] custom binding attribute being applied. The attribute decorates the notifications parameter that is of type IAsyncCollector<PushoverNotification>. (Hopefully it’s a bit clearer now how all the parts fit together…).

Now in the function body code, a notification can be send by calling the AddAsync method and passing the PushoverNotification to be sent.

Also notice in the [Pushover] attribute the binding expressions to the app key and user key stored in app settings (or local.settings.json in the development environment) This means these sensitive keys do not need to be hard coded.

There are a few final steps to getting this all to work.

The first is to register the custom binding extension by creating the following Startup class in the function app project:

using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Hosting;
using PushoverBindingExtensions;

namespace DontCodeTiredDemosV2
{
    public class Startup : IWebJobsStartup
    {
        public void Configure(IWebJobsBuilder builder)
        {
            builder.AddExtension<PushoverExtensions>();
        }
    }
}

And in the AssemblyInfo.cs (which you can create manually) add the following:

using Microsoft.Azure.WebJobs.Hosting;

// Register custom extension of Function App startup
[assembly: WebJobsStartup(typeof(DontCodeTiredDemosV2.Startup))]

This points to the Startup class we just created.

Testing It All Out

Now just run the function app and post the following JSON to the function address (e.g. http://localhost:7071/api/SendPushoverNotification):

{
    "Title" : "Functions App",
    "Message" : "I like cheese!"
}

This will result in the Azure Function executing and sending a request to the Pushover API, will will result in a notification arriving on your phone as the following screenshot shows:

I hope that’s help clarify the process a  little on how to create custom binding expressions in Azure Functions. If you like me to throw the code up on GitHub let me know :)

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

In the this final part of this series we wrap up by briefly discussing some ways to check for blobs that have not been processed correctly.

When using Azure Functions, a timer trigger can be used to automatically periodically execute a function based on a CRON expression. The following code is and example of a timer-triggered function:

public static class CheckBlobs
{
    [FunctionName("CheckBlobs")]
    public static void Run(
        [TimerTrigger("0 */5 * * * *")]TimerInfo myTimer, 
        ILogger log)
    {
        log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");

        // Blob checking logic here
    }
}

There are a number of ways to check for unprocessed blobs depending on the solution you are building, some examples:

• Check that an output blob exists for every input blob
• Use a database to keep track of blobs that were uploaded and compare this to actual output blobs
• If blobs are deleted after they have been processed, check there are no blobs in the container
• Etc.

Some things to bear in mind if implementing this kind of checking include:

• How often/when to run the function?
• How long after a blob is written should you give it to be processed normally?
• Will running this function interfere with any other processing in the system?
• What if a blob is due to be processed (e.g. message sitting in a queue but not yet processed)? Could this create false positives or cause duplication of processing?
• How long does the checking function take to execute? Will it take too long as the number of blobs increases and will the function be terminated by the runtime?
• How/who do you notify of missed blobs (email, SMS, create ticket in CRM/bug system, etc.)?
• Do you try to perform auto-retry of processing? Again, could this cause duplication, errors, etc.?

You could also use logging/Application Insights to provide you with information, or write every incoming blob name to a database and update that record when a blob has been processed, this way unprocessed blobs can be found with a simple “IF NOT PROCESSED” query.

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

In the previous part of the series we saw how to improve the reliability of responding to new blobs by introducing a queue.This required the introduction of a Storage Queue to the solution and also that the writer of new blobs also write a queue message.

In this article, instead of manually writing messages to a queue on blob creation, we use Event Grid events.

Azure Event Grid has support for Blob Storage, meaning that when a new blob is written, Event Grid will notice this. We can then trigger an Azure Function from this Event Grid event.

This approach can improve the reliability and responsiveness compared to using a simple blob trigger: “Blob storage events are reliably sent to the Event grid service which provides reliable delivery services to your applications through rich retry policies and dead-letter delivery.” [Microsoft]

Creating an Event Grid Triggered Function

The following Azure Function code is a modified version of the code used in the previous article:

public static class ProcessFoodBlobsEventGrid
{
    private static readonly string[] _meats = { "steak", "chicken", "venison" };

    [FunctionName("ProcessFoodBlobsEventGrid")]
    public static void Run(
     [EventGridTrigger]EventGridEvent blobCreatedEvent,
     [Blob("{data.url}")] string foods, // assumes small blob size so using string not stream
     [Blob("{data.url}.vegetarian")] out string vegetarian,
     [Blob("{data.url}.nonvegetarian")] out string nonVegetarian,
     ILogger log)
    {
        log.LogInformation("Processing a blob created event");

        StorageBlobCreatedEventData createdEvent = ((JObject)blobCreatedEvent.Data).ToObject<StorageBlobCreatedEventData>();

        log.LogInformation($"Blob: {createdEvent.Url}");
        log.LogInformation($"Api operation: {createdEvent.Api}");

        vegetarian = null;
        nonVegetarian = null;

        string[] foodLines = foods.Split(new[] { "\r\n", "\n" }, StringSplitOptions.RemoveEmptyEntries);


        foreach (var food in foodLines)
        {
            var isMeat = _meats.Contains(food);

            if (isMeat)
            {
                nonVegetarian += food + Environment.NewLine;
            }
            else
            {
                vegetarian += food + Environment.NewLine;
            }
        }
    }
}

In the preceding code, the [EventGridTrigger]EventGridEvent blobCreatedEvent will cause the function to be trigged based on an Event Grid event being directed to the function.

The input blob binding [Blob("{data.url}")] string foods uses a binding expression and accesses the data.url property from the JSON data that’s contained in the event (this comes from the event schema for Blob Storage). The 2 output bindings also use the original blob path/name and append .vegetarian or .nonvegetarian. This implementation writes output blobs to the same container as the input blob. You could also use dynamic binding in Azure Functions with imperative runtime bindings to just extract the filename from the blob and write the output blobs to a different container.

Creating an Event Subscription for New Blobs

The function needs an event subscription to be created in Azure to recognize when new blobs are written and invoke the function. This can be done by navigating to the storage account (requires storage account v2) in the Azure Portal and clicking the Events link. You can then add a new event subscription as the following screenshot shows (note the Defined Event Types is set to Blob Created):

You can also specify subject filters to limit the event to a specific container and/or file type as the following screenshot shows:

You could also specify dead-lettering and retry policies in case the Function App is unable to respond.

Now when a blob is added, the event subscription will notice it and invoke the function.

Ultimately “Use the Event Grid trigger instead of the Blob storage trigger for blob-only storage accounts, for high scale, or to reduce latency.” [Microsoft]

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

This is the second part of a series or articles.

When you add a new blob, your blob-triggered function may not be triggered immediately: “If the blob container being monitored contains more than 10,000 blobs, the Functions runtime scans log files to watch for new or changed blobs. This process can result in delays. A function might not get triggered until several minutes or longer after the blob is created.” [Microsoft]

Also when scanning log files to find new blobs that need processing, there’s “no guarantee that all events are captured. Under some conditions, logs may be missed.” [Microsoft]

This means that it is possible for some new blobs to be missed and not processed.

Using a Storage Queue to Trigger Processing of New Blobs

One alternative to reduce the likelihood of missed blobs and also improve the responsiveness of blob processing is to use a slightly more  complex (but still relatively straight forward) approach.

Essentially this alternative approach has the following workflow:

1. New blob written to blob storage
2. Write message to storage queue containing new blob path
3. Queue-triggered function gets message from step 2
4. Blob processing occurs

(Note that this alternative approach may not suit all situations depending on how new blobs are making their way into blob storage – who or whatever is writing the blob in step 1 also needs to be able to write a queue message.)

Blob Writing

This approach requires that when a blob is written, a queue message is also written.

As a simple example, this could be from client code as follows:

using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Blob;
using System.IO;
using System.Threading.Tasks;

namespace AddNewBlob
{
    class Program
    {
        static async Task Main(string[] args)
        {
            CloudStorageAccount storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
            CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
            CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference("food-in");
            CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference("recipe1.txt");

            await WriteBlob();
            await WriteMessage();

            async Task WriteBlob()
            {
                using (var stream = await cloudBlockBlob.OpenWriteAsync())
                using (var sw = new StreamWriter(stream))
                {
                    await sw.WriteLineAsync("carrot");
                    await sw.WriteLineAsync("steak");
                    await sw.WriteLineAsync("apple");
                }
            }

            async Task WriteMessage()
            {
                var queueClient = storageAccount.CreateCloudQueueClient();
                var queue = queueClient.GetQueueReference("food-in");
                await queue.AddMessageAsync(new Microsoft.WindowsAzure.Storage.Queue.CloudQueueMessage("recipe1.txt"));
            }
        }

        
    }
}

Or perhaps the blob data comes in via an HTTP-triggered function as follows:

public static class AddRecipe
{
    [FunctionName("AddRecipe")]
    [return: Queue("food-in")]
    public static async Task<string> Run(
        [HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req,            
        ILogger log)
    {
        log.LogInformation("C# HTTP trigger function processed a request.");
        
        string ingredients = await new StreamReader(req.Body).ReadToEndAsync();

        // validation/error code omitted for demo purposes

        var blobName = Guid.NewGuid().ToString();

        await WriteBlob(); // ensure blob is written *before* function returns and add message to the queue

        return blobName; // write to queue


        async Task WriteBlob()
        {
            var account = CloudStorageAccount.DevelopmentStorageAccount; // In real app load this from secure config location
            var blobClient = account.CreateCloudBlobClient();
            var blobContainer = blobClient.GetContainerReference("food-in");
            var cloudBlockBlob = blobContainer.GetBlockBlobReference(blobName);
            await cloudBlockBlob.UploadTextAsync(ingredients);
        }
    }
}

Notice in the preceding code , the writing of the blob is being done explicitly in code to ensure that the queue message isn’t added until the blob is definitely available to be processed by the next function in the chain. (See this related GitHub issue).

More Reliable Blob Processing

The next function is where the actual processing of the new blob is carried out, it is however triggered from a queue rather than relying on a blob trigger:

public static class ProcessFoodBlobs
{
    private static readonly string[] _meats = { "steak", "chicken", "venison" };      

    [FunctionName("ProcessFoodBlobs")]
    public static void Run(
        [QueueTrigger("food-in")]string newBlobPath, 
        [Blob("food-in/{queueTrigger}")] string foods,
        [Blob("food-out/{queueTrigger}.vegetarian")] out string vegetarian,
        [Blob("food-out/{queueTrigger}.nonvegetarian")] out string nonVegetarian,
        ILogger log)
    {
        vegetarian = null;
        nonVegetarian = null;

        string[] foodLines = foods.Split(new[] {"\r\n", "\n"  }, StringSplitOptions.RemoveEmptyEntries);


        foreach (var food in foodLines)
        {
            var isMeat = _meats.Contains(food);

            if (isMeat)
            {
                nonVegetarian += food + Environment.NewLine;
            }
            else
            {
                vegetarian += food + Environment.NewLine;
            }
        }    
    }
}

In the preceding code we’re making use of automatic input blob binding.

Summary

This approach may offer some benefits at the cost of some additional complexity if you have a lot of blobs being written/stored/processed. It also has some other considerations to bear in mind such as what happens if the blob is deleted or changed before the message is picked up off the queue? As with all things you should consider your own requirements and ensure you do thorough testing which includes performance/load/stress testing.

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

This is the first part of a series or articles.

When creating blob-triggered Azure Functions there are some memory usage considerations to bear in mind.

“The consumption plan limits a function app on one virtual machine (VM) to 1.5 GB of memory. Memory is used by each concurrently executing function instance and by the Functions runtime itself.” [Microsoft]

A blob-triggered function can execute concurrently and internally uses a queue: “the maximum number of concurrent function invocations is controlled by the queues configuration in host.json. The default settings limit concurrency to 24 invocations. This limit applies separately to each function that uses a blob trigger.” [Microsoft]

So, if you have 1 blob-triggered function in a Function App, with the default concurrency setting of 24, you could have a maximum of 24 (1 * 24) concurrently executing function invocations. (The documentation describes this as per-VM concurrency, with 2 VMs you could have 48 (2vm * 1 * 24 concurrently executing function invocations.)

If you had 3 blob-triggered functions in a Function App (assuming 1 VM) then you could have 72 (3 * 24) concurrently executing function invocations.

Because the consumption plan “limits a function app on one virtual machine (VM) to 1.5 GB of memory”, if you are processing blobs that are non-trivial in size then you may need to consider overall memory usage.

OutOfMemoryException When Using Azure Functions Blob Trigger

As an example, suppose the following function exists:

public static class BlobPerformanceAndReliability
{
    [FunctionName("BlobPerformanceAndReliability")]
    public static void Run(
        [BlobTrigger("big-blobs/{name}")]string blob, 
        string name, 
        [Blob("big-blobs-out")] out string foundData,
        ILogger log)
    {
        log.LogInformation($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {blob.Length} Bytes");

        // Code to find and output a specific line
        foundData = "This line will never be reached if out of memory";
    }
}

The preceding function code is triggered by blobs in the big-blobs container, the omitted code towards the end of the function would find a specific line of text in the blob and output it to big-blobs-out.

We can create a large file (appx. 1.8 GB) with the following code in a console app:

using System.IO;

namespace ConsoleApp1
{
    class Program
    {
        static void Main(string[] args)
        {
            using (var sw = new StreamWriter(@"c:\temp\bigblob.txt"))
            {
                for (int i = 0; i < 40_000_000; i++)
                {
                    sw.WriteLine("Some line we are not interested in processing");
                }
                sw.WriteLine("Data: 42");
            }
        }
    }
}

The contents of the last line in the file will be set to “Data: 42”.

If we run the function app locally and upload this big file to the Azure Storage Emulator, the function will trigger and will error with: “System.Private.CoreLib: Exception while executing function: BlobPerformanceAndReliability. Microsoft.Azure.WebJobs.Host: One or more errors occurred. (Exception binding parameter 'blob') (Exception binding parameter 'name'). Exception binding parameter 'blob'. System.Private.CoreLib: Exception of type 'System.OutOfMemoryException' was thrown.”.

The reason for this is that when you bind a blob trigger/input and bind to string or byte[] the entire blob will be read into memory, if the blob is too big (and/or there are other function invocations executing concurrently also processing big files) it will exceed the memory restrictions of the Functions Runtime.

Processing Large Blobs with Azure Functions

Instead of binding to string or byte[], you can bind to a Stream. This will not load the entire blob into memory and will allow you to instead process it incrementally.

The function can be re-written as follows:

public static class BlobPerformanceAndReliability
{
    [FunctionName("BlobPerformanceAndReliability")]
    public static void Run(
        [BlobTrigger("big-blobs/{name}")]Stream blob,
        string name,
        [Blob("big-blobs-out/{name}")] out string foundData,
        ILogger log)
    {
        log.LogInformation($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {blob.Length} Bytes");

        // Code to find and output a specific line            

        foundData = null; // Don't write an output blob by default

        string line;

        using (var sr = new StreamReader(blob))
        {                
            while (!sr.EndOfStream)
            {
                line = sr.ReadLine();

                if (line.StartsWith("Data"))
                {
                    foundData = line;
                    break;
                }                    
            }
        }            
    }
}

If you’re not familiar with using streams in .NET, check out my Working with Files and Streams in C# Pluralsight course.

If we force the same blob to be reprocessed with this new function code, there will be no error and the output blob containing “Data: 42” will be seen in the big-blobs-out container.

Another thing to bear in mind when processing large files is that there is a timeout on function execution.

In the next part of this series we’ll look at how to improve the responsiveness of function execution when new blobs are written and also improve the reliability and reduce the chances of blobs being missed.

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE:

I started using the Three Wins Technique about five years ago. It has proven to be a simple technique to help me focus on, and deliver, what is important.

The Three Wins Technique is very simple to implement, you can do it on paper, sticky notes, OneNote, Evernote or any other tool for that matter. You can read more about it in this 2014 article. I first learned about the concept from Getting Results the Agile Way, and while I don’t use that whole system, I’ve found that using the idea of three wins is beneficial as a standalone technique.

A Quick Overview of the Three Wins Technique

Essentially you define (and actually write down) what three things you want to get done. Sounds simple but it can be incredibly powerful when done properly. The wins should not just be simply tasks to be ticked off, they should be “wins” – things that make you feel a sense of accomplishment and progress when you complete them.

How I Use the Three Wins Technique

Wins can be defined for whatever timeframe you want (weekly, monthly, etc.). I define wins at four levels:

1. What are the 3 wins for this year (I define these in January each year).
2. What are the 3 wins for this month (defined at the start of each month).
3. What are the 3 wins for this week (defined on Monday morning).
4. What are the 3 wins for today.

When defining the daily wins I glance at this week’s wins. When I’m, writing down this week’s 3 wins I glance at this month’s wins. When I’m writing down this month’s wins I glance at this year’s 3 wins.

This method means that the overarching wins for the year are always kept in mind when defining more granular wins. The more granular wins (daily, weekly) should usually contribute to progress on the year’s wins. This isn’t always the case though as other things still need to be done that don’t directly contribute to a yearly win.

If you find that your more granular wins are almost never contributing to the yearly wins then you either have a problem with your focus or incorrect or unrealistic yearly wins. - Tweet This

I use OneNote for most of my note taking/planning/etc. Below is a screenshot mock-up of what my 3 wins page looks like, I create a new page at the start of each year so I can look back at all the wins from the previous years.

Notice in the preceding screenshot, one of the Jan wins is marked with a “!” – I use this when I fail to achieve a win, this helps me to notice failure patterns.

You could have separate 3 wins pages for work life, personal life, fitness/health, etc. or combine categories – whatever works best for you. I tend to use the technique for work-related things, but given the success I’ve had with it I should probably start using it in other areas of my life. I may also start to create 3 wins for the decade to provide some long term perspective.

If you give this technique a try and find it useful, please share this page with your fellow developers, friends, and colleagues.

SHARE:

(This article applies to Azure Functions V2)

An Azure Function can be triggered by new blobs being written (or updated). If an unhandled exception occurs in the function, by default Azure Functions will retry the blob 5 times. This means the function will be triggered again for the same blob up to 5 times. If the same blob causes errors 5 times, no further attempts will be made and the processing of the blob will be “lost”.

Understanding Blob Processing Errors in Azure Functions

When a new (or updated) blob triggers a function, the Azure Functions runtime makes sure that the same blob is not processed twice (if no error occurs in the function execution). To do this the runtime makes use of “blob receipts”. These are stored in the Azure storage account associated with the function app (as defined in the AzureWebJobsStorage Function App settings).

As an example, suppose a new blob (called “followupletterrequest.data”) triggered the following function:

class FollowupLetterRequest
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
}

public static class PoisonBlobExampleFunctions
{
    [FunctionName("PoisonBlobExampleFunctions")]
    public static void Run(
        [BlobTrigger("followup-letters/{blobname}.data")]string blobData, 
        string blobname,
        [Blob("followup-letters/{blobname}.txt")] out string letter,
        ILogger log)
    {
        var settings = new JsonSerializerSettings
        {
            MissingMemberHandling = MissingMemberHandling.Error
        };

        // This code assumes blob JSON is valid, if not an exception will be thrown
        var request = JsonConvert.DeserializeObject<FollowupLetterRequest>(blobData, settings);

        string firstName = request.FirstName;
        string lastName = request.LastName;

        letter = RenderFollowUpLetterText(firstName, lastName);
    }
    
    private static string RenderFollowUpLetterText(string firstName, string lastName)
    {
        string simulateLetterText = WaffleEngine.Text(paragraphs: 3, includeHeading: false);

        return $"Dear {firstName} {lastName}\r\n \r\n{simulateLetterText}";
    }
}

After the function runs, in the storage account under a path like “azure-webjobs-hosts/blobreceipts” the blob receipt can be seen. On a development machine using the local storage emulator the full path would be something like: “blobreceipts/desktop/DontCodeTiredDemosV2.PoisonBlobExampleFunctions.Run/"0x8D69224161F4590"/followup-letters/followupletterrequest.data”.

This full path to the blob receipt blob represents:

• Function Id that the blob triggered (DontCodeTiredDemosV2.PoisonBlobExampleFunctions.Run)
• Blob Container Name (followup-letters)
• Name of triggering blob (followupletterrequest.data)
• Triggering blob version ETag (“0x8D69224161F4590”)

If we now added another new blob called “followupletterrequest_bad.data” that contains bad data (e.g. a missing JSON property), so that an exception is thrown, a second blob receipt will be generated: “blobreceipts/desktop/DontCodeTiredDemosV2.PoisonBlobExampleFunctions.Run/"0x8D692245985E910"/followup-letters/followupletterrequest_bad.data”.

Because this blob generated an error, after the default number of retries (5) there will be no more attempts to process it.

Manually Forcing a Blob to Be Reprocessed

The documentation states that if the blob receipt is manually deleted, this will force the blob to reprocessed. This may be suitable to force reprocessing of a set of blobs that failed processing due to some transient error such as a database or network being temporarily offline. You should obviously take care that reprocessing blobs wont cause problems such as duplicate orders, emails, etc. or other errors in the system. You  may also need to consider what would happen if blobs are retried in a different order and/or interleaved with new blobs being added. Also blobs may not be reprocessed immediately. Using the local function runtime development environment, once the blob receipt has been deleted, it seems that the function app needs restarting to cause the blob to be reprocessed (either that or I didn’t wait long enough…). Once deployed to Azure there can be a delay between when the blob receipt is deleted and the blob being retried, the following timeline shows the delay between the blob receipt being deleted and the retry attempt 1.

2019-02-14 03:40:24.374 <attempt 1 - failure>
2019-02-14 03:40:24.763 <attempt 2 - failure>
2019-02-14 03:40:24.891 <attempt 3 - failure>
2019-02-14 03:40:25.007 <attempt 4 - failure>
2019-02-14 03:40:25.117 <attempt 5 - failure>
<blob receipt deleted>
2019-02-14 04:24:24.327 <retry attempt 1 - failure>
2019-02-14 04:24:25.155 <retry attempt 2 - failure>
2019-02-14 04:24:25.288 <retry attempt 3 - failure>
2019-02-14 04:24:25.455 <retry attempt 4 - failure>
2019-02-14 04:24:25.592 <retry attempt 5 - failure>

Automatically Responding to Blob Failures in Azure Functions

When a blob fails for the last time, information about the failure will written as a message to a Storage queue called “webjobs-blobtrigger-poison”. The message contains a JSON payload describing the triggering blob that didn’t complete processing successfully, for example:

{
  "Type": "BlobTrigger",
  "FunctionId": "DontCodeTiredDemosV2.PoisonBlobExampleFunctions.Run",
  "BlobType": "BlockBlob",
  "ContainerName": "followup-letters",
  "BlobName": "followupletterrequest_bad.data",
  "ETag": "\"0x8D692245985E910\""
}

The information contained in the JSON can be used to alert support people about the error and take appropriate action as required such as writing to a support ticket database or sending an email. You could also implement logic to automatically delete the blob receipt to force reprocessing but there would probably want to be some retry count otherwise bad data could cause an infinite processing loop. Exactly how you handle failed blob processing will depend on the business scenario.

As an example, the following function monitors the “webjobs-blobtrigger-poison” queue and grabs the information about the failed blob:

[FunctionName("PoisonBlobQueueProcessor")]
public static void PoisonBlobQueueProcessor(
    [QueueTrigger("webjobs-blobtrigger-poison")] string message,
    ILogger log)
{
    var poisonBobDetails = JsonConvert.DeserializeObject<dynamic>(message);

    log.LogInformation($"Found an unprocessed blob {poisonBobDetails.ContainerName}/{poisonBobDetails.BlobName}\r\n");
    
    // Send an email, log a ticket in a fault system, log a CRM issue, etc.            
}

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE: