How To Notify Clients of Cosmos DB Changes with Azure SignalR and Azure Functions

This is the fourth part in a series of articles.

The Cosmos DB Azure Functions trigger can be used in conjunction with the Azure SignalR Service to create real-time notifications of changes made to data in Cosmos DB, all in a serverless way.

Azure Cosmos DB Change Notifications with Azure Functions and SignalR

If you want to learn more about how to setup the SignalR Service, check out this previous article.

In this series we’ve been using the domain of pizza delivery. In this article we’ll see how  updates in Cosmos DB can trigger a notification on an HTML client – you may have seen this if you’ve ordered pizza online where the delivery driver is tracked by GPS on a map.

The first thing to do is set up an Azure SignalR Service in Azure and add the SignalR Service connection string in the local.settings.json file which was covered in this article.

We can now add the negotiate function:

[FunctionName("negotiate")]
public static SignalRConnectionInfo GetDriverLocationUpdatesSignalRInfo(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post")] HttpRequest req,
    [SignalRConnectionInfo(HubName = "driverlocationnotifications")] SignalRConnectionInfo connectionInfo)
{
    return connectionInfo;
}

The preceding code sets the SignalR hub to be used in this example as “driverlocationnotifications”, we’ll use the same hub name in the function that actually sends updates to clients.

The next thing to do is add a function that is triggered when changes are made to the location of pizza delivery drivers (note the trigger will also fire for new documents that are added):

[FunctionName("PizzaDriverLocationUpdated")]
public static async Task Run([CosmosDBTrigger(
    databaseName: "pizza",
    collectionName: "driver",
    ConnectionStringSetting = "pizzaConnection")] IReadOnlyList<Document> modifiedDrivers,
    [SignalR(HubName = "driverlocationnotifications")] IAsyncCollector<SignalRMessage> signalRMessages,
    ILogger log)
{
    if (modifiedDrivers != null)
    {
        log.LogInformation($"Total modified drivers: {modifiedDrivers.Count}");

        foreach (var modifiedDriver in modifiedDrivers)
        {
            var driverName = modifiedDriver.GetPropertyValue<string>("Name");
            var driverLat = modifiedDriver.GetPropertyValue<double>("Latitude");
            var driverLong = modifiedDriver.GetPropertyValue<double>("Longitude");

            log.LogInformation($"Driver {modifiedDriver.Id} {driverName} was updated (lat,long) {driverLat}, {driverLong}");

            var message = new DriverLocationUpdatedMessage
            {
                DriverId = modifiedDriver.Id,
                DriverName = driverName,
                Latitude = driverLat,
                Longitude = driverLong
            };

            await signalRMessages.AddAsync(new SignalRMessage
            {
                Target = "driverLocationUpdated",
                Arguments = new[] { message }
            });
        }
    }
}

The preceding code uses the [CosmosDBTrigger] to fire the function when there are changes made to the “driver” collection in the “pizza” database.

The [SignalR] binding attribute allows outgoing SignalR messages to be sent to connected clients by adding messages to the IAsyncCollector<SignalRMessage> signalRMessages object. Notice that the type of the Cosmos DB trigger is IReadOnlyList<Document>, to get the actual data items of the document we use the GetPropertyValue method, for example to get the driver name: modifiedDriver.GetPropertyValue<string>("Name").

In the HTML, we can connect to the “driverlocationnotifications” SignalR hub, and then get notifications when any driver location changes, in reality we would probably only want to get messages for the driver that is delivering our pizza but for simplicity we won’t worry about it in this demo code.

The following is the HTML to make this work:

<html>

<head>
     <!--Adapted from: https://azure-samples.github.io/signalr-service-quickstart-serverless-chat/demo/chat-v2/ -->

    <title>SignalR Demo</title>
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@4.1.3/dist/css/bootstrap.min.css">

    <style>
        .slide-fade-enter-active, .slide-fade-leave-active {
            transition: all 1s ease;
        }

        .slide-fade-enter, .slide-fade-leave-to {
            height: 0px;
            overflow-y: hidden;
            opacity: 0;
        }
    </style>
</head>

<body>
    <p>&nbsp;</p>
    <div id="app" class="container">
        <h3>Your Pizza Is On Its Way!!</h3>

        <div class="row" v-if="!ready">
            <div class="col-sm">
                <div>Loading...</div>
            </div>
        </div>
        <div v-if="ready">
            <transition-group name="slide-fade" tag="div">
                <div class="row" v-for="driver in drivers" v-bind:key="driver.DriverId">
                    <div class="col-sm">
                        <hr />
                        <div>
                            <div style="display: inline-block; padding-left: 12px;">
                                <div>
                                    <span class="text-info small"><strong>{{ driver.DriverName }}</strong> just changed location:</span>
                                </div>
                                <div>
                                    {{driver.Latitude}},{{driver.Longitude}}
                                </div>
                            </div>
                        </div>
                    </div>
                </div>
            </transition-group>
        </div>
    </div>

    <script src="https://cdn.jsdelivr.net/npm/vue@2.5.17/dist/vue.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/@aspnet/signalr@1.1.2/dist/browser/signalr.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/axios@0.18.0/dist/axios.min.js"></script>

    <script>
        const data = {
          drivers: [],
          ready: false
        };

        const app = new Vue({
          el: '#app',
          data: data,
          methods: {
          }
        });

        const connection = new signalR.HubConnectionBuilder()
          .withUrl('http://localhost:7071/api')
          .configureLogging(signalR.LogLevel.Information)
          .build();

        connection.on('driverLocationUpdated', driverLocationUpdated);
        connection.onclose(() => console.log('disconnected'));

        console.log('connecting...');
        connection.start()
            .then(() => data.ready = true)
            .catch(console.error);

        let counter = 0;

        function driverLocationUpdated(driverLocationUpdatedMessage) {
        driverLocationUpdatedMessage.id = counter++; // vue transitions need an id
        data.drivers.unshift(driverLocationUpdatedMessage);
    }
    </script>
</body>

</html>

Now when changes are made to the documents in the driver collection, the Azure Function will execute and send out Azure SignalR Service messages to clients. In this example we’re just writing out the messages in the page, but you can image the latitude and longitude being used to draw a little car image on top of a map, etc.

SHARE:

Writing Azure Cosmos DB Data from Azure Functions

This is the third part in a series of articles.

In part 2 of this series we saw how to read data from Cosmos DB, in this article we’ll see how to write data to Cosmos DB from an Azure Function.

As with input, output is achieved by the use of the [CosmosDB] binding attribute and as before, the database name, collection name, id, partition, and connection string can be defined.

In part 2 we saw how to use binding expressions to take data from the HTTP querystring and use that to select a document to read in (e.g. by Id). In this article we’ll see how we can do a similar thing when the trigger type is for a storage queue.

The following function is triggered when a message is written to the “update-pizza-driver-location” queue:

[FunctionName("UpdatePizzaDriverLocation")]
[return: CosmosDB(databaseName: "pizza", collectionName: "driver", ConnectionStringSetting = "pizzaConnection", Id = "{Id}", PartitionKey = "{StoreId}")]
public static Driver Run(
    [QueueTrigger("update-pizza-driver-location")] PizzaDriverLocationUpdate locationUpdate,
    [CosmosDB(databaseName: "pizza", collectionName: "driver", ConnectionStringSetting = "pizzaConnection", Id = "{Id}", PartitionKey = "{StoreId}")] Driver driver,            
    ILogger log)
{
    if (driver is null)
    {
        log.LogError($"Driver Id/partition {locationUpdate.Id}/{locationUpdate.StoreId} not found in database.");
        return null;
    }

    driver.Latitude = locationUpdate.NewLat;
    driver.Longitude = locationUpdate.NewLong;

    return driver;
}

The first thing to notice in the preceding function that there are 2 instances of the [CosmosDB] binding attribute, one as the return binding for the method (the output binding to perform the update) and the input binding in the Run method parameter (the input binding to read the current state of the Driver).

Another thing to notice is the use of the {Id} and {StoreId} binding expressions. These expressions assume that the incoming queue message contains JSON properties that match these expressions, for example:

{
    "Id" : "1",
    "StoreId" : "101",
    "NewLat" : 111.2,
    "NewLong" : 3110.3
}

The Driver and PizzaDriverLocationUpdate classes look like the following:

public class PizzaDriverLocationUpdate
{
    public string Id { get; set; }
    public string StoreId { get; set; }
    public double NewLat { get; set; }
    public double NewLong { get; set; }
}

 

public class Driver
{
    [JsonProperty(PropertyName = "id")]
    public string Id { get; set; }
    public string StoreId { get; set; }
    public string Name { get; set; }
    public double Latitude { get; set; }
    public double Longitude { get; set; }
}

Note in the Driver class, the [JsonProperty(PropertyName = "id")] attribute is mapping Id to id in Cosmos DB.

If you want to write multiple documents from a single function invocation you make use of an IAsyncCollector, for example IAsyncCollector<Driver> drivers and then use drivers.AddAsync(newDriver) method to write multiple documents.

In the next part of this series we’ll see how we can combine Azure Cosmos DB triggers with SignalR to create notifications to clients when data changes.

SHARE:

Reading Azure Cosmos DB Data In Azure Functions

This is the second part in a series of articles.

In addition to triggering a function when Cosmos DB data changes (as we saw in part one) you can also read data in from Cosmos DB when a function executes. The simplest way to do this is to use an input binding.

The [CosmosDB] binding attribute can be used both as an input binding and an output binding. When used as an input binding it allows one or more documents to be retrieved from the database.

When using the attribute there are a number of ways to configure it including:

  • The Cosmos DB database name
  • The collection name
  • The document Id to retrieve
  • The partition key
  • And the connection string app setting

Reading a Single Cosmos DB Document in an Azure Function

The following code shows an Azure Function that is triggered from an HTTP GET request:

[FunctionName("GetDriver")]
public static async Task<IActionResult> Run(
    [HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest req,
    [CosmosDB(databaseName: "pizza",collectionName: "driver", Id = "{Query.id}", PartitionKey = "{Query.storeId}", ConnectionStringSetting = "pizzaConnection")] Driver driver,
    ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    if (driver is null)
    {
        return new NotFoundResult();
    }

    return new OkObjectResult(driver);
}

Notice first the [CosmosDB] binding attribute. The combination of the Id and PartitionKey will determine the Driver object (if any) that will be retrieved. Note the format of these two: {Query.id} and {Query.storeId}.This will look for query string parameters in the incoming HTTP GET called id and storeId, for example: http://localhost:7071/api/GetDriver?id=1&storeId=101

If no document is found, driver will be null.

If the document was found it will be returned as JSON to the caller.

Reading Multiple Cosmos DB Documents in an Azure Function Using SqlQuery

The following function will get the latest 100 drivers (as sorted by the built in timestamp _ts property):

[FunctionName("GetDrivers")]
public static async Task<IActionResult> GetDrivers(
    [HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest req,
    [CosmosDB(databaseName: "pizza", collectionName: "driver", SqlQuery = "SELECT top 100 * FROM driver order by driver._ts desc", ConnectionStringSetting = "pizzaConnection")] IEnumerable<Driver> drivers,
    ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    if (drivers is null)
    {
        return new NotFoundResult();
    }

    foreach (var driver in drivers)
    {
        log.LogInformation(driver.Name);
    }

    return new OkObjectResult(drivers);
}

There’s a couple of things to notice in the preceding code. The first is that instead of a single Driver, the binding now returns IEnumerable<Driver> drivers. Also notice that the binding no longer has Id and PartitionKey.

Reading Multiple Cosmos DB Documents Based on Query String Parameter

A more advanced technique is to bind to an instance of DocumentClient. This allows more fine grained/low level/more specific access of data, such as using a LINQ query to perform the search:

[FunctionName("GetDriversForStore")]
public static async Task<IActionResult> GetDriversForStore(
    [HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest req,
    [CosmosDB( ConnectionStringSetting = "pizzaConnection")] DocumentClient client,
    ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");
    
    // Validation and error checking omitted for demo purposes
               
    string storeId = req.Query["storeId"]; // read storeId to get driver for from querystring

    Uri driverCollectionUri = UriFactory.CreateDocumentCollectionUri(databaseId: "pizza", collectionId: "driver");

    var options = new FeedOptions { EnableCrossPartitionQuery = true }; // Enable cross partition query

    IDocumentQuery<Driver> query = client.CreateDocumentQuery<Driver>(driverCollectionUri, options)
                                         .Where(driver => driver.StoreId == storeId)
                                         .AsDocumentQuery();

    var driversForStore = new List<Driver>();

    while (query.HasMoreResults)
    {
        foreach (Driver driver in await query.ExecuteNextAsync())
        {
            driversForStore.Add(driver);
        }
    }                       

    return new OkObjectResult(driversForStore);
}

The preceding code returns all drivers that belong to a specific store passed in as a querystring storeId parameter.

In the next part of this series we’ll see how to write data out to Cosmos DB when a function executes.

SHARE:

Getting Started with Azure Cosmos DB and Azure Functions

This is the first part in a series of articles.

Azure Cosmos DB is a “globally distributed, multi-model database service. With a click of a button, Cosmos DB enables you to elastically and independently scale throughput and storage across any number of Azure regions worldwide. You can elastically scale throughput and storage, and take advantage of fast, single-digit-millisecond data access using your favorite API including SQL, MongoDB, Cassandra, Tables, or Gremlin” [Microsoft]

One way to respond to changes in Cosmos is to use Azure Functions. When changes occur (currently limited to inserts and updates, e.g. not deletions ) the Function can be notified and executes.

Azure Function Cosmos DB triggers under the hood make use of the Azure Cosmos DB change feed to know when to execute functions.

Installing the Azure Cosmos Emulator

You can use the Azure Cosmos Emulator to get started locally without even needing an Azure subscription.

Once the emulator is downloaded and installed on your development PC (a Docker version is also available) you can start the  emulator from the start menu.

When running you can see the icon in the Windows taskbar notification area and it will pop up a browser window pointing to https://localhost:8081/_explorer/index.html as the following screenshot shows:

Azure Cosmos Emulator portal

Notice in the preceding screenshot the Primary Connection String, you will need to copy this for use in the local app settings later.

Creating an Azure Function Triggered from Cosmos DB

Once you have created an Azure Function project in Visual Studio, you need to add the Microsoft.Azure.WebJobs.Extensions.CosmosDB NuGet package to get access to the .NET C# bindings.

Now the package is installed (and with an up to date installation of Visual Studio) you can right click the Azure Functions project and choose Add –> New Azure Function…

Give the function a name and for the trigger type choose Cosmos - this will create a boiler plate function like the following:

public static class PizzaDriverLocationUpdated
{
    [FunctionName("PizzaDriverLocationUpdated")]
    public static void Run([CosmosDBTrigger(
        databaseName: "databaseName",
        collectionName: "collectionName",
        ConnectionStringSetting = "",
        LeaseCollectionName = "leases")]IReadOnlyList<Document> input, ILogger log)
    {
        if (input != null && input.Count > 0)
        {
            log.LogInformation("Documents modified " + input.Count);
            log.LogInformation("First document Id " + input[0].Id);
        }
    }
}

The preceding code uses the [CosmosDBTrigger] attribute to tell this function to execute when there are changes as specified by the following attribute parameters/properties:

  • databaseName - Azure Cosmos DB database with the monitored collection
  • collectionName – Collection being monitored for changes
  • ConnectionStringSetting – App setting name containing the connection string to the Azure Cosmos DB being monitored
  • LeaseCollectionName - name of the collection used to store leases, defaults to “leases” if not specified

The LeaseCollectionName is required by the trigger to store leases over Cosmos DB partitions, one thing to note: “If multiple functions are configured to use a Cosmos DB trigger for the same collection, each of the functions should use a dedicated lease collection or specify a different LeaseCollectionPrefix for each function. Otherwise, only one of the functions will be triggered” [Microsoft]

How to Create a Database and Collection in the Azure Cosmos Emulator

In the emulator portal in the browser, click Explorer.This will allow you to create collections in the emulator.

Click the New Collection button and enter the following:

  • Database id: pizza
  • Collection Id: driverLocation
  • Partition key: /storeId

Creating a new collection in the Azure Cosmos DB Emulator

Save this new collection.

Configuring an Azure Function Cosmos DB Trigger

Modify the function that was created earlier to be as follows:

public static class PizzaDriverLocationUpdated
{
    [FunctionName("PizzaDriverLocationUpdated")]
    public static void Run([CosmosDBTrigger(
        databaseName: "pizza",
        collectionName: "driverLocation",
        ConnectionStringSetting = "pizzaConnection")] IReadOnlyList<Document> input,
        ILogger log)
    {
        if (input != null && input.Count > 0)
        {
            log.LogInformation("Documents modified " + input.Count);
            log.LogInformation("First document Id " + input[0].Id);
        }
    }
}

Notice in the preceding code that the databaseName and collectionName settings match what we just created in the emulator.

The ConnectionStringSetting is set to “pizzaConnection” – this needs to be in the function settings, in the case of local development in the local.settings.json file:

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet",
    "pizzaConnection": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="
  },
  "Host": {
    "LocalHttpPort": 7071,
    "CORS": "http://localhost:3872",
    "CORSCredentials": true
  }
}

Notice in the preceding settings, the value for the pizzaConnection item is the Primary Connection String copied from the Azure Cosmos Emulator.

Testing the Function Locally

Now that the function code is configured and the database and collection exist in the emulator, hit F5 in Visual Studio (or click Run) and the local functions runtime will start.

Once the runtime has started, head back to the Cosmos Emulator in the browser, click on Pizza –> driverLocation –> Documents and click the New Document button.

Creating a new document in the Cosmos DB emulator

Add the following:

{
    "id": "1",
    "storeId": 42,
    "name": "Sarah",
    "lat" : 52,
    "long": 2
}

And click Save.

Head back to the locally running functions runtime window and you will see the function has noticed this new document and executed:

Executing 'PizzaDriverLocationUpdated' (Reason='New changes on collection driverLocation at 2019-05-10T04:10:29.9913171Z', Id=203ae791-c7d4-4270-ac6b-501f313c3805)
Documents modified 1
First document Id 1
Executed 'PizzaDriverLocationUpdated' (Succeeded, Id=203ae791-c7d4-4270-ac6b-501f313c3805)

If you head back to the emulator, modify the document (e.g. change the name to “Amrit”) and click Update, the function will trigger a second time:

Executing 'PizzaDriverLocationUpdated' (Reason='New changes on collection driverLocation at 2019-05-10T04:13:00.8233214Z', Id=778caddd-9f19-42fa-9d9d-6c5dd5892a25)
Documents modified 1
First document Id 1
Executed 'PizzaDriverLocationUpdated' (Succeeded, Id=778caddd-9f19-42fa-9d9d-6c5dd5892a25)

SHARE: