This is the sixth part in a series of articles.
Sometimes you may want more than one Azure Function to execute when a document is changed or inserted in Cosmos DB.
You could just use one function that performs multiple logical operations on the changed document but there are some things to consider when doing this:
- What if the function throws an exception during the first logical operation? (operation 2 may not be executed).
- What scaling do you want, you/Azure won’t be able to scale the 2 logical operations independently.
- How long will the function execute for if performing multiple operations in a single function, will you risk function timeouts?
- How will you monitor operations when they are all contained in a single function.
- How will you update the code/fix bugs: you will have to update the entire function even if the bug is only related to one operation.
- How will you write automated tests? They will be more complex if there are multiple operations in a single function.
In some cases you may decide the preceding points don’t matter, but if they do you will need to split the operations into multiple separate Azure Functions.
As an example, the following function contains two logical operations in a single function:
[FunctionName("PizzaDriverLocationUpdated")]
public static void RunMultipleOperations([CosmosDBTrigger(
databaseName: "pizza",
collectionName: "driver",
ConnectionStringSetting = "pizzaConnection")] IReadOnlyList<Document> modifiedDrivers,
ILogger log)
{
if (modifiedDrivers != null)
{
foreach (var modifiedDriver in modifiedDrivers)
{
var driverName = modifiedDriver.GetPropertyValue<string>("Name");
// Simulate running logical operation 1
log.LogInformation($"Running operation 1 for driver {modifiedDriver.Id} {driverName}");
// Simulate running logical operation 2
log.LogInformation($"Running operation 2 for driver {modifiedDriver.Id} {driverName}");
}
}
}
The preceding function could be separated into two separate functions, each one containing only a single logical operation:
[FunctionName("PizzaDriverLocationUpdated1")]
public static void RunOperation1([CosmosDBTrigger(
databaseName: "pizza",
collectionName: "driver",
ConnectionStringSetting = "pizzaConnection")] IReadOnlyList<Document> modifiedDrivers,
ILogger log)
{
if (modifiedDrivers != null)
{
foreach (var modifiedDriver in modifiedDrivers)
{
var driverName = modifiedDriver.GetPropertyValue<string>("Name");
log.LogInformation($"Running operation 1 for driver {modifiedDriver.Id} {driverName}");
}
}
}
[FunctionName("PizzaDriverLocationUpdated2")]
public static void RunOperation2([CosmosDBTrigger(
databaseName: "pizza",
collectionName: "driver",
ConnectionStringSetting = "pizzaConnection")] IReadOnlyList<Document> modifiedDrivers,
ILogger log)
{
if (modifiedDrivers != null)
{
foreach (var modifiedDriver in modifiedDrivers)
{
var driverName = modifiedDriver.GetPropertyValue<string>("Name");
log.LogInformation($"Running operation 2 for driver {modifiedDriver.Id} {driverName}");
}
}
}
If you try and run the function app (for example in the local development functions runtime) you will see some error such as the following:
Unhealthiness detected in the operation AcquireLease for localhost_...==_...=..1 Owner='626d5aec...' Continuation="49" Timestamp(local)=...
Unhealthiness detected in the operation AcquireLease for localhost_...==_...=..0 Owner='626d5aec... Continuation="586" Timestamp(local)=...
If you then make updates/inserts you may see that only one of the two functions is executed, rather than both of them. This is due to change feed leases.
Understanding Azure Cosmos DB Change Feed Leases
The Azure Functions Cosmos DB trigger knows when documents are changed/insert ed by way of the Cosmos DB change feed.
The change feed at a simple level listens for changes made in a collection and allows these changes to be passed to other processes (such as Azure Functions) to do work on.
Without a way to keep track of what changes in the underlying collection have been “fed” out to other process(es) there would be no way to know what changed documents have been passed to external process(es). This is where the lease collection comes in.
The lease collection stores a “checkpoint” for an Azure Function that is using the Cosmos DB trigger. Without this checkpoint, the function would not know if it has processed changed documents or not.
When only one function exists for Cosmos DB collection there is not a problem as only one checkpoint needs to be stored, because there is only one function.
When more that one function exists, there needs to be a way to store different checkpoints for different functions.
One way to do this is to use lease prefixes.
Sharing a Single Lease Collection Across Multiple Azure Functions
To use a single lease collection when you have multiple Azure Functions, you can use the LeaseCollectionPrefix property of the [CosmosDBTrigger] attribute. The value for this property needs to be unique for every function, as the following code demonstrates:
[FunctionName("PizzaDriverLocationUpdated1")]
public static void RunOperation1([CosmosDBTrigger(
databaseName: "pizza",
collectionName: "driver",
LeaseCollectionPrefix = "PizzaDriverLocationUpdated1",
ConnectionStringSetting = "pizzaConnection")] IReadOnlyList<Document> modifiedDrivers,
ILogger log)
{
if (modifiedDrivers != null)
{
foreach (var modifiedDriver in modifiedDrivers)
{
var driverName = modifiedDriver.GetPropertyValue<string>("Name");
log.LogInformation($"Running operation 1 for driver {modifiedDriver.Id} {driverName}");
}
}
}
[FunctionName("PizzaDriverLocationUpdated2")]
public static void RunOperation2([CosmosDBTrigger(
databaseName: "pizza",
collectionName: "driver",
LeaseCollectionPrefix = "PizzaDriverLocationUpdated2",
ConnectionStringSetting = "pizzaConnection")] IReadOnlyList<Document> modifiedDrivers,
ILogger log)
{
if (modifiedDrivers != null)
{
foreach (var modifiedDriver in modifiedDrivers)
{
var driverName = modifiedDriver.GetPropertyValue<string>("Name");
log.LogInformation($"Running operation 2 for driver {modifiedDriver.Id} {driverName}");
}
}
}
In the preceding code, notice LeaseCollectionPrefix = "PizzaDriverLocationUpdated1", and LeaseCollectionPrefix = "PizzaDriverLocationUpdated2".
If the function app is run now there is no startup error and changes made to a document trigger both functions:
Executing 'PizzaDriverLocationUpdated2' (Reason='New changes on collection driver at 2019-05-31T02:49:55.6946671Z', Id=b1476848-7f98-4362-a25f-69beb714c379)
Executing 'PizzaDriverLocationUpdated1' (Reason='New changes on collection driver at 2019-05-31T02:49:55.6946679Z', Id=366fa257-3b94-4d41-94f2-2777e0b8249a)
Running operation 2 for driver 1 Amrit
Running operation 1 for driver 1 Amrit
Executed 'PizzaDriverLocationUpdated2' (Succeeded, Id=b1476848-7f98-4362-a25f-69beb714c379)
Executed 'PizzaDriverLocationUpdated1' (Succeeded, Id=366fa257-3b94-4d41-94f2-2777e0b8249a)
If you check the lease collection behind the scenes notice the lease prefixes in use as the following screenshot shows:
Using Multiple Azure Cosmos DB Lease Collections with Azure Functions
Rather than sharing a single lease collection, you can instead specify completely separate collections with the LeaseCollectionName property:
[FunctionName("PizzaDriverLocationUpdated1")]
public static void RunOperation1([CosmosDBTrigger(
databaseName: "pizza",
collectionName: "driver",
LeaseCollectionName = "PizzaDriverLocationUpdated1",
CreateLeaseCollectionIfNotExists = true,
ConnectionStringSetting = "pizzaConnection")] IReadOnlyList<Document> modifiedDrivers,
ILogger log)
{
if (modifiedDrivers != null)
{
foreach (var modifiedDriver in modifiedDrivers)
{
var driverName = modifiedDriver.GetPropertyValue<string>("Name");
log.LogInformation($"Running operation 1 for driver {modifiedDriver.Id} {driverName}");
}
}
}
[FunctionName("PizzaDriverLocationUpdated2")]
public static void RunOperation2([CosmosDBTrigger(
databaseName: "pizza",
collectionName: "driver",
LeaseCollectionName = "PizzaDriverLocationUpdated2",
CreateLeaseCollectionIfNotExists = true,
ConnectionStringSetting = "pizzaConnection")] IReadOnlyList<Document> modifiedDrivers,
ILogger log)
{
if (modifiedDrivers != null)
{
foreach (var modifiedDriver in modifiedDrivers)
{
var driverName = modifiedDriver.GetPropertyValue<string>("Name");
log.LogInformation($"Running operation 2 for driver {modifiedDriver.Id} {driverName}");
}
}
}
Notice in the preceding code LeaseCollectionName = "PizzaDriverLocationUpdated1", and LeaseCollectionName = "PizzaDriverLocationUpdated2". Also notice the CreateLeaseCollectionIfNotExists = true, as its name suggests this will create the lease collections if they don’t already exist.
Running the function app and once again and changing a document will result in both functions executing.
Because there are now two separate collections being used for leases there will be a cost associated with having these two collections. Also be sure to read up on RUs for your lease collections, especially if sharing a lease collection and using lease prefixes you should keep an eye on the metrics and make sure you are not getting throttled requests on your lease collection(s).
If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.
SHARE: