If an Azure Function that is triggered by a message on a Storage Queue throws an exception, the message will automatically be returned to the queue and retried again in the future.

In addition to specifying how soon the message will be retried, you can also configure how many times the message will be retried by editing the host.json file. By default a message will be retried 5 times before finally failing. The following host.json specifies that a message should be retried 10 times before finally failing:

{
  "version": "2.0",
  "extensions": {
    "queues": {      
      "maxDequeueCount": 10
    } 
  }  
}

Handling Poison Messages in Azure Functions

Once a message has been retried the maximum number of times, it will be considered a poisonous message, essentially that if we keep it on the queue it will “poison” the application/function and cause harm. Poison messages will be removed from the queue and placed onto a poison queue.

For example, if the queue that triggers the function is called “input-queue”, poison messages will be moved to a queue called “input-queue-poison”.

Because we know the name of the poison queue, we can process these poison messages somehow. Exactly how you choose to process these messages will depend on the application you are building.

One thing to think about is why the message may have failed:

• Is the message content itself corrupted?
• Is the function code itself defective/have a bug?
• Were the exceptions caused by a transient error in a service the function uses?
• Etc.

You could have some automated process (function) attempt to resolve the poison messages or forward them to a human to resolve (for example writing the message to database that a human can query).

Triggering an Azure Function From a Poison Message Queue

As an example, the following function retrieves messages from the “input-queue-poison” queue and writes out to table storage for a human to manually correct somehow:

using System;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage.Queue;

namespace DontCodeTiredDemosV2
{
    public class PoisonMessageDetails
    {        
        public string RowKey { get; set; }
        public string PartitionKey { get; set; }
        public string OriginalMessageContent { get; set; }
        public string OriginalMessageId { get; set; }
    }

    public static class HandlePoisonMessages
    {
        [FunctionName("HandlePoisonMessages")]
        [return: Table("HumanInterventionRequired")]
        public static PoisonMessageDetails Run(
            [QueueTrigger("input-queue-poison")]CloudQueueMessage poisonMessage,            
            ILogger log)
        {
            log.LogInformation($"Processing poison message {poisonMessage.Id}");            

            return new PoisonMessageDetails
            {
                RowKey = Guid.NewGuid().ToString(),
                PartitionKey = "input-queue",
                OriginalMessageContent = poisonMessage.AsString,
                OriginalMessageId = poisonMessage.Id
            };                       
        }
    }
}

Once a poison message is processed (for example with the content of “Amrit”) a row will be added to the table as the following screenshot shows:

If you want to fill in the gaps in your C# knowledge be sure to check out my C# Tips and Traps training course from Pluralsight – get started with a free trial.

SHARE: