Tuesday, 3 April 2018

Azure ServiceBus Queue - DeadLetterQueue (DLQ)

Azure Service Bus queues and topic subscriptions provide a secondary sub-queue, called a dead-letter queue (DLQ). The dead-letter queue does not need to be explicitly created and cannot be deleted or otherwise managed independent of the main entity.

The dead-letter queue

The purpose of the dead-letter queue is to hold messages that cannot be delivered to any receiver, or messages that could not be processed. Messages can then be removed from the DLQ and inspected. An application might, with help of an operator, correct issues and resubmit the message, log the fact that there was an error, and take corrective action.
From an API and protocol perspective, the DLQ is mostly similar to any other queue, except that messages can only be submitted via the dead-letter operation of the parent entity. In addition, time-to-live is not observed, and you can't dead-letter a message from a DLQ. The dead-letter queue fully supports peek-lock delivery and transactional operations.
Note that there is no automatic cleanup of the DLQ. Messages remain in the DLQ until you explicitly retrieve them from the DLQ and call Complete() on the dead-letter message.

Moving messages to the DLQ

There are several activities in Service Bus that cause messages to get pushed to the DLQ from within the messaging engine itself. An application can also explicitly move messages to the DLQ.
As the message gets moved by the broker, two properties are added to the message as the broker calls its internal version of the DeadLetter method on the message: DeadLetterReason and DeadLetterErrorDescription.
Applications can define their own codes for the DeadLetterReason property, but the system sets the following values.
ConditionDeadLetterReasonDeadLetterErrorDescription
AlwaysHeaderSizeExceededThe size quota for this stream has been exceeded.
!TopicDescription.
EnableFilteringMessagesBeforePublishing and SubscriptionDescription.
EnableDeadLetteringOnFilterEvaluationExceptions
exception.GetType().Nameexception.Message
EnableDeadLetteringOnMessageExpirationTTLExpiredExceptionThe message expired and was dead lettered.
SubscriptionDescription.RequiresSessionSession id is null.Session enabled entity doesn't allow a message whose session identifier is null.
!dead letter queueMaxTransferHopCountExceededNull
Application explicit dead letteringSpecified by applicationSpecified by application

Exceeding MaxDeliveryCount

Queues and subscriptions each have a QueueDescription.MaxDeliveryCount and SubscriptionDescription.MaxDeliveryCount property respectively; the default value is 10. Whenever a message has been delivered under a lock (ReceiveMode.PeekLock), but has been either explicitly abandoned or the lock has expired, the message BrokeredMessage.DeliveryCount is incremented. When DeliveryCount exceeds MaxDeliveryCount, the message is moved to the DLQ, specifying the MaxDeliveryCountExceeded reason code.
This behavior cannot be disabled, but you can set MaxDeliveryCount to a very large number.

Exceeding TimeToLive

When the QueueDescription.EnableDeadLetteringOnMessageExpiration or SubscriptionDescription.EnableDeadLetteringOnMessageExpiration property is set to true (the default is false), all expiring messages are moved to the DLQ, specifying the TTLExpiredException reason code.
Note that expired messages are only purged and moved to the DLQ when there is at least one active receiver pulling from the main queue or subscription; that behavior is by design.

Errors while processing subscription rules

When the SubscriptionDescription.EnableDeadLetteringOnFilterEvaluationExceptions property is enabled for a subscription, any errors that occur while a subscription's SQL filter rule executes are captured in the DLQ along with the offending message.

Application-level dead-lettering

In addition to the system-provided dead-lettering features, applications can use the DLQ to explicitly reject unacceptable messages. This can include messages that cannot be properly processed due to any sort of system issue, messages that hold malformed payloads, or messages that fail authentication when some message-level security scheme is used.

Dead-lettering in ForwardTo or SendVia scenarios

Messages will be sent to the transfer dead-letter queue under the following conditions:
  • A message passes through more than 3 queues or topics that are chained together.
  • The destination queue or topic is disabled or deleted.
  • The destination queue or topic exceeds the maximum entity size.
To retrieve these dead-lettered messages, you can create a receiver using the FormatTransferDeadletterPath utility method.

Example

The following code snippet creates a message receiver. In the receive loop for the main queue, the code retrieves the message with Receive(TimeSpan.Zero), which asks the broker to instantly return any message readily available, or to return with no result. If the code receives a message, it immediately abandons it, which increments the DeliveryCount. Once the system moves the message to the DLQ, the main queue is empty and the loop exits, as ReceiveAsync returns null.
C#
var receiver = await receiverFactory.CreateMessageReceiverAsync(queueName, ReceiveMode.PeekLock);
while(true)
{
    var msg = await receiver.ReceiveAsync(TimeSpan.Zero);
    if (msg != null)
    {
        Console.WriteLine("Picked up message; DeliveryCount {0}", msg.DeliveryCount);
        await msg.AbandonAsync();
    }
    else
    {
        break;
    }
}

Clearing the Dead Letter Queue  

[TestMethod]
public void ClearDeadLetterQueue()
{
    string deadLetterQueueName = "myQueue/$DeadLetterQueue";
    QueueClient client = QueueClient.CreateFromConnectionString("connectionString",
            deadLetterQueueName, ReceiveMode.PeekLock);
     while (client.Receive() != null)
    {
        var receivedMessage = client.Receive();
//do something with the message here
        receivedMessage?.Complete();
    }
}

sources: https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dead-letter-queues

https://medium.com/@DomBurf/clearing-the-dead-letter-queue-on-an-azure-service-bus-queue-3c942b312f98