Transactional remote receive in MSMQ

Can you implement publish-subscribe load balancing with MSMQ?

Surprisingly, prior to Windows Server 2008 the answer was no! (Well, technically the capability is in Vista too, but who runs Vista in their datacenter?)

It's a common scenario. I have work that I want to share among several load-balanced, redundant servers. Only one server can be working on an item at a time. If a server crashes while working on an item, then another server should pick it up instead.

What's the solution? Publish the work items to a queue and have each of the servers subscribe. The servers receive the work items transactionally, so that if the work fails or the server crashes it is aborted back to the queue. But while one server has the work item locked, other servers will not receive it. They will get work items further down in the queue.

The problem is that MSMQ 3.0 and prior cannot receive messages transactionally from a remote queue. This feature only works with local queues. Instead of a publish-subscribe model, you have to use a push model. Push the message to the machine that you want to process it.

John Breakwell posted your options. You can upgrade to Windows Server 2008, implement a dispatcher (turning your pub-sub into a push), or peek-then-receive. The last is not really an option, because it violates the requirement that only one server is working on an item at a time.

Here's my solution
I propose a fourth option. Don't use MSMQ. Instead, write a journal table in SQL Server. Getting the locking right is a bit tricky, but it's doable. (Hint: you will make use of UPDLOCK.)

Honestly, this is a primary use case for a queuing technology. No, this is the primary use case. How could you even release version 1.0 without support for pub-sub? I've gone my entire career assuming that this feature was there, only to find out in integration testing that it is lacking.

Leave a Reply

You must be logged in to post a comment.