NET
MSMQ is ideal for use in a variety of systems as a means of transferring data, but if
you are going to use it as part of your system, it is critical that you understand how
to make it reliable. If you use MSMQ to send a customer's order from your front-end
Web application to your back-end order processing system, you want to make sure
the order does not get lost along the way. MSMQ can be configured and used so that
it becomes a very reliable system, but increasing reliability comes at the expense of
decreased performance, so you have to determine the best mix for your specific
application.
Recoverable Messages
By default, MSMQ stores some messages in memory for increased performance, and
a message may be sent and received from a queue without ever having been written
to disk. This is not normally a problem, and it is completely transparent to you as the
developer, but if the MSMQ service were to unexpectedly terminate due to a
hardware or software failure, or due to some external circumstance such as a power
failure, messages stored only in memory will be lost forever. You can control this
behavior, forcing MSMQ to store a message to disk, by specifying that a message
should be recoverable. A message can be marked as recoverable by setting the
Recoverable property of a Message object before sending, or if you are not using
Message objects and are just sending objects directly the simple method of sending
messages) you can control this behavior by setting the DefaultPropertiesToSend
property on the MessageQueue object. In the code snippet to follow, a Message object
is created, marked as recoverable, and then sent to the local private queue named
"Orders."
recoverableMessage.Recoverable = True
msgQ.Send(recoverableMessage)
Visual C# .NET
recoverableMessage.Recoverable = true;
msgQ.Send(recoverableMessage);
The next code snippet shows how to configure the default properties information of
the MessageQueue so that all messages sent without using an actual Message object
will be marked as Recoverable.
Visual Basic .NET
msgQ.DefaultPropertiesToSend.Recoverable = True
Visual C# .NET
msgQ.DefaultPropertiesToSend.Recoverable = true;
msgQ.Close();
Transactions
The concept of transactions is part of a variety of different computing technologies,
but the general meaning is always the same: to execute a multiple-step process such
that either all or none of the steps will complete. In reality, transactions are handled
by rolling back any steps that have already occurred if the entire transaction is not
completed successfully. Regardless of the technology used to implement
transactions, they are described using several basic concepts:
If you are creating a queue programmatically and you intend on using that queue in
transactions, you must specify this when you call the MessageQueue.Create method.
The code snippet to follow creates a new transactional queue by supplying an
additional parameter to the Create method.
MessageQueue.Create(".\private$\Orders", True)
Visual C# .NET
MessageQueue.Create(@".\private$\Orders", True);
You can determine if a queue supports transactions after it has been created by
viewing the queue properties through the MSMQ management interfaces, or by
checking the Transactional property of the MessageQueue object programmatically. The
next code snippet illustrates how to create a new instance of a MessageQueue object
and check if it supports transactions.
If msgQ.Transactional Then
Else
End If
Visual C# .NET
if (msgQ.Transactional)
else
Attempting to use transactions with a non-transactional queue will fail and result in
an exception of type MessageQueueException. The reverse, sending or receiving
messages from a transactional queue without using transactions, will also fail. If you
are using a transactional queue, and you are just sending a single message, you
must use a special overload of the MessageQueue.Send method to indicate that each
individual send or receive should be wrapped in its own single transaction. This is
accomplished by passing the value MessageQueueTransactionType.Single as the second
parameter to the Send method.
MSMQ provides built-in support for transactions, allowing you to execute multiple
actions against one or more queues (sending or receiving messages) all wrapped in a
transaction so that you can guarantee that either all or none of the actions will take
effect. The transaction support provided within MSMQ is limited to MSMQ actions,
however, and you will often be accessing other resources, such as a database, that
you also wish to include in the same transaction. To allow for this, MSMQ also
supports external transactions, where the Microsoft Distributed Transaction
Coordinator (MS DTC), used by COM+ and SQL Server, is used to allow a variety of
resources including databases and MSMQ to be included in a single transaction.
To perform any MSMQ action(s) within an internal transaction, you need to create a
MessageQueueTransaction object, begin the transaction, and then supply that object
along with each of the actions you wish to include in the transaction.
msgTx.Begin()
Visual C# .NET
msgTx.Begin();
Once you have this MessageQueueTransaction object, the Send and Receive methods of
the MessageQueue object both have an overloaded version that can accept a
transaction object along with the other required information. As noted earlier, before
you can send any messages as part of this new transaction, you must explicitly mark
the beginning of the transaction by using the MessageQueueTransaction.Begin method.
More details on controlling the transaction are provided in the next section, but note
that your actions (send and receive) won't permanently affect the queue unless you
commit the transaction, so a call to the MessageQueueTransaction.Commit method is
required after the send or else the receive may not find any messages to retrieve.
The two pieces of code to follow show how to both send and receive messages within
a transaction.
msgTx.Begin()
msgTx.Commit()
msgTx.Begin()
msg = msgQ.Receive(msgTx)
msgTx.Commit()
Visual C# .NET
msgTx.Begin();
msgTx.Commit();
msgTx.Begin();
Message msg;
msg = msgQ.Receive(msgTx);
msgTx.Commit();
Controlling the transaction
msgTx.Begin()
Try
msgTx.Commit()
Catch
msgTx.Abort()
Finally
msgQ.Close()
End Try
Visual C# .NET
msgTx.Begin();
try
{
msgQ.Send("First Message",msgTx);
msgQ.Send("Second Message",msgTx);
msgTx.Commit();
catch
msgTx.Abort();
finally
msgQ.Close();
• Initialized. The transaction object has been created, but no actions have
occurred yet, so the transaction itself has not been started.
• Pending. Actions have occurred, but the transaction is still in progress—it has
not committed or aborted.
• Committed. The transaction has completed successfully.
• Aborted. The transaction was aborted, and has been rolled back.
The Status property is an enumeration representing the four possible values. The
code to follow shows how you could use it in your program.
Console.WriteLine("Transaction Aborted!")
End If
Visual C# .NET
if (msgTx.Status = MessageQueueTransactionStatus.Aborted)
Console.WriteLine("Transaction Aborted!");
.
A real-world example
There are many different reasons to use transactions when you are working with
MSMQ, but here is a specific example to assist you in understanding this technology.
Consider a situation where orders are being sent into your back-end system via
MSMQ. Your system retrieves messages off of a queue, processes the message, and
then sends it on to another queue. At the same time you fire another message off
into a log queue that you use to audit the movement of orders within your system.
Without transactions, you could retrieve the order (removing it from the incoming
queue) but never send it on to another queue due to some serious error. At this
point, you have lost the order completely, and that is certainly not acceptable in
most systems. A slightly less serious problem is that you could retrieve and process
the order successfully but an error prevents you from sending an audit message to
your log queue, leaving you with a log that incorrectly states that this order is still
waiting to be processed. Either problem can be avoided by wrapping the entire
activity, including the read from the incoming queue, the write to the outgoing
queue, and the write to the audit queue in a transaction.
Note DTC only supports resources that have implemented compatible interfaces that
manage the transaction for that specific resource. These interfaces are referred to as
"Resource Managers" (see the .NET Framework Developer's Guide documentation on
Distributed Transactions for more information on this topic) and are provided by
several resources at this time, including MSMQ, Microsoft SQL Server, Oracle,
Sybase, and others.
To use DTC transactions from your .NET system, you need to take advantage of the
services provided by MTS/COM+ by running your code within their environment.
Before .NET, this was accomplished by creating a COM component and registering
that component as part of a COM+ application (known as a package in MTS) and
setting the properties of that application so that COM+ would run it as part of a
transaction. Once your component was placed into COM+, and set up to require a
transaction, any access from that component to a DTC-compatible resource became
part of the same transaction and could be committed or rolled back as a group. In
.NET, you accomplish the same results by building a class that inherits from the
System.EnterpriseServices.ServicedComponent class and then setting this class up with
attributes indicating that it requires a transaction. Creating a class like this enables it
to run inside COM+. The process to create, register, and use serviced components is
covered in detail in the .NET Framework Developer's Guide documentation titled
"Writing Serviced Components." The code to follow shows a sample class configured
to run within COM+, with assembly attributes that set up the COM+ application
properties and a class attribute <Transaction(TransactionOption.Required)> that
indicates a transaction is required when this class is used.
Imports System.EnterpriseServices
<Assembly: ApplicationName("BDAdotNETAsync2")>
<Assembly: ApplicationActivation(ActivationOption.Server)>
Inherits ServicedComponent
End Sub
End Class
Visual C# .NET
using System.EnterpriseServices;
[assembly: ApplicationName("BDAdotNETAsync2_csharp")]
[assembly: ApplicationActivation(ActivationOption.Server)]
[Transaction(TransactionOption.Required)]
{
}
Try
insertLogEntry.ExecuteNonQuery()
ContextUtil.SetComplete()
Catch e As Exception
ContextUtil.SetAbort()
End Try
Visual C# .NET
try
insertAuthor.ExecuteNonQuery();
ContextUtil.SetComplete();
catch(exception e)
ContextUtil.SetAbort();
There are many more features of serviced components in general and the ContextUtil
class specifically, but they are outside the scope of this article. Check out the link on
"Writing Serviced Components" in the .NET Framework for more information on
creating and using COM+ services with .NET.
Within the COM+ component you would then execute whatever operations were
needed against the resources that will be participating in your transaction. This
might include sending and receiving MSMQ messages and/or accessing a database.
There is no special coding required to indicate that you are participating in a
transaction; just the fact that you are calling these resources from within a COM+
hosted component is sufficient to enlist them into the current transaction. You will
want to add code to control your transaction, however, wrapping your activities in a
try/catch/finally block and using the ContextUtil object to abort or commit the
transaction as required.
targetTypes(0) = GetType(Order)
Try
incomingOrder = _
orderIDParam.Value = incomingOrder.ID
receiveParam.Value = DateTime.Now
insertLog.Parameters.Add(orderIDParam)
insertLog.Parameters.Add(receiveParam)
sqlConn.Open()
insertLog.ExecuteNonQuery()
Catch e As Exception
ContextUtil.SetAbort()
Finally
sqlConn.Close()
End Try
Visual C# .NET
Order incomingOrder;
targetTypes[0] = typeof(Order);
try
orderIDParam.Value = incomingOrder.ID;
receiveParam.Value = DateTime.Now;
insertLog.Parameters.Add(orderIDParam);
insertLog.Parameters.Add(receiveParam);
sqlConn.Open();
insertLog.ExecuteNonQuery();
ContextUtil.SetComplete();
catch
ContextUtil.SetAbort();
finally
sqlConn.Close();
'....
End Sub
In the end, when you have built your component and installed it into COM+, clients
can create and call your component just like any other .NET class, and your
component will automatically run inside COM+ and act as part of a transaction.
Note The Microsoft .NET Framework Developer's Guide article on "Writing Serviced
Components" should assist you in building a serviced component, but remember that
your assembly must have a strong name and must be installed into the global
assembly cache (using the gacutil command-line tool) to work correctly as a COM+
application.
Using Acknowledgements
When you perform an action synchronously, like calling a method on an object, you
receive immediate feedback on the success of that action, but with asynchronous
processes, a mechanism needs to be provided that will give you the same level of
feedback. For this purpose, MSMQ provides acknowledgements, special MSMQ-generated
messages that are sent to inform the sending application that a message was
successfully delivered to and/or retrieved at the destination. In the examples
included in this article, messages are sent to a private queue that is located on the
local machine, so successful delivery doesn't require an acknowledgement message.
But this isn't always the case. In many situations, MSMQ is used in an offline
(disconnected) mode, where messages are destined for some remote server but
must be stored locally and then forwarded on to that remote server (or another
server along the way) when a connection is available. In such a situation, the
message has not been delivered when the call to MessageQueue.Send completes, so an
acknowledgement of delivery becomes useful.
MessageQueue.Create(".\private$\Ack", False)
End If
myMessage.Body = "Sample"
myMessage.AcknowledgeType = AcknowledgeTypes.FullReachQueue _
Or AcknowledgeTypes.FullReceive
msgQ.Send(myMessage, MessageQueueTransactionType.Single)
Visual C# .NET
myMessage.Body = "Sample";
myMessage.AcknowledgeType = AcknowledgeTypes.FullReachQueue
AcknowledgeTypes.FullReceive;
msgQ.Send(myMessage, MessageQueueTransactionType.Single);
With msgQ.DefaultPropertiesToSend
.AcknowledgeType = AcknowledgeTypes.FullReachQueue _
Or AcknowledgeTypes.FullReceive
End With
msgQ.Send("Sample", MessageQueueTransactionType.Single)
Visual C# .NET
if (!MessageQueue.Exists(@".\private$\Ack"))
MessageQueue.Create(@".\private$\Ack",false);
msgQ.DefaultPropertiesToSend.AdministrationQueue =
new MessageQueue(@".\private$\Ack");
msgQ.DefaultPropertiesToSend.AcknowledgeType =
AcknowledgeTypes.FullReachQueue
| AcknowledgeTypes.FullReceive;
msgQ.Send("Sample", MessageQueueTransactionType.Single);
Processing acknowledgements
For acknowledgements to be of much use, your application has to do more than just
request them—it also needs to receive them. Acknowledgement messages are a little
different than regular MSMQ messages; they consist only of header information with
no body, but receiving them works in the same manner as a regular message.
Because there is no body, it is the other properties of the Message object that are
important when you are receiving an acknowledgement message, but not every
property is automatically received by the MessageQueue.Receive() method. There is
another property of the MessageQueue object: MessageReadPropertyFilter, which is used
to control exactly which message properties are received. If you look at the
documentation in the .NET Framework Class Library for the MessagePropertyFilter
class, you will find a list of which properties are retrieved by default, but the best bet
is to explicitly tell the queue to receive the properties you need. The code snippet to
follow does just that, informing the queue that the CorrelationID property should be
received.
adminQ.MessageReadPropertyFilter.CorrelationId = True
'Send Message
With msgQ.DefaultPropertiesToSend
.AcknowledgeType = AcknowledgeTypes.FullReachQueue Or _
AcknowledgeTypes.FullReceive
End With
msgQ.Send("Sample", MessageQueueTransactionType.Single)
'Receive Message
adminQ.MessageReadPropertyFilter.CorrelationId = True
Do
Try
Case Acknowledgment.ReachQueue
msg.CorrelationId)
Case Acknowledgment.Receive
msg.CorrelationId)
End Select
End If
Catch e As Exception
Console.WriteLine(e.Message)
msg = Nothing
End Try
Visual C# .NET
//Send Message
msgQ.DefaultPropertiesToSend.AcknowledgeType =
AcknowledgeTypes.FullReachQueue |
AcknowledgeTypes.FullReceive;
msgQ.DefaultPropertiesToSend.AdministrationQueue =
new MessageQueue(@".\private$\Ack");
msgQ.Send("Sample");
//Receive Message
Message msg;
adminQ.MessageReadPropertyFilter.CorrelationId = True;
do
try
if (msg.MessageType = MessageType.Acknowledgment)
switch(msg.Acknowledgment)
case Acknowledgment.ReachQueue:
{0}",
msg.CorrelationId);
break;
case Acknowledgment.Receive:
msg.CorrelationId);
break;
}
catch(Exception e)
Console.WriteLine(e.Message);
msg = null;
With myMessage
.AcknowledgeType = AcknowledgeTypes.FullReachQueue Or _
AcknowledgeTypes.FullReceive
End With
msgQ.Send(myMessage, MessageQueueTransactionType.Single)
Visual C# .NET
myMessage.AcknowledgeType = AcknowledgeTypes.FullReachQueue
| AcknowledgeTypes.FullReceive;
The preceding code example will successfully send the message, but if the message
is not received off of the Orders queue within one hour (you may wish to change the
time span to a shorter value for easier testing, such as one minute) it will be
removed from the Orders queue and a new message will appear in the
Administration queue (Ack in this example). If you choose to use these time-out
features, you will need to have a process in place for dealing with the messages that
have timed out, such as resending at a higher priority or sending the message to a
person for direct intervention.
Journal queues have a similar purpose to the acknowledgement mechanism that was
discussed earlier They exist as a way to track the movement of messages in an
MSMQ system. Journal queues are system-provided queues that are automatically
created, but are not automatically used. Whenever a message is sent or received, it
is possible for a copy of that message to be stored into a journal queue as a form of
auditing. Each application queue has its own associated journal queue, to hold copies
of messages as they are received from the application queue, and each MSMQ
machine has a single journal queue that can be used to keep copies of each message
that is sent from that machine.
For the first case, keeping copies of messages received from a queue, the
UseJournalQueue property of the MessageQueue object must be set to true before the
receive actions occur.
msgQ.UseJournalQueue = True
myMessage = msgQ.Receive(MessageQueueTransactionType.Single)
Visual C# .NET
Message myMessage;
msgQ.Send("Sample", MessageQueueTransactionType.Single);
msgQ.UseJournalQueue = true;
myMessage = msgQ.Receive(MessageQueueTransactionType.Single);
For sending messages, the UseJournalQueue property of the Message object controls
this behavior—for those sends that use a Message object. You can also set the
DefaultPropertiesToSend.UseJournalQueue property to ensure that all sends that do not
use Message objects will place copies into the journal queue. Note that in the case of
sending messages, journaling only occurs if you are sending to a queue located on
another machine; for sends to local queues, journaling does not occur. For this
example to work, you will need to use two machines, one of which must be capable
of running public queues, meaning that it will need to be a Server OS (either
Windows NT 4.0 or Windows 2000). From the client machine, you can execute the
code shown next to send a message to a public queue on the server machine.
myMessage.UseJournalQueue = True
msgQ.Send(myMessage, MessageQueueTransactionType.Single)
msgQ.DefaultPropertiesToSend.UseJournalQueue = True
Visual C# .NET
myMessage.UseJournalQueue = true;
msgQ.Send(myMessage, MessageQueueTransactionType.Single);
msgQ.DefaultPropertiesToSend.UseJournalQueue = True;
MSMQ on its own will not remove items from journal queues, but you can write code
to receive messages from these queues just like any other queue. One possible use
for this feature would be creating a handler for negative acknowledgements. If your
administration queue receives a message indicating that a message failed to be
delivered or was not received before it timed out, journal queues provide you a way
to find and resend that message. You can use the CorrelationID from the
acknowledgement message to find the original in the journal queue (matching to the
journal message's ID property) and then take that original message and send it
again. This is an example of how these two features, acknowledgements and
journaling, can be used together to make your messaging application more robust
and reliable. The code to follow watches an administration queue (.\private$\ack in
this example) for negative acknowledgements (delivery and receipt) and then looks
in the local machine's journal queue for that message. Using the ReceiveByID method
of the MessageQueue, this can be done without having to scan the entire journal
queue. Finally, having found the original message, a new message is created,
populated with the body from the original, and then resent.
Note Once the journal message is received from the journal queue, it is removed
from the queue, so the journal queue is not a permanent record of every message
sent or received.
Visual Basic .NET
adminQ.MessageReadPropertyFilter.CorrelationId = True
journalQ.MessageReadPropertyFilter.SetAll()
Do
myMessage = adminQ.Receive()
Case Acknowledgment.Purged, _
Acknowledgment.QueueExceedMaximumSize, _
Acknowledgment.QueuePurged, _
Acknowledgment.ReachQueueTimeout, _
Acknowledgment.ReceiveTimeout
Try
originalMessage = _
journalQ.ReceiveById(myMessage.CorrelationId)
& originalMessage.DestinationQueue.FormatName())
originalMessage.TimeToBeReceived = _
New TimeSpan(1, 0, 0, 0)
targetTypes(0) = GetType(String)
originalMessage.Formatter = _
New XmlMessageFormatter(targetTypes)
resendMessage = New Message()
With resendMessage
.Body = CStr(originalMessage.Body)
.UseJournalQueue = True
.AcknowledgeType = AcknowledgeTypes.FullReachQueue _
Or
AcknowledgeTypes.FullReceive
.AdministrationQueue = _
New
MessageQueue(".\private$\Ack")
End With
msgQ.Send(resendMessage, _
MessageQueueTransactionType.Single)
Catch e As Exception
Console.WriteLine(e.Message)
Pause()
End Try
Case Else
End Select
Loop
Visual C# .NET
MessageQueue msgQ;
Message originalMessage;
adminQ.MessageReadPropertyFilter.CorrelationId = true;
journalQ.MessageReadPropertyFilter.SetAll();
*/
do
* to stop execution. */
myMessage = adminQ.Receive();
switch(myMessage.Acknowledgment)
case Acknowledgment.Purged:
case Acknowledgment.QueueExceedMaximumSize:
case Acknowledgment.QueuePurged:
case Acknowledgment.ReceiveTimeout:
//failed
try
originalMessage =
journalQ.ReceiveById(myMessage.CorrelationId);
+ originalMessage.DestinationQueue.FormatName());
originalMessage.TimeToBeReceived = new
TimeSpan(1, 0, 0, 0);
originalMessage.Formatter =
new XmlMessageFormatter(targetTypes);
resendMessage.Body =
(string)originalMessage.Body;
resendMessage.UseJournalQueue = true;
0, 0);
resendMessage.AcknowledgeType =
AcknowledgeTypes.FullReachQueue |
AcknowledgeTypes.FullReceive;
resendMessage.AdministrationQueue =
new MessageQueue(@".\private$\Ack");
msgQ.Send(resendMessage,
MessageQueueTransactionType.Single);
catch(Exception e)
{
Console.WriteLine(e.Message);
Pause();
break;
default:
break;
while(true);
targetTypes(0) = GetType(String)
Try
With myMessage
.UseJournalQueue = True
.AcknowledgeType = AcknowledgeTypes.NegativeReceive
End With
msgQ.Send(myMessage, MessageQueueTransactionType.Single)
Catch e As Exception
Console.WriteLine(e.ToString)
End Try
Visual C# .NET
try
myMessage.UseJournalQueue = true;
myMessage.AcknowledgeType = AcknowledgeTypes.NegativeReceive;
myMessage.AdministrationQueue = new
MessageQueue(@".\private$\Ack");
msgQ.Send(myMessage, MessageQueueTransactionType.Single);
catch(Exception e)
Console.WriteLine(e.ToString);
}
Conclusion
Message Queuing (MSMQ) support for transactions allows you to execute multiple
actions against both MSMQ and other resources that are compatible with Microsoft
DTC and ensure that all of these actions are treated as a single unit. Combining
those transactional abilities with the various tracking features of MSMQ allows you all
the flexibility you need to build a robust system for asynchronous messaging