1 Flow Architecture considerations

The most fundamental recommendation to consider when writing a message flow is to be as concise as possible. There is a cost associated with passing through each of the primitive nodes, so in performance terms it is best to write flows in as few nodes as possible. By default, WMQI message flows treat all messages as transactional. For non-persistent messages it is often unnecessary to treat them in this way. By setting Transaction Mode to automatic as a property of the MQInput node, persistent messages will be treated, as transactions while non-persistent messages are not. If the message flow routes messages in several directions, a RouteToLabelNode may be a cheaper alternative than a number of Filter nodes. With RouteToLabelNode there are a number of things to consider. RouteToLabelNode nodes offer a significant benefit when the number of Label nodes is large. If you have a small number of destinations, it may be cheaper to use two or three Filter nodes rather than RouteToLabelNode and Label node style architecture. Flow order node can be used to handle scenarios wherein; we need to work on two messages in a message flow. Consider the scenario wherein a message flow using the data from one message say Msg1 we need to create a new message Msg2 that has to be discarded further down the flow and we need Msg1 from then on. In this case, Flow order node can be used such that the creation and manipulation of Msg2 takes place through First Terminal of the Flow order node and the Msg2 is discarded, and when its comes back to the second terminal its gets back the original message, Msg1.

2 Tips for WMQI Development

2.1 Efficient way to Merge XML trees One of the common things in the ESQL manipulation is to merge two XML trees. The code snippet below elucidates the usage of certain advanced ESQL commands and of the Reference variables, which is discussed below.
/* A Reference variable pointing to a tag in the input message. */ DECLARE TempTree Reference TO InputRoot.XML.msg.*[]; /* CREATE PARSE combination is used construct a tree without using a ResetContentDescriptor */ CREATE LASTCHILD OF OutputRoot DOMAIN 'XML' PARSE (Environment.variables.xmltree); DECLARE Output Reference TO OutputRoot.XML.msg; DECLARE J INTEGER; DECLARE K INTEGER; SET K = 1; SET J = CARDINALITY(OutputRoot.XML.msg.*[]); WHILE K <= J DO SET Output.temp1.msg.*[K] = Output.*[K]; SET K=K+1; END WHILE; DECLARE RefTree Reference TO OutputRoot.XML.msg.temp1.*[1]; DECLARE C CHARACTER;

/* Usage of the Reference variable and some advanced ESQL commands to efficiently traverse the tree */ WHILE LASTMOVE(TempTree) DO SET C = FIELDNAME(TempTree); IF(EVAL('RefTree.' || C) is null) THEN CREATE LASTCHILD OF Output FROM TempTree; END IF; MOVE TempTree NEXTSIBLING; END WHILE; SET OutputRoot.XML.msg.temp1 = null;

In the above example, two XML messages one coming from the input in the form of InputRoot and the other coming in the Environment Variable are merged together. This code snippet makes use of Reference variables extensively. It also uses ESQL commands like Lastmove, Nextsibling, CreateParse and does the traversal and construction of the output tree in an efficient manner. 2.2 Usage of Reference variables for Simplification of ESQL One of the most immediate benefits of utilizing reference variables, as in the above case is the simplification and increased readability of ESQL statements. Instead of long and difficult to read path traversals, a significantly simpler and clearer set of ESQL assignments may be made by pointing reference

variables deep within the tree to the starts of records and referencing subtree nodes relative to the reference variable. 2.3 Performance benefits from Reference variables The performance gain from using reference variables can be dramatically substantial. When not using reference variables, each reference to a node in the message tree involves a traversal down the tree from the root. This will increase the path length especially if array indexing is involved. By dynamically traversing the tree, it is not required to know in advance the number of elements in the tree at a given level. The CARDINALITY function returns this value. By using reference variables, this, potentially expensive, operation may also be eliminated. 2.4

A WHILE loop in a large XML array takes a long time to process

Scenario: A WHILE loop in a large XML array takes a long time to process. Explanation: This problem arises when you use the CARDINALITY() function to determine the size of the array in the WHILE statement. With large arrays, the cost of determining the number of elements is proportional to the size of the array. The CARDINALITY function is invoked each time the WHILE statement is executed. The message has many elements, therefore there is a significant overhead when running the loop in this way. Solution: Unless you have a message in which the number of elements of the array grows dynamically, determine the size of the array before entering the WHILE loop. Use code similar to the following example: DECLARE i,c INTEGER; SET i = 1; SET c=CARDINALITY(OutputRoot.XML.MyMessage.Array[ ]); WHILE i <= c DO . . . . . . . . . loop processing

END WHILE;

2.5

Performance problem with MQSI when the number of data tags in an xml message increases, the response times increase dramatically. The performance problem experienced is a result of the combination of the number of repeating fields and the navigation technique being used (the WHILE loop). These performance issues arise as a result of the following two points : (1)Field reference notation is used to access the parsed data in an underlying data model, which can change as a result of the execution of ESQL statements.

(2)In ESQL, each statement is executed as a discrete operation independent of any other statement. Therefore one ESQL statement is not 'aware' of the operations of a preceding statement. Therefore, each time MQSI performs an assignment, MQSI needs to navigate from the top of the message tree, to locate the correct position for the assigned element. This is because of point (2), which means the message tree could have been updated/changed since the last time a field reference was used. So when analyzing the ESQL : WHILE currentIndex < nrOfTags DO SET OutputRoot.XML.(XML.tag)"performancetest"."datauit"?currentIndex] = InputRoot.XML.(XML.tag)"performancetest"."datain"?currentIndex]; SET currentIndex = currentIndex + 1; END WHILE; ....then for every instance of the repeating field identified by "currentIndex" then MQSI need to re-navigate the tree to locate the correct data value that field references. It is possible to process repeating fields using the ESQL SELECT statement. This offers the advantage of being a single statement that has clauses for iteration over repeating fields. Because it is a single statement then it is not possible for the message tree to change during its execution. Therefore the message tree does not need to be re-navigated for every instance of the repeating field. In the example, the WHILE loop can be rewritten in a single SELECT statement, as follows : SET OutputRoot.XML.(XML.tag)"performancetest"."datauit"?] = (SELECT ITEM A FROM InputRoot.XML.(XML.tag)"performancetest"."datain"?] AS A ); This produces the same output as required, but the operations take tenths of seconds instead of seconds and minutes.

2.6 Message flow performance is reduced when you access message trees with many repeating records
Scenario: Message flow performance is reduced when the following conditions are true: o You are using ESQL processing to manipulate a large message tree. o The message tree consists of repeating records or a large number of fields. o You have used explicit SET statements with field reference paths to access or create the fields. o You have observed a gradual slowing of message flow processing as the ESQL processes more fields or repetitions. Explanation: This problem occurs when you use field references, rather than reference variables, to access or create consecutive fields or records. Consider the following example, in which independent SET statements use field reference paths to manipulate the message tree. The SET statement takes a source and target parameter, where either or both parameters are field references: SET OutputRoot.XML.TestCase.StructureA.ParentA.field = '1'; The problem arises when the SET statement is used to create many more fields, as shown in the following example: SET OutputRoot.XML.TestCase.StructureA.ParentA.field1 = '1'; SET OutputRoot.XML.TestCase.StructureA.ParentA.field2 = '2'; SET OutputRoot.XML.TestCase.StructureA.ParentA.field3 = '3'; SET OutputRoot.XML.TestCase.StructureA.ParentA.field4 = '4'; SET OutputRoot.XML.TestCase.StructureA.ParentA.field5 = '5'; In this example, the five fields that are created are all children of ParentA. Before the specified field can be created or modified, the broker must navigate the named message tree to locate the point in the message tree that is to be altered. For example: o To access field 1, the SET statement navigates to ParentA, then to the first field, therefore involving two navigations. o To access field 5, the SET statement navigates to ParentA, then traverses each of the previous fields until it reaches field 5, therefore involving six navigations. Navigating over all the fields that precede the specified field causes the loss in performance. Now consider a scenario that accesses repeating fields in an input message tree; for example: DECLARE myChar CHAR; DECLARE thisRecord INT 0; WHILE thisRecord < 10000 DO SET thisRecord = thisRecord + 1; SET myChar = InputRoot.MRM.myParent.myRepeatingRecord[thisRecord]; END WHILE; When index notation is used, as the count increases, the processing needs to navigate over all the preceding fields to get the one it wants; that is, it has to count over the previous records to get to the one that is represented by the current indexed reference. o When accessing InputRoot.MRM.myParent.myRepeatingRecord[1], one navigation takes place to get to the first record.

When accessing InputRoot.MRM.myParent.myRepeatingRecord[2], two navigations take place to get to the second record. o When accessing InputRoot.MRM.myParent.myRepeatingRecord[N], N navigations take place to get to the Nth record. Therefore, the total number of navigations for this WHILE loop is: 1 + 2 + 3 + .... + N, which is non linear. Solution: If you are accessing or creating consecutive fields or records, use reference variables. When you use reference variables, the statement navigates to the main parent, which maintains a pointer to the field in the message tree. The following example shows the ESQL that can be used to reduce the number of navigations when creating new output message tree fields: o SET OutputRoot.XML.TestCase.StructureA.ParentA.field1 = '1'; DECLARE outRef REFERENCE TO OutputRoot.XML.TestCase.StructureA.ParentA; SET outRef.field2 = '2'; SET outRef.field3 = '3'; SET outRef.field4 = '4'; SET outRef.field5 = '5';

When referencing repeating input message tree fields, you could use the following ESQL: DECLARE myChar CHAR; DECLARE inputRef REFERENCE TO InputRoot.MRM.myParent.myRepeatingRecord[1]; WHILE LASTMOVE(inputRef) DO SET myChar = inputRef; MOVE inputRef NEXTSIBLING NAME 'myRepeatingRecord'; END WHILE; For further information, see Creating dynamic field references.

2.7

Avoid nested IF-ELSE statements Use ELSEIF and CASE/WHEN clauses Do not overuse the STRING manipulation function Do not use CARDINALITY statements inside loops

2.8 2.9

2.10 use REPLACE function in preference to a complete re-parsing 2.11 Header copy In the compute node, the message headers can be copied by just checking the copy message headers radio button, but it doesnt generate an optimal code. The cardinality operator was called each time the while loop is entered. The following code snippets shows a clear approach in to copy headers.
DECLARE IN_Loop_CTR INTEGER; DECLARE IN_Cardinality INTEGER; SET IN_Loop_CTR = 1; SET IN_Cardinality = CARDINALITY(InputRoot.*[]); WHILE IN_Loop_CTR < IN_Cardinality DO SET OutputRoot.*[IN_Loop_CTR] = InputRoot.*[IN_Loop_CTR]; SET IN_Loop_CTR=IN_Loop_CTR+1; END WHILE;

2.12 Arrays in ESQL To realize arrays in ESQL Environment variables can be used. The following code snippet elucidates its usage.
SET Environment.Variables.NameArr[1] = 'Name1'; SET Environment.Variables.NameArr[2] = 'Name2'; SET Environment.Variables.NameArr[3] = 'Name3'; SET Environment.Variables.NameArr[4] = 'Name4'; DECLARE TCNT INT; DECLARE CNT INT; SET TCNT = CARDINALITY(Environment.Variables.NameArr[]); SET CNT = 1; WHILE (I <= TCNT) DO -- Processing goes here. -- You could refer to array using Environment.Variables.NameArr[I] SET I = I + 1; END WHILE;

2.13 NewLine in ESQL Code The easiest way to declare a new line character in ESQL:
DECLARE nl CHAR; nl = CAST(x'<code>' AS CHAR CCSID <characterset>)

In the above <code> is the code of the new line character in the system, For example 0d0a in DOS or 0a in most of the Unix systems. <characterset> is the coded character set id of the system. For example, in an NT system:
nl = CAST(x'0d0a' AS CHAR CCSID 437)

The CCSID part is essential or otherwise WMQI will convert the code x'0d0a' to a string 0d0a instead of string <CR><LF>.

2.14 Routing to Multiple Output Queues/ Dynamic Routing The output from a message flow can be routed to more than one output Queue Destinations by using the OutputLocalEnvironment tree. It is also possible to route dynamically to an output Queue based on some criteria. Following code snippet illustrations how to achieve it. If the scenario is as given below If field1 = A then send to queue A If field1 = B then send to queue B Then In the compute node, ESQL code will look similar to this:
If (field1 = A) THEN SET OutputLocalEnvironment.Destination.MQ.DestinationData[1].queueName = 'queueA'; ELSE IF ( field1 = 'B') THEN SET OutputLocalEnvironment.Destination.MQ.DestinationData[1].queueName = 'queueB'; END IF; END IF;

Note: In MQOutput node, under advanced tab, you should select DestinationList instead of the default value of queueName.

2.15 How to use Compute mode property on the Advanced tab Advanced tab in the compute node allows us to choose the compute mode that will be used to process information being passed through the compute node. It is possible to choose which of the following combination Message LocalEnvironment Exception of components are generated and modified by the Compute node. Those components not included in the selection are passed on unchanged, even if modifications are done to those components and these modifications remain will be local to this node. The Environment component of the message tree is not affected by the mode setting. If it is not empty, it is passed on from this node. 2.16 Writing BLOB data to database with CCSID If a blob data is inserted into database from WMQI, it is inserted as a bit stream irrespective of the platform. If we need to parse this retrieved data, we should explicitly use CCSID of the original data.

This solves the problem that occurs when code developed in one platform is ported to another platform. Typical example is In the windows platform blob data gets inserted into DB2 database with CCSID 437. And subsequent operations on the data retrieved from database are done based on the assumption that data is in the CCSID 437, which is specific to windows. But when the same code is ported to another platform (OS/390) data insertion happens with CCSID 500. But the subsequent data handling operations are still in CCSID 437 leading to erroneous data and error. So we should always parse the data retrieved from the database with the CCSID of the original data. PARSE function in ESQL allows parsing with the required CCSID. 2.17 Routing Reply based on the Requestor In scenarios where the request might come from more than one kind of client/requestor and the reply being expected in different queues, instead of hard coding the queue names in the programs, the requestor can set the queue where it expects the reply in the ReplyToQ field of the MQMD. The responding application can copy the ReplyToQ value to the MQMD of the response message. Then we can use the MQReply Node to route reply to respective client which made the request.

3 Error Handling and Trace

There are three basic nodes for this area namely, Throw Node, Trace Node and TryCatch Node. The Throw Node has just an in terminal and is used within the message flow to throw exceptions. These may be caught by TryCatch nodes earlier in the message flow or may cause the processing of that particular message to cease and associated transactional activity to be rolled back. The Throw Node may be used to throw an exception based on message content to prevent additional failures downstream in the message flow. The Trace Node is used to aid in debugging the message flow. It has an in terminal and an out terminal. The out terminal passes on the input message unmodified. However, the Trace Node will format and write out a trace record to a specified destination, assisting in creating a record of the route a message has taken through the message flow. The trace format can be selected using a variety of options. The purpose of the TryCatch Node is to prevent exceptions of downstream nodes from terminating the processing of messages or transactions, which is likely to happen if the exception drops back to the MQInput Node, which is the root node in the message flow. The message is received by an in terminal and forwarded on unchanged using the try terminal. If this node catches the exception it will be propagated using the catch terminal if this is connected. Error handling of the exception can then occur. If exceptions occur in the message flow have not been caught by other nodes, such as a TryCatch Node, the MQInput Node will catch the exception, as it was the initiator of the thread and the catch terminal on the MQInput node will propagate the message. 3.1 Considerations for Backout processing 1. When an exception is thrown within a message flow and is not caught by the inclusion of a TryCatch node, the original MQInput node catches it. 2. If a catch terminal is connected, the message is propagated to it and it is processed according to the message flow logic developed by you. 3. If a catch terminal is not connected, any transaction is rolled back. If the message was received under syncpoint, the original input message remains in the queue but MQSeries increments the MQMD Backout Count field accordingly. The MQInput node then reads this message again.

4. The MQMD BackoutCount is examined before the message is processed. If it is not zero, the message has been backed out and the broker performs its backout processing. This allows a message to be written to another queue that will allow subsequent messages on the input queue to be processed or removed in some other way. 5. If the MQMD Backout Count field is not zero, the backout processing performed by the broker follows these conditions: If the MQMD BackoutCount is less than the Backout Requeue Threshold attribute of the input MQSeries queue, the message is propagated to the output terminal of the MQInput node. This is the normal case. If a node is connected to the failure terminal of the MQInput node, the message is propagated to that node. If the Backout Requeue Name attribute of the input MQSeries queue is set, the message is written to it. If the dead-letter queue (DLQ) is defined, the message is written to it with an appropriate DLH (dead-letter header) attached. If the message cannot be processed according to the above, it remains in the input queue and is retried from time to time until it can be processed. 3.2 User Defined Exceptions ESQL code can be used validate business conditions and on violation, can be made to throw user defined business exceptions. The format to throw user defined exception is as shown below. If (true condition) then do something Else Throw user exception catalog 'WMQIv210' message 2949 values ('This is an used defined Exception') End if; Alternately, the Throw node can be used to force an error path through the message flow if the content of the message contains unexpected data. For example, to back out a message if it does not contain a particular field, you could check (using a Filter node) to see if the field exists. If it does not, the message could be passed to a Throw node that contains details (in the Message Text field for e.g. as: 'This is an used defined Exception') about the exception caused. 3.3 Points to consider in designing a Error Handling Sub flow This section basically elucidates the technicalities needs to be considered in designing a Generic Error Handling Subflow.

There are three categories of exceptions that can occur in any Business scenario namely, Business Exception - Throwing a user defined exception if the input field violates some business conditions. System Exception - Exception occurs if a queue or any system resource is not available. Application/WMQI Exception - Parsing errors that occur when WMQI tries in parse the input message expecting it in some format. A Generic Error handling subflow should be like a global error handler that can be included anywhere in the Message Flow where exception is expected to occur and is able to meet the objectives stated above. If any of the failure terminal or catch terminal is wired to do any error handling, then rolling back everything the message flow did until this point like, database updates, messages consumed off and written in to queue, has to be done. This rolling back happens only when the flow reaches back to the MQInput Node after an exception had occurred. To ensure this, Subflow is designed in a way that after doing the necessary processing to populate error-handling queue, it throws exception back so that flow retracts back to the MQInput Node of the Main flow. It is advisable to have a backout Queue for the Input queue to avoid the input message from staying as a poison message remaining in the input queue and being retried from time to time until it can be processed. At the Input Node, the error handling should be done at its catch terminal that is still within the UOW (Unit Of Work). The issue with MQInput's failure terminal processing, is that when the message is propagated there, it is in a completely new UOW. The message has been rolled back onto the queue, a new MQGET has happened, and no exception information is available. Handling of database exceptions is controlled by the checkbox 'ThrowExceptionOnDatabaseError' in the Advanced Tab of the database and compute nodes. It can be checked so that the broker throws an exception. The Generic Error Handling Subflow should also handles the userdefined Exceptions caused by the Throw ESQL statements. Parsing Exceptions, System Exceptions like queues not available should also be handled and appropriate diagnostic message be generated.