TIBCO BWSE Performance Best Practices

TIBCO ActiveMatrix BWSE Performance Best Practices
Munira Baldiwala
Seema Deshmukh
[Please send comments and suggestions to perf-qa@tibco.com]
Version History
Version Date Author(s) Change Description

0.1 01/01/2009 Amol Khire Designed scenarios, ran test cases, created
Shashank document.
Jahagirdar
Munira Baldiwala
Seema Deshmukh
Raymond Xu
TIBCO Software Inc.
TIBCO Confidential
© Copyright 2008 TIBCO Software Inc. All rights reserved.

All product names are the trademarks of their respective companies
Document Overview
This document discusses the performance of the TIBCO ActiveMatrix BusinessWorks Service Engine
5.7.0. The intent is to make the results of the comprehensive benchmark tests available to the users
to make informed architecture decisions.
Prior knowledge of TIBCO ActiveMatrix BW and TIBCO ActiveMatrix BWSE concepts is desirable for
the best use of this document.
Disclaimer: The performance results in this document are provided for the reference only.
They can be reproduced only under the exact environment and conditions as existed when the
tests were done. The performance numbers should be used only as a guideline and TIBCO
does not guarantee their accuracy.
TIBCO Software Inc.

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 2
TIBCO3165 Porter Drive, Palo Alto, CA 94304
Confidential
Tel: 650-846-5714 Fax: 650-846-1005
http://www.tibco.com
Table of Contents
Version History...........................................................................................................1
Introduction................................................................................................................5
BWSE Architecture and Message Flow...................................................................6

Basic Architecture of an Un-Wired BWSE Service....................................................................6
Basic Architecture of a Wired BWSE Service (Same Node)......................................................8
Basic Architecture of a Wired BWSE Service (Two Different Nodes).....................................11
Architecture of a Wired BWSE Service with InvokePartner (Same Node).............................14
Comparative Performance and Tuning Guidelines..............................................18

Performance with varying concurrent users (BWSE Un-Wired scenario).............................18
Performance with varying concurrent users (BWSE Wired scenario)...................................20
Performance Comparison with varying Payload size (BWSE Unwired vs. BW Standalone)22
Performance Comparison with varying Concurrent users.....................................................23
Memory and Thread Footprint................................................................................25

Footprint of Different AMX Components..................................................................................25
Footprint at Runtime................................................................................................................... 26
Heap Usage Evaluation...........................................................................................27

Scenario (Varying Doc size)....................................................................................................... 27
Observation................................................................................................................................. 27
Analysis....................................................................................................................................... 27
Application of BW Tuning Properties in BWSE....................................................29
Use Cases.................................................................................................................30
SOAP (HTTP)-BWSE-Java Service............................................................................................30
References................................................................................................................33
References................................................................................................................33
Appendix A: Service Implementations...................................................................34

TIBCO Confidential
Appendix B: Hardware Configuration....................................................................36
Appendix C: Tuning Parameters.............................................................................38
Appendix D: Thread and Memory Footprint..........................................................39

TIBCO Confidential
Introduction
ActiveMatrix BusinessWorks Service Engine (BWSE) enables ActiveMatrix BusinessWorks to
participate in the ActiveMatrix environment which supports the development, deployment, and
management of applications that conform to a service-oriented architecture (SOA).
BWSE allows other ActiveMatrix components to discover and invoke ActiveMatrix BusinessWorks
services. It also allows ActiveMatrix BusinessWorks to consume services provided by other
components in the ActiveMatrix platform.
This document discusses the performance considerations for TIBCO BusinessWorks Service Engine
(BWSE). A brief overview of the BWSE message flow is provided to facilitate understanding of the
tuning parameters. A summary of the comparative performance is provided to help users understand
the performance characteristics of the product. A few use cases are discussed to illustrate the
application of tuning parameters.
All the tests in this document are performed on the configuration as stated in Appendix B:
Hardware Configuration

TIBCO Confidential
BWSE Architecture and Message Flow

An understanding of the BWSE architecture and message flow is essential before we dive into the
tuning parameters.
Basic Architecture of an Un-Wired BWSE Service

Here we take a simple use-case in which an Un-Wired BWSE Service is invoked by an external
SOAP Client over HTTP. The following diagram depicts a typical message flow for an in-out message
exchange for an invocation of this service. The service implementation details are in Appendix A:
Light Weight Service
Diagram1: Architecture of an Un-Wired BWSE Service

TIBCO Confidential
Step 1:
Concurrent users send SOAP requests (over HTTP) to be serviced by the BWSE service provider in
the AMSG Environment.
Step 2:
The SOAP (over HTTP) Requests from the client reach the BWSE Service (Service resource) in the
AMSG Environment. They are serviced by the Tomcat threads of the BW Engine. The MIN and MAX
size of the HTTP Server Thread pool can be set through the “bw.properties” file. The default values
of these properties are 10 and 75, respectively. An HTTP Server thread is allocated for each request
that is concurrently executing and is released only after the response is sent back (i.e. The HTTP
Server thread is blocked till the response is received). The HTTP Server thread routes the request for
further processing to the Engine thread (aka Job Courier thread)
Step 3:
The Job courier thread in BW picks up the message for processing. The max value of the Job courier
thread pool can be set in the bw.properties file using the property:
Engine.Threadcount
(This property determines the maximum Engine thread pool size. This property is applicable to that
specific BW service unit deployed on that node. An Engine thread pool is created for every service
unit deployed on the node.)
Step 4:
The response is then sent back by the Engine thread to the Tomcat thread (which is blocked waiting
for a response).
Step 5:
The Tomcat thread picks up the response and delivers it to the external SOAP Client.
In this scenario, the throughput depends on the latency of the service, the number of
concurrent users and the Engine thread pool size.

TIBCO Confidential
Basic Architecture of a Wired BWSE Service (Same Node)

Here we take a simple use-case in which an Wired BWSE Service is invoked by an external SOAP
Client over HTTP. The following diagram depicts a typical message flow for an in-out message
exchange for an invocation of this service. The service implementation details are in Appendix A:
Diagram2: Architecture of a Wired BWSE Service (Same Node)
SOAP Service Unit and BWSE Service Unit are deployed on the same AMSG node

TIBCO Confidential
Step 1:
Concurrent users send SOAP requests(over HTTP) to be serviced by a service provider in the
AMSG Environment.
Step 2:
The SOAP (over HTTP) Requests from the client reach the SOAP Service in the AMSG Environment.
They are serviced by the HTTP Server Shared Resource threads (HTTP Resource Adapter
threads) which are initialized for the HTTP Server Shared Resource configured through the AMSG
Administrator. The MIN and MAX size of the HTTP Server Thread pool can be can be set through the
AMSG Administrator. The default values of these properties are 10 and 75, respectively. An HTTP
Server thread is allocated for each request that is concurrently executing and is released only after
the response is sent back (i.e. The HTTP Server thread is blocked till the response is received).The
HTTP Server thread routes the request for further processing to service provider (BWSE). The BWSE
provider is a Local Provider (residing on the same node).The message is routed directly to the
appropriate provider for further processing.
Step 3:
The max value of the BWSE provider thread pool, can be set in the Node’s TRA configuration file
using the property:
java.property.com.tibco.matrix.amxframework.messaging.maxpoolsize
(This property determines the maximum thread pool size. This property is applicable to every service
unit deployed on that node. A separate thread pool is created every service unit deployed on the
node.)
Step 4:
The thread from the Engine Thread pool picks up the routed message for further processing. The max
value of the Engine thread pool can be set in the bw.properties file using the property:
Engine.Threadcount
specific BW service unit deployed on that node. A Engine thread pool is created for every service unit
deployed on the node.)
Step 5:
The SOAP Consumer pool thread picks up the response and delivers it to the HTTP Server Thread
(which is blocked waiting for a response).
The max value of the thread pool can be set in the Node’s TRA configuration file using the property:
The default and min values of this property are 10 and 5, respectively.
(This property determines the maximum thread pool size. This property is applicable to every
Component Service, Component Reference, Composite Service and Composite Reference deployed
on that node. A separate thread pool is created for every Component Service, Component
Reference, Composite Service and Composite Reference deployed on the node.)

TIBCO Confidential
Step 6:
The Tomcat thread picks up the response and delivers it to the external SOAP Client.
In a one node scenario, the throughput depends on the latency of the service, the number of
concurrent users and the Engine thread pool size. The maxpoolsize does not affect the throughput.

TIBCO Confidential
Basic Architecture of a Wired BWSE Service (Two Different Nodes)
Here we take a simple use-case in which a Wired SOAP-BWSE Composite Service is invoked by
an external SOAP Client over HTTP. The following diagram depicts a typical message flow for an in-
out message exchange for an invocation of this service. The service implementation details are in
Appendix A: Light Weight Service
Diagram3: Architecture of a Wired BWSE Service (2 Node)

TIBCO Confidential
SOAP Service Unit and Java Service Unit are deployed on 2 different nodes.(SOAP on one
node and Java on another node)
Step 1:
AMSG Environment.
Step 2:
They are serviced by the HTTP Server Shared Resource threads (HTTP Server threads) which are
initialized for the HTTP Server Shared Resource configured through the AMSG Administrator. The
MIN and MAX size of the HTTP Server Thread pool can be can be set through the AMSG
Administrator. The default values of these properties are 10 and 75, respectively. An HTTP Server
thread is allocated for each request that is concurrently executing and is released only after the
response is sent back (i.e. The HTTP Server thread is blocked till the response is received). The
Delivery Channel routes the request for further processing to the service provider (BWSE). The
BWSE provider, in this case, is a Remote Provider (residing on a different node).Hence, the Delivery
Channel routes the requests to the BWSE Component Queue on the EMS Server.
Step 3:
using the property:
node.)
Each provider pool thread acquires an available JMS session from the session-pool. It then gets the
request message from the BWSE Component queue (on EMS) and processes the request. The
number of concurrent sessions that can be created depends on the max value specified in the Node’s
TRA configuration file.
java.property.com.tibco.matrix.amxframework.messaging.sessioncount
(This property determines the maximum number of JMS sessions that may be created with the EMS
server. Like max thread pool property, this property also applies to every service unit deployed on that
node. A separate session pool is created for each service unit)
Step 4:
The thread from the Engine Thread pool picks up the routed message for further processing. At this
instant, the session is released and the BWSE provider thread is available for the next request. This
makes the BWSE Provider threads asynchronous in behavior. The max value of the Engine thread
pool can be set in the bw.properties file using the property:
Engine.Threadcount

TIBCO Confidential
Step 5:
The response is delivered by the Engine thread to the SOAP Service queue on EMS.
Step 6:
The SOAP Consumer pool thread picks up the response from the queue, the max value of which can
be set in the Node’s TRA configuration file using the property:
node.)
Each SOAP Consumer thread acquires a session from the session pool with EMS and picks up the
response delivered in the SOAP Service queue. The response is then delivered to the HTTP Server
Thread that is blocked on the response. The number of sessions that can be created depends on the
max value specified in the Node’s TRA configuration file.
java.property.com.tibco.matrix.amxframework.messaging.sessioncount
Step 7:
The HTTP thread sends back the response to the external SOAP Client.
In a two node scenario, the throughput depends on the latency of the service, the number of
concurrent users and the Engine thread pool size. Each Provider pool thread creates its own session
with the EMS Queue to fetch/deliver the request/response. In case of a BWSE component, as soon
as the Engine Thread finishes processing the request, the provider thread is released (synchronous
behavior).

TIBCO Confidential
Architecture of a Wired BWSE Service with InvokePartner (Same

Node)
Here we take a simple use-case in which a Wired SOAP-BWSE Composite Service is invoked by
an external SOAP Client over HTTP. The following diagram depicts a typical message flow for an in-
out message exchange for an invocation of this service. The service implementation details are in
Appendix A: Light Weight Service

TIBCO Confidential
SOAP Container BWSE Container IT Container

E
H x Consumer
J J IT Impl
1 T 2c o
B o
Network
T 10h 3 A b 4 5 Delivery b
Delivery
P C C Channel
Channel P
BWSE D A P 6
h o o
Impl
SOAP R a
Provider
o o
Clients A n l l
9 7
Legend (Threads)
HTTP Threads
SOAP Service Legend (Tuning)

Consumer Pool
Threads
A MaxPoolSize
Java
Provider Pool HTTPMaxConcurrent
B Connections
Threads
C JobCourier Pool
BWSE Provider
Pool Threads
D InvokePartner Pool
Job Courier
Pool Threads
InvokePartner
Pool Threads
BWSE Consumer
Pool Threads
Diagram4: Architecture of a Wired BWSE Service with Invoke Partner (Same Node)

TIBCO Confidential
SOAP Service Unit, BWSE Service Unit and Java Service Unit are deployed on the same
node.
Step 1:
AMSG Environment.
Step 2:
They are serviced by the HTTP Server Shared Resource threads (HTTP Server threads) which are
initialized for the HTTP Server Shared Resource configured through the AMSG Administrator. The
MIN and MAX size of the HTTP Server Thread pool can be can be set through the AMSG
Administrator. The default values of these properties are 10 and 75, respectively. An HTTP Server
thread is allocated for each request that is concurrently executing and is released only after the
response is sent back (i.e. The HTTP Server thread is blocked till the response is received). The
HTTP Server thread routes the request to service provider (BWSE) for further processing. In this
case, BWSE provider is a Local Provider (residing on the same node). The message is routed directly
to the appropriate provider for further processing.
Step 3:
using the property:
node.)
Step 4:
The thread from the Engine Thread pool picks up the routed message for further processing. The max
value of the Engine thread pool can be set in the bw.properties file using the property:
Engine.Threadcount
Step 5:
The Engine thread processes this message. Since BWSE has a reference partner, the request is then
forwarded and picked up by a thread in the InvokePartnerActivity Thread pool.
The max value of this thread pool can be set in the bw.properties file using the property:
InvokePartnerServiceActivityThreadPool.ThreadCount
The InvokePartner thread forwards the message to the Component it is wired to (in this case Java)
and waits for the response. Hence, the InvokePartner thread makes a synchronous call.

TIBCO Confidential
Step 6:
The Java provider pool thread picks up the request, the max value of which can be set in the Node’s
TRA configuration file using the property:
node.)
The message is then processed in Java. After Java finishes processing the message, the response is
delivered by the Java thread to the BW Consumer thread pool
Step 7:
A thread from the BW Consumer thread pool picks up the response, the max value of which can be
set in the Node’s TRA configuration file using the property:
node.)
BWSE consumer thread forwards this request to the Invoke Partner thread.
Step 8:
The InvokePartner thread which was waiting on this request picks up the response from the BW
Consumer thread. The response is then forwarded to the SOAP Consumer thread pool.
Step 9:
The SOAP Consumer pool thread picks up the response from the queue, the max value of which can
be set in the Node’s TRA configuration file using the property:
node.)
Step 10:
The HTTP thread sends back the response to the external SOAP Client.
In this scenario, the throughput depends on the latency of the service, the number of
concurrent users, the maxpoolsize, Jobcourier pool size and InvokePartner pool size value. The
maxpoolsize limits the number of requests that can be handled concurrently by the AMSG
Containers. The InvokePartner threads being synchronous, the throughput to a large extent depends
on their value.

TIBCO Confidential
Comparative Performance and Tuning Guidelines

This section discusses the impact on performance (of a service hosted in the AMSG environment) by
varying the number of concurrent users, payload, transport options and service implementations. It
also lists performance guidelines and tips for tuning certain performance critical parameters.
Performance with varying concurrent users (BWSE Un-Wired scenario)

The impact of increasing the concurrent users is verified by understanding the impact on throughput
and CPU Utilization by keeping the endpoint configuration and the service implementations constant.
The results for SOAP over HTTP Transport with increasing concurrent users for a 1K Request and 1K
Response document size are depicted below:
Graph1: Results for BWSE Unwired SOAP Service (Appendix A: Light Weight Service)
As the number of concurrent users is increased, the throughput increases linearly until it reaches a
peak (at 65% CPU utilization) and then drops from there.

TIBCO Confidential
Tips for scaling performance with a large number of concurrent users
1. Increase the MaxPoolsize and Engine.Threadcount as the number of concurrent

requests increase. If the MaxPoolsize value is lower than the number of concurrent
users, there would be a contention for pool threads and it may cause the latency to
increase. Hence, as a rule of thumb MaxPoolsize should be kept proportionate to
the number of users. (For further information on tuning the Engine.Threadcount,
refer to BusinessWorks Best Practices Guide)
2. As seen in the graph above, it is observed that as the concurrent users increase, the
throughput also increases until CPU utilization reaches 65%. It does not scale
beyond 65% with the below mentioned hardware.

TIBCO Confidential
Performance with varying concurrent users (BWSE Wired scenario)

The impact of increasing the concurrent users is verified by understanding the impact on throughput
and CPU Utilization by keeping the endpoint configuration and the service implementations constant.
The results for SOAP over HTTP Transport with increasing concurrent users for a 1K Request and 1K
Response document size are depicted below:
Graph2: Results for BWSE Wired SOAP Service (Appendix A: Light Weight Service)
As the number of concurrent users is increased, the throughput increases linearly until it reaches a
peak and then drops from there.
Tips for scaling performance with a large number of concurrent users
1. Increase the maxpoolsize and/or Engine.Threadcount as the number of

concurrent requests increase. If the maxpoolsize value is lower than the number of
concurrent users, there would be a contention for pool threads and it may cause the
latency to increase. Hence, as a rule of thumb maxpoolsize should be kept
proportionate to the number of users.
throughput also increases till a point where it reaches its peak (at 65% CPU
utilization).
3. For scalability with a very large number of concurrent users consider using an
external HTTP load balancer (for SOAP over HTTP), especially when the service is

TIBCO Confidential
light weight and responds immediately. This is because with such a service, the
ability of the SOAP processing limits the throughput. Using an external load balancer
will effectively “clone” the service. However, if the service is CPU intensive,
scalability can be achieved by opting for the inbuilt load balancer in ActiveMatrix
framework. This is further explained in the section on Load Balancing. Similarly for
scalability using SOAP over JMS, multiple instances of an identical service can listen
to the same EMS queue and you can load balance using EMS queues.
4. For HTTP transport, increase the HTTP server threads as the number of concurrent
users accessing the service increase. If the HTTP server max threads are lower than
the concurrent users, the Node will throw connection errors and an error response
shall be returned to the clients those are unable to retrieve an HTTP server thread.
The HTTP server threads are synchronous in nature as hence merely increasing the
framework pool size will not increase throughput. The process address space
available for the process, thread stack size and the maximum heap size will
influence the maximum number of HTTP Server threads that can be created. So if
you have an extremely large (1000+) number of concurrent users, HTTP may have
some inherent limitations. Using JMS as the transport will help in this case. The
messages will accumulate on the EMS queue (this will impact latency) but at-least
they can be processed serially.
 For comparative understanding of HTTP versus JMS, refer ActiveMatrix

Service Grid Performance Whitepaper at
https://newsalescentral.tibco.com/docs/DOC-9657

TIBCO Confidential
Performance Comparison with varying Payload size (BWSE Unwired vs. BW

Standalone)
It is observed that as the payload size (both request size and response size are identical) is
increased, the throughput significantly reduces. For large document sizes (1MB+) the degradation is
particularly prominent. SOAP Processing (serialization/de-serialization, parsing of SOAP Envelope) is
CPU intensive and the cost is a function of the message size. As the payload size increases so do the
CPU cycles spent in processing the payload and this impacts performance. For all the scenarios
below the CPU consumption on the machine which the node was running was maxed out.
The results for varying document size are depicted below:
Graph3: Results for BWSE Unwired and Standalone BW Service (Appendix A: Light Weight Service)

TIBCO Confidential
Performance Comparison with varying Concurrent users

The impact of wiring the BWSE Service with a SOAP service is verified by
understanding the impact on throughput and CPU Utilization by keeping the
endpoint configuration and the service implementations constant for two
scenarios, namely, Unwired scenario and Wired SOAP Scenario.
The comparison results for SOAP Service (SOAP over HTTP Transport) with BW
Standalone, BWSE Unwired and BWSE Wired for a 1K Request and 1K Response
document size are depicted below:
Graph4: Performance Comparison BW, BWSE Unwired, BWSE wired for SOAP Service (Appendix A: Light
Weight Service)
Note: Results are for Implementation Used in Appendix A: Light Weight Service

TIBCO Confidential
1. Performance of the Wired scenario gets better as compared to the Unwired scenario as
the number of users increases. The above graph proves that BWSE Unwired does not scale
appropriately with increasing users, where BWSE Wired depicts better scaling.
throughput also increases till a point where it reaches its peak.
3. In case of the Wired scenario, increase the maxpoolsize and/or Engine.Threadcount as

the number of concurrent requests increase. If the maxpoolsize value is lower than the
number of concurrent users, there would be a contention for pool threads and it may cause
the latency to increase. Hence, as a rule of thumb maxpoolsize should be kept proportionate
to the number of users. Also, here, the HTTP RA threads of the SOAP Container are used to
service the request.
4. In case of the Unwired scenario, the maxpoolsize does not come into play, since the
service is directly accessed. Increase the Engine.Threadcount as the number of concurrent
requests increase. Though this may not always be required depending on the service
response time and hardware configuration. Also, here, the Tomcat threads of the BW Engine
itself are used to service the request.

TIBCO Confidential
Memory and Thread Footprint

To understand the memory footprint and the thread usage with different AMX
components at deploy time and run time, multiple tests were conducted. This
section compares the memory and thread usage of different BWSE scenarios.
Footprint of Different AMX Components

The memory footprints of other AMX components are as follows:
Table1: Memory Footprint of different AMX components
All these tests have been run with the following settings:
Heap Settings:
Max Heap=1024 MB (default)

Stack Size=256 KB (default)
[Maximum stack size depends on whether it is 32 bit/64 bit. With a 32 bit Windows OS, the maximum stack size is 256 KB and
with a 64 bit Windows OS, the maximum stack size is 512 KB. ]
Other Application settings:
Min number of HTTP Shared Resource threads = 10

Max number of HTTP Shared Resource threads = 75
Max AMX Framework threads = 10
Engine = 8
Max Session count = 10
On Node startup, (without having deployed any service assemblies), maximum heap memory
reaches 175 MB, whereas stable state memory is about 85MB. The total number of live threads are
about 53.

TIBCO Confidential
Footprint at Runtime
Following tests were conducted with varying the type of services/components/references and their
number:
The above tests were conducted with all components, services and references on the same node.
For details of the above threads per component, refer Appendix D: Thread and Memory Footprint.
Table2: Runtime Memory and Thread Count with different scenarios.
NOTE:: The above memory values do not include the non-heap memory, which is a constant of 60 MB throughout.

TIBCO Confidential
Heap Usage Evaluation
Scenario (Varying Doc size)

The scenario considered for analysis is SOAP Service scenario with constant 1
user and varying payload size. Here, the response is an echo of the request.
Hence, response data size is equal to the request data size.
The Max heap size of the Engine and Node are set to 512MB and 640MB,
respectively.
Observation
Graph5: Memory Graph with increasing concurrent Users and Doc Sizes
Analysis
For BW Standalone:

TIBCO Confidential
Heap Usage = (6.9 * M * N) + 11.5 Where M = Max (Request doc size or

Response doc size)
N = number of concurrent requests in a system
For BWSE Unwired:
Heap Usage = (19.2 * M * N ) + 85.6 Where M = Max (Request doc size or

Response doc size)
N = number of concurrent requests in a system
For BWSE Wired:
Heap Usage = 113.98 * e ^ (0.1761 * M * N) Where M = Max (Request doc size

or Response doc size)
N = number of concurrent
requests in a system
For Example:
If, Request Doc size = 1 MB, Response Doc size= 2
MB, Number of concurrent requests = 5
Then, M= 2, N=5.
For BWSE Unwired: Heap Usage = (19.2 * 2 * 5) +

85.6 =277.6 MB
For BWSE Wired: Heap Usage = 113.98 * e ^

(0.1761 * 2 * 5) =663.164447 MB
Please note: The above equations apply only to the above payload size range
indicated.

TIBCO Confidential
As observed, the graph for BWSE Wired use case depicts an exponential behavior. Thus,
with increase in document size, the heap usage does not linearly grow, but increases
exponentially.

TIBCO Confidential
Application of BW Tuning Properties in

BWSE
Two properties that are important for BW engine memory management are
MaxJobs and FlowLimit. If the expected workload would not exceed the available
memory resources, one can set MaxJobs=0 and FlowLimit=0. Calculating
MaxJobs and FlowLimit for a process engine depends on the following factors:
 Amount of memory reserved for the engine JVM.

 Maximum size of a process instance object.
 Number of process instances per engine
A best practice is to design the process deployment in such a way that the target
peak loads can be accommodated by the BW engine(s) without exceeding the
memory limit, and then set MaxJobs and/or FlowLimit to reflect the number of
concurrent jobs that are expected under this peak load.
Setting MaxJobs=n and ActivationLimit=enabled, causes only ‘n’ jobs to be

processed at a time. However, if this does not suffice, one may consider setting
the FlowLimit at the job creation level itself.
For Unwired scenarios, the effect of setting the above-mentioned parameters

remains the same (apart from the additional footprint of a node). However, in case
of Wired scenarios in BWSE, settings these parameters does not help in limiting
memory resource consumption.
Take a look at the table below to understand what this exactly means.
The scenario used below is a simple SOAP (HTTP) -> BWSE scenario which
echoes the request as the response. A payload size of 5MB is sent as data. The
input rate is 0.4 txn/sec.
Heap settings: Min=256M, Max=1024M

BW Settings: Engine.Threadcount = 16
Value Memory (MB)
MaxJobs + FlowLimit BW Standalone BWSE Unwired BWSE Wired

0+0 OOM OOM OOM
1+0 50 200 OOM
4+0 300 480 OOM
8+0 600 OOM OOM
16 + 0 OOM OOM OOM
0+1 80 175 OOM
0+4 400 450 OOM
0+8 650 OOM OOM
8 + 16 600 OOM OOM

TIBCO Confidential
Table3: Memory Footprint of BW Standalone, BWSE Unwired and BWSE Wired
Note: OOM = Out of Memory.
Use Cases
SOAP (HTTP)-BWSE-Java Service
Diagram5: Soap-bwse-Java Service
This is SOAP over HTTP (Service)-BWSE-Java Composite Service scenario. The

BWSE Component merely forwards the request to the Java Component, which
echoes back the request as the response.
The Java Component has a sleep of 1 sec between getting the request and
sending the response back, so the theoretical throughput for a single concurrent
user is 1 tps.
Scenario 1:
Considering that the SOAP Service, Java and BWSE Components are on the same
node. As explained in the Architecture above, being on the same node, the
requests do not pass through the EMS (hence, the maxsessioncount property
does not play a role here), but are directly routed to the Java and BWSE Provider
through the Delivery Channel. Following are the results with varying Thread Pool
settings:
Table4: Throughput with Varying Thread Pool
The above results owe to the thread model designed for the One Node scenario.

TIBCO Confidential
Case 1:
When the MaxFrameworkThreadPool and Engine.ThreadCount is set to 32, the 32

concurrent requests get blocked in the Java implementation due to the Sleep
incorporated in the Java code. As there are enough Framework and Engine
threads compared to the concurrent requests available, the requests thread
switch from the HTTP Threads to the Engine threads to the Framework threads.
Hence, the throughput of 32 tps.
Case 2:
When the Engine.Threadcount is set to 32 and the MaxFrameworkThreadPool is

set to 10, the first 10 concurrent requests are processed and forwarded by the
Engine threads to the Java Provider. Here, these threads are blocked due to the
Sleep. As there are no further Framework threads available, the remaining 22
requests keep waiting. Hence, the throughput of 10 tps.
Case 3:
When the MaxFrameworkThreadPool is set to 10, the first 10 concurrent requests

get blocked in the Java implementation due to the Sleep incorporated in the Java
code. When the HTTP Threads find the Framework threads busy for the next
requests, a thread switch does not take place and the HTTP threads themselves
handle the further processing of the requests. Hence, the throughput of 40 tps.
Scenario 2:
Considering that the SOAP Service and each Java and BWSE Component are on
different nodes. As explained in the Architecture above, being on different nodes,
the requests pass through the EMS and are routed to the appropriate remote
BWSE and Java Provider through the Delivery Channel. Following are the results
with varying ThreadPool and Engine.Threadcount settings:
Table5: Throughput with Varying Thread Pool &Engine Thread Count
** SessionCount is kept equal to the MaxThreadPool

TIBCO Confidential
The above results can be explained as follows:
Case 1: When the maxpoolsize value of all the nodes is set to 10 and
Engine.threadcount is set to 32, the throughput is 10 tps as the number of
concurrent requests that can be processed are 10.
Case 2: When the maxpoolsize and Engine.threadcount value of the Java and
BWSE Nodes is set to 32 whereas that of the SOAP Node is set to 10, the
throughput is 32 tps. The maxpoolsize value of the SOAP Node does not affect the
throughput, because at the SOAP Node, the Consumer threads merely pick up the
requests and forward it to the HTTP Server threads which are waiting for the
response. The Consumer threads do not do too much of processing and hence 40
concurrent users do not cause a SOAP Consumer Thread contention.
Case 3 and Case 4: When the maxpoolsize value of even one of the Java Nodes is
set to 10, the throughput is 10 txn/sec. Hence the throughput gets limited by the
maxpoolsize of any of the Components, namely MaxThreadPool size of N2 in Case
3 and N3 in Case 4 being set to 10 in this case.
Case 5 and case 6: When the maxpoolsize value of the Java, SOAP and BWSE
Nodes is set to 32 whereas the Engine.threadcount is set to 16, the throughput is
16 tps. The number of Engine threads, here, controls the throughput.
One Node Scenario ::
The throughput is limited by the size of the Framework Thread Pool and the Engine
threads, and not by the MAX size of the HTTP Server Thread pool.
Two Node Scenario ::
The throughput depends on the size of the framework Thread Pool and Engine Thread
Pool. Each thread from the framework Thread Pool creates a session with the EMS
Queue. For maximum possible performance, the sessioncount should be kept equal to the
maxpoolsize. If the sessioncount value is set less than the value of the maxpoolsize, the
throughput would be limited by the sessioncount.

TIBCO Confidential

TIBCO Confidential
References
 BW Performance Analysis and Tuning Guide
http://insidearc2.tibco.com/kb_new/article2.asp?
BN=&arid=106146&mode=&comment=&rec=&wf=
 ActiveMatrix Service Grid Performance Whitepaper
 ActiveMatrix Service Bus Performance Whitepaper
 TIBCO ActiveMatrix Policy Agent Performance Best Practices
 TIBCO ActiveMatrix High Availability and Fault Tolerance
http://qa.tibco.com/EngDocLibrary/index.html
 Memory monitoring Guide http://insidearc.tibco.com/kb_new/article2.asp?

 Thread Monitoring Guide

http://insidearc.tibco.com/kb_new/article2.asp?
 Profiling TIBCO Applications with JProfiler

http://insidearc.tibco.com/kb_new/article2.asp?
 Thread Behavior
http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html

TIBCO Confidential
Appendix A: Service Implementations
Operation Input Output Implementation

Message Message
(Request) (Response)
EchoString 1
String 1 (string) (string)
EchoString String 2 (string)
String 3 (string)
……
Memory Intensive Service

Message Message
EchoString 1
(string)
String 2 (string)
String 1 (string)
EchoString String 3 (string)
5 MB payload
5 MB payload
……
CPU Intensive Service

TIBCO Confidential
Message Message
EchoString 1
EchoString
……
Batch Service

Message Message
EchoString 1
EchoString (5 MB file is
read)
……

TIBCO Confidential
Appendix B: Hardware Configuration

Hardware Used
Machine 3
(AMSG Node Machine 1 (Load
1/BW Engine ) Generator)
Server Class Proliant HP Proliant DL-380
BL460c G1 (C- G4
Class)
Chip Vendor Intel Intel
CPU Model Intel XEON 5160 Intel XEON
Total Number 1 2
Of Processors
Cores Per 2 1
Processor
Threads Per 2 2
Processor
Clock 3 GHz 3 GHz
Memory 4 GB 4 GB
OS Windows 2003 Windows 2003
Server Server
L2 Cache 4 MB 2 MB
FSB 1333 MHz 800 MHz
Network Gigabit Ethernet, Gigabit Ethernet,

Full Duplex Full Duplex

TIBCO Confidential
Single Node scenarios deployment architecture
Diagram6: Single Node scenario deployment Architecture
Multi Node scenarios deployment architecture
Diagram7: Multi Node scenario deployment Architecture

TIBCO Confidential
Appendix C: Tuning Parameters
The default parameters in the Node .tra were used for the JVM.

TIBCO Confidential
Appendix D: Thread and Memory Footprint

TIBCO Confidential

TIBCO Confidential

TIBCO BWSE Performance Best Practices

Diunggah oleh

Informasi Dokumen

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

TIBCO BWSE Performance Best Practices

Diunggah oleh

Hak Cipta:

Format Tersedia

TIBCO ActiveMatrix BWSE Performance Best Practices

[Please send comments and suggestions to perf-qa@tibco.com]

Version Date Author(s) Change Description

TIBCO Software Inc.

© Copyright 2008 TIBCO Software Inc. All rights reserved.

TIBCO Software Inc.

BWSE Architecture and Message Flow...................................................................6

Comparative Performance and Tuning Guidelines..............................................18

Memory and Thread Footprint................................................................................25

Heap Usage Evaluation...........................................................................................27

Application of BW Tuning Properties in BWSE....................................................29

Appendix A: Service Implementations...................................................................34

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 3

Appendix B: Hardware Configuration....................................................................36

Appendix C: Tuning Parameters.............................................................................38

Appendix D: Thread and Memory Footprint..........................................................39

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 4

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 5

BWSE Architecture and Message Flow

Basic Architecture of an Un-Wired BWSE Service

Diagram1: Architecture of an Un-Wired BWSE Service

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 6

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 7

Basic Architecture of a Wired BWSE Service (Same Node)

Diagram2: Architecture of a Wired BWSE Service (Same Node)

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 8

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 9

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 10

Basic Architecture of a Wired BWSE Service (Two Different Nodes)

Diagram3: Architecture of a Wired BWSE Service (2 Node)

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 11

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 12

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 13

Architecture of a Wired BWSE Service with InvokePartner (Same

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 14

SOAP Container BWSE Container IT Container

SOAP Service Legend (Tuning)

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 15

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 16

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 17

Comparative Performance and Tuning Guidelines

Performance with varying concurrent users (BWSE Un-Wired scenario)

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 18

Tips for scaling performance with a large number of concurrent users

1. Increase the MaxPoolsize and Engine.Threadcount as the number of concurrent

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 19

Performance with varying concurrent users (BWSE Wired scenario)

Tips for scaling performance with a large number of concurrent users

1. Increase the maxpoolsize and/or Engine.Threadcount as the number of

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 20

 For comparative understanding of HTTP versus JMS, refer ActiveMatrix

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 21

Performance Comparison with varying Payload size (BWSE Unwired vs. BW

The results for varying document size are depicted below:

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 22

Performance Comparison with varying Concurrent users

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 23

3. In case of the Wired scenario, increase the maxpoolsize and/or Engine.Threadcount as

© Copyright 2008 TIBCO Software Inc. All rights reserved. Page 24

Memory and Thread Footprint

Footprint of Different AMX Components

Table1: Memory Footprint of different AMX components

Max Heap=1024 MB (default)

Other Application settings: