On the PurePath Go Beyond Profiling

Tracing transactions across distributed Java/.NET application components at production-safe overhead

Contents
EXECUTIVE SUMMARY..................................................................................................... 1 THE CHALLENGE OF SOFTWARE PERFORMANCE............................................................. 2 SOFTWARE PERFORMANCE ANALYSIS ............................................................................. 2 OVERHEAD VS. QUALITY AND PRECISION OF MEASUREMENT.......................................... 3 TRANSACTIONAL PERFORMANCE DATA COLLECTION ..................................................... 4 DIAGNOSING PERFORMANCE PROBLEMS IN 7X24 ENVIRONMENTS ................................ 6 CAPTURING OF DIAGNOSTIC INFORMATION................................................................. 6 ANALYSIS OF DISTRIBUTED ENVIRONMENTS.................................................................... 7 DYNATRACE DIAGNOSTICS EDITIONS ............................................................................ 8 COMPARISON MATRIX.................................................................................................... 9 CONCLUSION ............................................................................................................. 10

>> rapidly resolve performance problems in distributed java & .net applications

Executive Summary
Modern business runs on the mission-critical applications implemented and sustained by the IT organization. Efficient automation of business processes is no longer a luxury it is a necessity borne out of a fiercely competitive global marketplace. Business performance is often directly tied to software performance, and unplanned outages or poor application performance can result in the wrong kind of front page headlines. Application performance analysis is a complicated, far-reaching exercise that touches many organizations and processes, and requires the proper tools and best practices to guarantee consistent service delivery. One tool commonly utilized as part of this process is a system or application profiler, which relies on a number of assumptions that for many situations actually render it inadequate: The collection of simple statistical performance values may mask systemic performance problems and often makes root cause isolation difficult. Problematic end-user transactions cannot be fully reconstructed from profiling information.

High execution overhead and the lack of long-term monitoring capabilities mean profilers are not suitable for diagnosing problems in production environments. The high number of influential environmental factors, however, requires diagnosis in production. Missing information beyond basic execution times makes the isolation of context-specific performance problems nearly impossible. Additionally, the lack of support for diagnosis in distributed, heterogeneous .NET and Java applications makes profilers unsuitable for the diagnosis of performance problems in typical enterprise environments. dynaTrace Diagnostics steps in to fulfill all the requirements which profilers miss. dynaTraces PurePath Technology enables the collection of transactional performance metrics and related context information. Tracing across technology and server boundaries make it an enterprise level performance management solution. Low overhead and hot reconfiguration at runtime enable 7x24 diagnosis even in production environments.

On the PurePath Go Beyond Profiling

The Challenge of Software Performance

Sustaining adequate software performance is a daunting task. Fulfillment of functional and performance requirements are essential to software quality. Both are equal in their relevance to service delivery. Today, mission critical applications rely on functional richness as much as on adequate performance. Poorly performing software can potentially cause massive losses in revenue and put your business at risk. Ensuring software performance is a very challenging task, even more challenging than providing the correct functionality, because it depends on many variables: Bad code can cause poor algorithmic performance. User behavior and load volume directly impact response times and resource usage. Improper configuration of application servers or frameworks may degrade performance significantly. Inadequate distribution of software components can introduce bottlenecks. Many of these risk factors are specific to the underlying environment and can only be diagnosed in that specific setting, under specific circumstances. Performance engineers and developers find themselves increasingly confronted with performance problems that are difficult or impossible to diagnose. The increasing usage of thirdparty code and highly distributed software environments adds additional complexity.

Increasing complexity makes performance problems more and more difficult to diagnose.

Software Performance Analysis

Software Performance Analysis ideally adopts a systematic approach that supports the rapid isolation, diagnosis and resolution of performance problems in applications. The most widely used approach for software performance analysis is to sample the execution characteristics of code and collect execution time metrics to provide statistical data on the performance of individual methods. This is the approach taken by code profilers. Profilers, which are today among the most widely used tools for performance analysis, rely on a number of assumptions: They collect only statistical performance information, assuming that performance is independent of the actual user transaction context and flow. They can significantly alter the behavior of an application in order to collect performance metrics, assuming that there is no impact on the quality of the collected measurement (see Figure 1). They only collect CPU and execution time metrics, assuming that this information is sufficient for identifying performance bottleneck. They restrict information collection to a single runtime environment (either .NET CLR or Java VM), assuming that performance problems are limited only to single tiers of an application even in distributed Java Enterprise or SOA environments. These assumptions however do not reflect reality for most software applications. Consequently effective performance analysis cannot be provided by such tools alone.

Profilers rely on assumptions that do not hold true in modern environments

On the PurePath Go Beyond Profiling

We will therefore introduce dynaTrace Diagnostics as a best-of-breed solution for software performance management and diagnosis. dynaTrace Diagnostics provides functionality that goes far beyond the capabilities of profilers and thus enables a new level of performance analysis. The following chapters will address all major aspects of software performance analysis and illustrate dynaTrace Diagnostics added value.

Overhead vs. Quality and Precision of Measurement

The collection of application performance information requires additional code to be injected into the application. Common injection techniques for Java include the JVM Profiling Interfaces or byte code instrumentation. All these approaches affect system performance by adding overhead (see Figure 1).

Uninstrumented Application

Instrumented Application
Execution Time of Diagnosis Code

Figure 1 - Added overhead for performance metric collection

Profiling overhead influences the quality and precision of measurement.

The imposed overhead however is detrimental to the quality of that measurement. The more overhead created by the diagnosis tool the more inaccurate the measurement inherently becomes. Inadequate collection of performance measurements can even lead to the introduction of phantom performance problems. Some problems, such as synchronization issues for example, may even not be detectable as real world application behavior varies too widely from the original system. Profilers provide only limited support for the adjustment of measurement granularity in order to mitigate and control overhead. Most profilers collect performance information for each and every method that is executed. Others provide the ability to switch between a set of predefined granularity levels, while some have the ability to define exclusion rules at code level. However they all lack necessary features to fully support performance engineers who require fully flexible and fine-grained

On the PurePath Go Beyond Profiling

configuration of performance data collection in order to minimize overhead while maintaining all relevant diagnosis information. dynaTrace Diagnostics provides powerful, revolutionary features for the configuration of measurement collection. An intuitive class browser can be used to navigate packages, classes and methods in order to easily define flexible data collection configurations. Interactive, UI-based refinement of configuration and hot deployment without application restart adds additional convenience to the configuration process. Knowledge Sensor Packs allow the collection of relevant performance metrics for standard technologies like Java EE, ASP .NET, Spring or Hibernate (as well as many others) with no manual configuration effort.

Transactional Performance Data Collection

The ideal starting point for an in-depth analysis of application performance is a close examination of method execution times within critical user transactions. Performance management tools must strike a delicate balance between the granularity of collected performance information, which can range from recording every single method execution to recording only values aggregated over several seconds, and the overhead penalty required to collect it. Statistical performance Profilers present performance measurements only as aggregated values over a couple of seconds. This can easily lead to situations where performance problems metrics produced by are hidden especially in the case of statistical outliers. Moreover, collecting profilers often leave performance problems average values creates additional overhead above that required for collection of raw performance information. As only aggregated values are collected it is not undetected. possible to identify how many transactions experienced problems or to analyze the exact call tree of a problematic transaction. Analysis based on aggregated values can easily leave performance problems undiscovered. Figure 2 shows the sample output of a profiler for a portal-based web application using a generic request handler. Count handleRequest () buildWebControls () loadConfiguration () renderPage () 750 750 250 750 Time s (Total) 9500 7650 250 1500 Time s (Avg) 12.5 10.2 1 2

Figure 2 - Sample Profiler Output

PurePath Technology provides transactional performance metrics enabling better diagnosis and outlier isolation

dynaTrace Diagnostics uses its innovative PurePath Technology to collect transactional performance metrics. PurePath Technology enables the tracing of each individual transaction which has been executed within a system. Performance engineers get detailed, precise execution information including the complete call tree the PurePath. By recording each individual transaction, outliers can be identified easily even if there is only a single bad transaction within 100,000 executions.

On the PurePath Go Beyond Profiling

Figure 3 shows the simplified output for a portal application collected using PurePath Technology. By capturing all argument values in addition to detailed execution time metrics for each individual transaction, the performance problem which cannot even been seen in the profiler is immediately visible. Argument ... 500 of type handleRequest () buildWebControls () renderPage () ... 200 of type handleRequest () loadConfiguration () buildWebControls () buildWebControls () renderPage () ... 50 of type handleRequest () loadConfiguration () buildWebControls () buildWebControls () buildWebControls () buildWebControls () buildWebControls () renderPage () "show all details" 50 1 6 10 10 10 10 3
Figure 3 - PurePath based Execution Information

Time s

"start"

9 7 2

"overview"

12 1 4 5 2

On the PurePath Go Beyond Profiling

Diagnosing Performance Problems in 7x24 Environments

Performance Problems can very often only be analyzed in production as they are highly dependent upon the runtime environment. Performance problems, unlike many functional problems, can only be analyzed by taking the runtime environment into consideration. In previous sections weve described the factors influencing software performance, many of which are environment-specific. This is in large part the reason why developers are often unable to accurately reconstruct performance problems occurring in production environments within their local workstation environment. They require detailed performance data from the production system in order to perform the necessary performance analysis. Without this critical information they must expend significant time and effort in attempting to reconstruct the problem, or in the worst case scenario they are left to guess what the problem might be and try to fix it in a lengthy trial and error process. The significant performance overhead (10x to 1000x slower) introduced by profilers While profilers intromake their usage untenable in production environments. Installing a profiler in a duce far too much live production environment would result in immediate and significant performance overhead for production diagnosis, degradation and likely multiple SLA breaches. Also, as already mentioned, dynaTrace Diagnostics performance problems may occur only under very specific circumstances or conditions. This requires a performance diagnostics tool to run in a production can be used to find system for hours or even days to ensure capture of these transient, specific problem in 7x24 problems. Profilers are designed for only short-term data collection over a couple of production minutes. environments. In contrast, dynaTrace Diagnostics employs lightweight Diagnostics Agents, which only collect performance metrics. Correlation of measurements and execution path reconstruction is performed by the dynaTrace Diagnostics Server. Performing all resource intensive tasks outside the diagnosed system means that performance overhead is sustained at typically 3-5%, making dynaTrace Diagnostics ideally suited to usage within load tests and production environments

Capturing of Diagnostic Information

Effective root cause analysis requires capturing of diagnostic information beyond execution times. Practical experience shows that capturing performance metrics is only sufficient for identifying application components that have performance problems. Exact problem diagnosis, however, also requires gathering context-specific information. This is especially true for applications that use customizable frameworks (e.g., portals, Spring), usage of prepared database statements, generic argument-based based frameworks or Aspect Oriented Programming (AOP) frameworks impose additional complexity to diagnosis process, because their exact execution depends on specific invocation details(see Figure 3). Profilers only provide statistical execution time metrics. Therefore they cannot correlate additional context information to statistical performance metrics. As a consequence they fail to analyze performance problems that only occur in a specific scenario, which cannot be distinguished at the plain code level. Using PurePath Technology, dynaTrace Diagnostics can correlate context information to individual execution paths. Knowledge Sensors allow capturing of method arguments, SQL bind parameters, exceptions, etc. This information is put

On the PurePath Go Beyond Profiling

into to context of a specific transaction allowing the performance engineer or developer to pin down a context-dependent performance problem quickly and easily.

Analysis of Distributed Environments

Enterprise environments require transaction tracing across system boundaries and across technology domains. Todays application environments have become more and more distributed. Trends like SOA and EAI create systems that consist of a complex server infrastructure with services distributed across multiple servers implemented in different technologies. Performance diagnosis therefore requires the ability to follow transactions over server boundaries, irrespective of whether communication is handled using language-specific remoting protocols, Web Services or messaging systems. Profilers are designed to be used in single runtime developer environments. They do not provide any ability to diagnose performance problems in distributed environments. The diagnosis of problems in heterogeneous system is also impossible as these tools are limited to either Java or .NET.

Figure 4 Heterogeneous Transactional Diagnosis using PurePath

dynaTrace Diagnostics supports the diagnosis of distributed Java and .NET environments. Transactions can be followed across server boundaries. Tracing from .NET to Java and vice versa enables identifying performance problems in heterogeneous hybrid application environments. dynaTrace Diagnostics serves as the single solution for your company to analyze performance problem in enterprise environments as shown in Figure 4. With just one tool you can investigate your performance problems on every platform. Not having to learn multiple tools provides an immediate ROI of your tool investment.
On the PurePath Go Beyond Profiling 7

dynaTrace Diagnostics Editions

dynaTrace Diagnostics Editions are tailored to support performance analysis across the whole lifecycle. dynaTrace Diagnostics scales up with your analysis demands dynaTrace Diagnostics is offered in several product editions, each providing features for specific use cases. As dynaTrace Diagnostics targets the entire application lifecycle, supporting different requirements and usage scenarios helps differentiate each specific edition. Figure 5 provides an overview of the different editions and their target audience.

Figure 5 - dynaTrace Diagnostics Editions scaling with your requirements

dynaTrace Diagnostics Workstation Edition is used for performance analysis during code development. Developers immediately understand the dynamics and performance implications of their code. Development teams rely on dynaTrace Diagnostics Professional Edition to store configurations and Diagnostics Sessions on a central Diagnostics Server improving communication and speeding performance problem resolution. Automated build support enables integrating performance testing in continuous integration and automatic build environments. dynaTrace Diagnostics Professional and Enterprise Edition both enable monitoring and diagnosis in 24x7 production environments as well as testing and staging environments. Performance engineers and developers can reconstruct performance issues in production environments making reproduction in testing and staging environments obsolete.

On the PurePath Go Beyond Profiling

Comparison Matrix
dynaTrace Diagnostics fulfills or surpasses all the requirements for modern application performance diagnostics. The capabilities go far beyond those of Profilers. The following matrix lists the key capabilities required for efficient performance diagnosis and how they are supported by Profilers and dynaTrace Diagnostics.
Key Performance Analysis Capabilities Easy installation and integration into development and testing environments; requires no administration overhead Code level profiling details providing the information developers require to understand performance problems Configuration of profiling rules to limit overhead and restrict data collection to specific code paths Centralized Diagnostics Server for storing and managing configuration and monitoring results to support development teams Predefined configurations for standard technologies like Java EE, ASP.NET, etc. to automatically discover relevant application components Performance metrics for discrete transactions; permits identification of outliners Transaction tracing unveils the actual call tree showing the exact transaction flow Capture contextual diagnosis information (e.g., method arguments, exceptions, SQL calls, etc.) to accelerate root cause identification and error resolution Support tracing across technology and physical boundaries enabling the diagnosis of distributed enterprise applications Profiler dynaTrace Diagnostics Yes

Yes

Yes Limited1

Yes Yes1

No

Yes

No

Yes

No No

Yes Yes

No

Yes

No

Yes

1 Profilers limit configuration to certain level or use configuration files. dynaTrace Diagnostics provides intuitive visual configuration

On the PurePath Go Beyond Profiling

Conclusion
Todays mission critical business processes rely on high performance software. Performance problems however are much more difficult to find and fix than functional errors. They generally depend on usage characteristics or environmental influences. Increasingly complex and heterogeneous enterprise applications represent an additional challenge to the developer or performance engineer. Profilers have traditionally been the weapon of choice for analyzing performance problems. However they lack important features required for performance analysis in todays complex application environments. dynaTrace Diagnostics provides a unique set of capabilities, which exceeds profilers by far allowing developers, architects and performance engineers to: Quickly isolate performance outliers within a population of thousands of transactions Diagnose the root cause of performance problems immediately by analyzing the exact transaction flow and contextual information Improve team communication by cooperating and coordinating via a centralized dynaTrace Diagnostic Server Diagnose performance problems in 7x24 production environments Diagnose performance problems in heterogeneous enterprise environments Trace live transactions across server boundaries to support even large clustered distributed application environments

dynaTrace Diagnostics PurePath Technology provides the capabilities required for in-depth performance analysis in todays complex heterogeneous applications making it the first choice for solving performance problems

We invite you to learn more at http://www.dynaTrace.com .

On the PurePath Go Beyond Profiling

10

Headquarters Headquarters EMEA: dynaTrace software GmbH Freistdter Str. 313, 4040 Linz, Austria/Europe, T+ 43 (732) 908208, F +43 (732) 210100.008 Headquarters Headquarters North America: dynaTrace software software Inc. West Street 200, Waltham, MA 02451, USA, T +1 (339) 9330317 F +1 (781) 2075365 E: takeaction@dynatrace.com All rights reserved dynaTrace is a registered of dynaTrace software GmbH. All other marks and names On software the PurePath Go trademark Beyond Profiling mentioned herein may be trademarks of other respective companies. (071107)

11