Tomcat Architecture & Features

Ravi Saraswathi
August 25, 2009

What we will cover

Tomcat Architecture
Server topology Service Engine Connectors HTTP & AJP (and NIO) Host Valves Jasper Life Cycle Events Basic Administration Tomcat Listeners

Java Memory management

JVM Parameters and guidelines Performance tuning strategies with examples JVM Utilities

What we will cover (Continue..)

Tomcat Connectors HTTP, AJP Connector Options Load Balancing mod_JK & mod_proxy JSP and Servlet Servlet Life Cycle Context Events & Filter Tomcat Configuration Files Server.Xml Context.xml Class Loaders and Class path Bootstrap System Common & Shared , WebappX Web application structure and configuration Web apps directory structure Deployment descriptors (Web.XML)

What we will cover (Continue)

Integration of Tomcat and Apache mod_jk mod_proxy JNDI JNDI API & SPI LDAP and Initial Context

System Architecture Non-Dynapub Web application architecture Apache-Tomcat connection (Active-Active, Active-Passive)

Tomcat Architecture

Basics

Tomcat is a Web container from Apache open

source foundation.

It implements Servlet and JSP specification from

Sun Micro system

Tomcat is made up of pluggable components that

fit together in a nested manner.

Tomcat Server Topology

Tomcat The Server
Services Engine (Catalina) Hosts
AJP Connector 8009 SSL Connector 8443 HTTP Connector 8080
Valves Valves Valves Realm

Context JSPs Servlets

Architecture Basics
XML Representation One Server - One JVM Service/Engine/Host/Context
All are Containers All implement Lifecycle Listeners

of Tomcat Containers

<Server>
<Service> <Connector /> <Engine> <Host> <Context> </Context> </Host> </Engine> </Service> </Server>

Lifecycle Events
How objects get initialized, started & stopped

Object relationships are established during creation (digester parsing) Digester : XML-to-Java-object mapping

Overview of the major Tomcat components

Service represents grouping of connectors Connector for protocol handling Engine for request processing Host for virtual host handling Context for web application execution

Tomcat Server Architecture

Source: http://www.vsj.co.uk/java/display.asp?id=319

Server Configuration File

Server.xml
Main configuration file Builds server on the fly Parsed using commons-digester Tomcat has hard coded rule sets for the parsing Every element/component is swappable Define Server and Shutdown port Define Connectors Setup Virtual hosts Cannot be reloaded without restarting Tomcat

Entire server.xml parsed based on rules. Look for these rules:

Catalina.java org/apache/catalina/startup/

web.xml also is parsed using the digester

Relationship between the components

UML Associations: 1..* One or more 0..1 Zero or one 1 One only 0..* Zero or more

Realm

For Container Managed Security for Web apps (<security-constraint> elements, and
a <login-config> element in web.xml)

Database of username, Passwords & Roles Realm can be defined in Engine, Host or Context
JDBCRealm DataSourceRealm JNDIRealm MemoryRealm (tomcat-users.xml) JAASRealm UserDatabaseRealm

Architecture Basics

Copied from the book "Professional Apache Tomcat 5"

context.xml

The Context element represents a web application Process each HTTP request based on the context path Each Context MUST have a unique context path. META-INF/context.xml has all the resources E.g.. Database Can use for configuring web application class loader, session manager or Realm (database of users & roles).

Tomcat API (http://tomcat.apache.org/tomcat-5.5-doc/catalina/docs/api/index.html)

Valve

Tomcat Server Topology

Tomcat The Server
Services Engine (Catalina) Hosts Valves
AJP Connector 8009 SSL Connector 8443 HTTP Connector 8080
Realm

Context

Valves JSPs

Servlets

Valves

Architecture Basics - Valve Valve allows you to associate an instance of a Java class with a Engine/Host/Context container Component that can be inserted into the request processing pipeline Will preprocess all requests received by the connectors Similar concept to Servlet filter Interceptor Pattern Request processing pipeline Configured using a <Valve> element in the server.xml file

Architecture Basics - Valve

Can be associated with a Tomcat Engine, Host, or Context container

Architecture Basics Valve classes

Access Log Valve

It creates log files to track server access

The Remote Dump Filter

Dump the HTTP headers and cookies associated with the specified request <Valve className="org.apache.catalina.valves.RequestDumperValve"/>

Architecture Basics Valve classes

The Remote Host Filter
Compare the remote host address of the client
<Valve className="org.apache.catalina.valves.RemoteHostValve deny="virtuas*"/>

The Remote Address Filter

Compare the IP address of requesting client
<Valve className="org.apache.catalina.valves.RemoteAddrValve" deny="127.*"/>

Custom Valve

Create custom valve

Extends Valvebase or Implements Valve Interface

Register in Server.xml

Tomcat Listeners

Tomcat Listeners
Java Code modules that Performs actions when events occur
Just before startup During startup Just after startup Just before stop During stop Just after stop

Can be configured for a Server, Engine, Host or Context. Defined in Server.xml

Tomcat Default Listeners APR Lifecycle Listener Checks for the presence of the APR/native library and loads the library if it is present Jasper Listener Initializes the Jasper 2 JSP engine before any web applications that may use it are loaded

Tomcat Default Listeners

Server Lifecycle Listener Initializes the MBeanServer for the MBeans that may be used to manage Tomcat via JMX. Global Resources Lifecycle Listener Initializes the Global JNDI resources defined in server.xml as part of the Global Resource element

Custom Listeners

Best way to add code/functionality to Tomcat Register listener in Server.xml

JVM
Tomcat produces very little GC !

Resources for JVM trouble shooting

Log files messages -verbose:gc, XX:+PrintGC, XX:+PrintGCDetails,XX:+PrintGCTimeStamps, -XX: +HeapDumpOnOutOfMemoryError Java VisualVM Available from JDK 6. Federates several existing tools, including JConsole, jstat, jinfo, jstack, and jmap Other Tools Argus, Jprobe, Jconsole, HPROF - Heap Profiler, Eclipse Memory Analyzer, IBM Heap & thread analyzer JVM Utilities Jps, Jmap, Jinfo, Jhat, Jstack Unix Tools Top, vmstat, ulimit a , sar r , mpstat, netstat

GC

Java objects

Minor GC

Long Live objects

Full GC

Young generation
New space, when you create an object memory here is allocated.

Tenured generation
If it last longer than a while it is moved here

Permanent generation
Hold data needed by VM to describe objects (describing classes & Methods)

Typical JVM Parameters

-Xms1024m \ - Minimum Heap -Xmx1024m \ - Maximum Heap -Xss512k \ - Set maximum native stack size for any thread -XX:+UseConcMarkSweepGC -XX:GCTimeRatio=99 \ - The ratio of GC time to application time -XX:MaxGCPauseMillis=20 \ - Pause times of 20ms desired. -XX:PermSize=512m \ - Permanent Size -XX:MaxPermSize=512m \ -XX:NewSize=384m \ - Minor GC (Young Generation) -XX:MaxNewSize=384m \ -XX:SurvivorRatio=6 \

Java Virtual Machine and Memory issues

JVM Parameters -XX:NewSize=128m (Young Generation -Min) -XX:MaxNewSize=128m (Young Generation-Max) -Xms512m & -Xmx512m High means - Less often Minor collection occur Guidelines Set -XX:NewSize to be one-fourth the size of the max heap size. Larger Young Generation means smaller Tenured Set min heap (-Xms) equal to the max heap size (-Xmx) to minimize garbage collections.

60 % of available memory

Very High Heap size

Full GC is slower & GC Occurs less frequently.

Setting too high can cause wasted memory. 32-bit OS cap the heap size at between 1.5 and 2.5GB

Collectors

Mark and sweep Minimizes pause times, avoid "stopping the world Collect the tenured generation concurrently with the execution of the application Start VM - mark each object Remove -XX:+UseConcMarkSweepGC

Throughput collector Parallel Collector Distributes GC load across CPUs. Maximize throughput XX:+UseParallelGC Can be used for Multiprocessor Machines (with four or more processors)

JVM Tuning (-verbosegc -XX:+PrintGCDetails)

Young generations is too small (Adjust NewSize and MaxNewSize)
[GC [DefNew: 4032K->64K(4032K), 0.0429742 secs] 9350K->7748K(32704K), 0.0431096 secs]

Not Good: - Entire heap Decreased only by Good: - Young Generation Decreased by 3968K 1602k

This indicates that only about 40% objects in the young generation were garbage and the rest survive the collection and are being promoted into the old generation.

With an 8 Mbyte size, most of young generation is garbage at the time of the minor collection.
[GC [DefNew: : 8128K->64K(8128K), 0.0453670 secs] 13000K->7427K(32704K), 0.0454906 secs] Good: - Young Generation Decreased Good: - Total heap is decreased by 5573k 8064k
With an 8 Byte Young Generation size, most of young generation is garbage at the time of the minor collection

JVM Trouble Shooting

Connectors

Tomcat Server Topology

Tomcat The Server
Services Engine (Catalina) Hosts
AJP Connector 8009 SSL Connector 8443 HTTP Connector 8080
Valves Valves Valves Realm

Context JSPs Servlets

Tomcat Connector Architecture

A particular instance of this component listens for connections on a specific TCP port number on the server.

Catalin a Coyote HTTP Connector webap p Servlet Servlet Apache httpd Coyote AJP Connector context war file separate class loader

webap p Servlet Servlet

Load Balancing With mod_jk

round-robin, or weighted round-robin

Connectors

CoyoteAdapter.java - Creates Request/Response objects HTTP Connector protocol=

o.a.coyote.http11.Http11Protocol o.a.coyote.http11.Http11AprProtocol o.a.coyote.http11.Http11NioProtocol HTTP/1.1 aliased to Http11 and Http11Apr

AJP Connector mod_jk (Apache Jserv Protocol)

org.apache.jk.server.JkCoyoteHandler org.apache.coyote.ajp.AjpAprProtocol AJP/1.3 aliased to the two above

Connectors
NIO Allows a Servlet to process I/O asynchronously Available on Tomcat6 (Current Standard is Tomcat 5.5.27) NIO can be used to keep a connection open without wasting waiting threads. Tomcat 6 provide CometProcessor API to facilitate event-based model.

Communication between Connector and Apache modules

From Book Apache Tomcat Professional

Configure mod_proxy
Httpd.conf
Consumer

1. Add mod_proxy modules 2. Add ProxyPass Tomcat URL

Apache Host -1

Apache Host -2

Tomcat Host -3

Tomcat Host -4 Server.xml

Databas e

Define Connector options

Configure mod_jk
Httpd.conf
Consumer

1. Add mod_jk modules 2. Add workers. properties 3. Add the URL pattern (JkMount /myproject/* worker name

Apache Host -1

Apache Host -2

Tomcat Host -3

Tomcat Host -4 Server.xml

Databas e

Define Connector options

Mod_JK vs Mod_Proxy
Mod_jk Mod_Proxy

Mod_JK = AJP Connector Web server plug-in/ Tomcat Redirector Supports load balancing

Connectors available for FTP, HTTP, AJP or HTTPS mod_proxy, mod_proxy_balancer, mod_proxy_http, and mod_proxy_ajp Easy to Configure

Native mod_jk download available for different OS. Very Fast Complex Configuration Let Apache serve static pages

Mod_proxy modules comes as part of apache Lacks sticky session load balancing. mod_proxy_ajp supports load balancing

HTTP Connector Options MaxThreads Maximum number of simultaneous requests To Increase CPU usage Increase (from 150) MinSpareThreads Number of threads created at start time MaxSpareThreads Unused threads allowed to exist

HTTP Connector Options

AcceptCount Max incoming connections Increase to avoid connection refused errors Connectiontimeout Number of milliseconds wait for the request URL to be presented

Tomcat: The Definitive Guide, Jason Brittain (OReilley 2007)

Class loaders

Class loaders

Find Class files Parent Child relation ship Delegate the request to parent class loader

Class loaders

Tomcat 5.5

Tomcat 6.0

Class loaders

1. Bootstrap JVM Run time classes 2. System Load classes to initialize Tomcat - $CATALINA_HOME/bin 3. Common $CATALINA_HOME/lib, as well as classes and resources in JAR files. Preferably no Application Classes

Class loaders

4. Catalina Tomcat classes $CATALINA_HOME/server/classes & lib 5. Shared Put classes and resources that you wish to share across ALL web applications - $CATALINA_BASE/shared 6. WebAppX /WEB-INF/classes and /WEB-INF/lib

Deployment

Tomcat Directory Structure - Basics

Web Application Deployment - Options

HCM Console Copy the files to webapps folder HCM Composition Packages
From ADM Host - ssh root@<hostname> hcmbcs --sync --profile=<profile_id>)

Use Maven or Ant Copy to a Shortcut or Symbolic Link (webapps) Use the -d Option of javac or IDE

Web apps - Deployment Sequence

Any Context Descriptors will be deployed first.

Exploded web applications

.war files will be deployed /ROOT is hardcoded for path= Runtime deployment triggered by Lifecycle Event
Main programs are HostConfig.java & StandardHost.java

Web Application Structure - Basics

Deployment Descriptor

ROOT: The welcome screen application. Location

webapps/WEB-INF/web.xml Ignore conf/web.xml !! (Tomcat internal specific)

Tomcat monitor web.xml and reload webapps when web.xml changed

Deployment Descriptor Web.xml

<web-app version="2.5" . </web-app>

Works in Tomcat 6 Supports Servlets 2.5 vs. 2.4 or JSP 2.1 vs. JSP 2.0

Deployment Descriptor Web.xml

<web-app version="2.4 . </web-app>

Works in Tomcat 5 and 6 For JSP expression language 2.4 or later required.

Maven
Maven manage a project's build, reporting and documentation All the information required for the build is contained in a single POM, Project Object Model, XML file The developer provides information about what is being built and not how The POM contains detailed METADATA information about the project:
Versioning and configuration management Dependencies Project structure Application and testing resources

Maven - POM

Apache Tomcat

Apache- Tomcat Architecture Scenarios Non Dynapub

Load balancing for non-clustered applications. No failover strategy. Application may not be available incase of one Tomcat server down Apache & Tomcat installed in Same Server

Apache Host -1

Apache Host -1 Apache & Tomcat Host -1 Apache & Tomcat Host -2

Tomcat Host -1

Tomcat Host -2

Databas e

Databas e

Apache Tomcat Architecture Scenarios Non Dynapub

Consumer Consumer

Apache Host -1

Apache Host -2

Apache Host -1

Apache Host -2

Tomcat Host -1 Tomcat Host -1 Tomcat Host -2

Tomcat Host -2

Databas e

Databas e

Apache - Tomcat Architecture Scenarios: Non-Dynapub Systems

LB GSLB LB Apac he Host -1 JSP/Serv lets To To mca mca t t Hos Hos t -1 JDBC t -2 Apac he Host 2 / ajp13 / https http Apac he Host -3 LB Apac he Host http -4 / ajp13 / https

To mca t Hos JDBC t -3

To mca t Hos t -4

Databas e

Tomcat JNDI

JNDI Java Naming and Directory Interface

Provide access to naming and directory service (LDAP) from JAVA Classes SPI describes how to JNDI enable any naming and directory service

JNDI Java Naming and Directory Interface

Common Services are LDAP DNS Filesystems NIS etc

Resources stored in hierarchical order (Various Contexts)

JNDI Java Naming and Directory Interface

Operations Track Ver. 3/Course Title Ver.# /AOL LLC Privileged and Confidential Copied from book Professional Apache Tomcat 5

JNDI Java Naming and Directory Interface

Tomcat intercepts the JNDI requests from Webapps.

The GlobalNamingResources element - defines the global JNDI resources for the Server Per-web-application Initial Context can be configured in context.xml (META-INF) Reference the JNDI resources in web.xml

JNDI Java Naming and Directory Interface

Servlet

Operations Track Ver. 3/Course Title Ver.# /AOL LLC Privileged and Confidential

Servlet - Basics

Servlet is a program that generates a response upon a HTTP request Multi-threaded

means that one Servlet is capable of handling many requests

Servlet containers usually manage concurrent requests by creating a new Java thread for each request.

By using Servlets (Basics)

JspServlet.java - handles all JSP files Request response model Read form data Read HTTP request headers Set HTTP Status codes and response headers Use cookies and session tracking Share data among Servlets.

Servlet Life Cycle - Basics

Servlet methods - Basics

Init
Servlet loaded in to memory using the init call

Service
For each incoming request a service() is called

Destroy
Called once Servlet is garbage collected from the memory

Servlet Life Cycle - Basics

Operations Track Ver. 3/Course Title Ver.# /AOL LLC Privileged and Confidential

Deployment Descriptor Web.xml

context-param - Servlet context initialization parameters.
<param-name> and <param-value>. Access in code using javax.servlet.ServletContext. getInitParameter() Environment details specified for entire web app

Filter
Change client request/response before/after it goes to Servlet. Within filter-mapping Element define the <url-pattern>

Listener
Servlet Context events get notified when web apps deployed /shutdown etc HTTP Session Events get notified with session got activated etc

Operations Track Ver. 3/Course Title Ver.# /AOL LLC Privileged and Confidential

Deployment Descriptor Web.xml

Init-parm
Contains a name/value pair as an initialization parameter of the servlet
Access these parameters with the javax.servlet.ServletConfig. getInitParameter() method.

<security-constraint>
Defines the access privileges to a collection of resources defined by the <web-resource-collection> element.

<login-config> configure how the user is authenticated; the realm name that should be
used for this application; and the attributes that are needed by the form login mechanism

web-resource-collection - The <url-pattern> can refer to a directory or

filename
Operations Track Ver. 3/Course Title Ver.# /AOL LLC Privileged and Confidential

JMX

Operations Track Ver. 3/Course Title Ver.# /AOL LLC Privileged and Confidential

MBeans Usage Get and set applications configuration (Pull) Collect statistics (Pull) Performance Resources usage Problems Notify events (Push) Faults State changes

JMX Architecture

What are JMX and MBeans

Components:
Managed (Instrumented) Resources Agents (and their services) Connectors / Adapters

Goals:
Instrument Java Code Create Smart Java Agents Implement Management Middleware Integrate with Industry Standard Management Platforms

What are JMX and MBeans

AOL JMX tool ARGUS Config

MBeans can be interacted with programmatically or via JConsole/VisualVM MBeans have: Attributes that you can get or set Operations that you can invoke

Questions ?

Operations Track Ver. 3/Course Title Ver.# /AOL LLC Privileged and Confidential