Anda di halaman 1dari 68

CONTROL-M on UNIX Clusters

Implementation Manual

Supporting
CONTROL-M/Enterprise Manager version 6.2.01 with Fix Pack 1 CONTROL-M for Distributed Systems version 6.2.01 CONTROL-M/Server version 6.2.01 CONTROL-M/Agent for AIX version 6.2.01 CONTROL-M/Agent for HP-UX version 6.2.01 CONTROL-M/Agent for Solaris version 6.2.01 BMC Batch Impact Manager 6.2.01 CONTROL-M/Forecast 6.2.01

September 15, 2005

Contacting BMC Software


You can access the BMC Software website at http://www.bmc.com. From this website, you can obtain information about the company, its products, corporate offices, special events, and career opportunities.

United States and Canada


Address BMC SOFTWARE INC 2101 CITYWEST BLVD HOUSTON TX 77042-2827 USA Telephone 713 918 8800 or 800 841 2031 Fax 713 918 8000

Outside United States and Canada


Telephone (01) 713 918 8800 Fax (01) 713 918 8000

Copyright 2005 BMC Software, Inc., as an unpublished work. All rights reserved. BMC Software, the BMC Software logos, and all other BMC Software product or service names are registered trademarks or trademarks of BMC Software, Inc. IBM is a registered trademark of International Business Machines Corporation. Oracle is a registered trademark, and the Oracle product names are registered trademarks or trademarks of Oracle Corporation. All other trademarks belong to their respective companies. BMC Software considers information included in this documentation to be proprietary and confidential. Your use of this information is subject to the terms and conditions of the applicable End User License Agreement for the product and the proprietary and restricted rights notices included in this documentation.

Restricted rights legend


U.S. Government Restricted Rights to Computer Software. UNPUBLISHED -- RIGHTS RESERVED UNDER THE COPYRIGHT LAWS OF THE UNITED STATES. Use, duplication, or disclosure of any data and computer software by the U.S. Government is subject to restrictions, as applicable, set forth in FAR Section 52.227-14, DFARS 252.227-7013, DFARS 252.227-7014, DFARS 252.227-7015, and DFARS 252.227-7025, as amended from time to time. Contractor/Manufacturer is BMC SOFTWARE INC, 2101 CITYWEST BLVD, HOUSTON TX 77042-2827, USA. Any contract notices should be sent to this address.

Customer support
You can obtain technical support by using the Support page on the BMC Software website or by contacting Customer Support by telephone or e-mail. To expedite your inquiry, please see Before Contacting BMC Software.

Support website
You can obtain technical support from BMC Software 24 hours a day, 7 days a week at http://www.bmc.com/support_home. From this website, you can

read overviews about support services and programs that BMC Software offers find the most current information about BMC Software products search a database for problems similar to yours and possible solutions order or download product documentation report a problem or ask a question subscribe to receive e-mail notices when new product versions are released find worldwide BMC Software support center locations and contact information, including e-mail addresses, fax numbers, and telephone numbers

Support by telephone or e-mail


In the United States and Canada, if you need technical support and do not have access to the web, call 800 537 1813 or send an e-mail message to support@bmc.com. Outside the United States and Canada, contact your local support center for assistance.

Before contacting BMC Software


Before you contact BMC Software, have the following information available so that Customer Support can begin working on your problem immediately:

product information product name product version (release number) license number and password (trial or permanent)

operating system and environment information machine type operating system type, version, and service pack or other maintenance level such as PUT or PTF system hardware configuration serial numbers related software (database, application, and communication) including type, version, and service pack or maintenance level

sequence of events leading to the problem commands and options that you used messages received (and the time and date that you received them) product error messages messages from the operating system, such as file system full messages from related software

CONTROL-M on UNIX Clusters Implementation Manual

Contents
About this book 11 Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Syntax statements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Chapter 1 UNIX cluster architecture 15 16 17 17 18 18 19 19 20 21 22 22 25 25 25 26 26 27 27 28 28 28 29 30 30 30 33

Cluster network connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cluster storage media . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sharing media between multiple nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Application failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cluster resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Resource groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Monitoring clustered applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Quorum device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . UNIX cluster products. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Naming conventions in this document. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 2 CONTROL-M database configuration

Phase 1: Plan the database configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using a dedicated or existing CONTROL-M database . . . . . . . . . . . . . . . . . . . . . . Using the same or different file systems for database software and data files . . Using the same or different resource group for database software and CONTROL-M . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Making additional database preference choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . Phase 2: Install the CONTROL-M database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Phase 3: Modify the database network configuration files . . . . . . . . . . . . . . . . . . . . . . Sybase (interfaces file). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Oracle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Phase 4: Configure the policy file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Phase 5: Monitor CONTROL-M database processes . . . . . . . . . . . . . . . . . . . . . . . . . . . Oracle processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sybase processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 3 CONTROL-M/Enterprise Manager Installation

Phase 1: Create CONTROL-M/EM UNIX accounts . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Phase 2: Install CONTROL-M/EM server components . . . . . . . . . . . . . . . . . . . . . . . . . 34

Contents

Phase 3: Reconfigure TAO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Phase 4: Managing CONTROL-M/EM server components . . . . . . . . . . . . . . . . . . . . . 38 Managing CONTROL-M/EM components using the Administration facility . . 39 Letting the cluster manage CONTROL-M/EM components . . . . . . . . . . . . . . . . . 39 Phase 5: Global Condition Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Phase 6: Install and log onto the CONTROL-M/EM client components (Windows) 40 Phase 7: Configure the policy file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Administration facility administration mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Cluster administration mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Phase 8: Install BMC Batch Impact Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Phase 9: Install CONTROL-M/Forecast . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Phase 10: Monitor CONTROL-M/EM processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Phase 11: Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 The Administration Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 The Global Condition Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Chapter 4 CONTROL-M/Server Installation 53

Phase 1: Create CONTROL-M/Server UNIX accounts . . . . . . . . . . . . . . . . . . . . . . . . . 53 Phase 2: Install CONTROL-M/Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Phase 3: Configure the policy file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Phase 4: Monitor CONTROL-M/Server processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Chapter 5 CONTROL-M/Agent Installation 57

Phase 1: Plan CONTROL-M/Agent configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 Phase 2: Create CONTROL-M/Agent UNIX accounts . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Phase 3: Install CONTROL-M/Agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Job disappeared reasons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Job disappeared workaround. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Phase 4: Configure the policy file. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Phase 5: Monitor CONTROL-M/Agent processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Appendix A Index Example of installing Apache Tomcat on UNIX clusters 63 65

CONTROL-M on UNIX Clusters Implementation Manual

Figures
Cluster computer architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Typical two-node cluster architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Domain Settings wizard page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Naming Service wizard page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ports wizard page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary wizard page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Logging in to CONTROL-M/EM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Options menu reconnection panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 17 36 36 37 38 41 42

Figures

CONTROL-M on UNIX Clusters Implementation Manual

Tables
IT company web sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Publication web sites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Available UNIX cluster products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Naming conventions for CONTROL-M components on UNIX clusters . . . . . . . . . . Oracle processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sybase processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CONTROL-M/EM processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . CONTROL-M/Agent processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 12 22 22 30 30 49 61

Tables

10

CONTROL-M on UNIX Clusters Implementation Manual

About this book


This document contains detailed information about how to implement CONTROL-M on UNIX clusters. It is intended for system administrators and database administrators (DBAs). A relatively high level of understanding of CONTROL-M architecture and internals, as well as cluster administration, is needed in order to implement the content of this document. Therefore, you should consider contacting BMC Software personnel to assist you while you carry out these instructions. Like most BMC Software documentation, this book is available in online format. Visit the BMC Software Customer Support page at http://www.bmc.com/support_home to request additional printed books or to view online books and notices (such as release notes and technical bulletins). Some product shipments also include the online books on a documentation CD.

NOTE
Online books are formatted as Portable Document Format (PDF) or HTML files. To view, print, or copy PDF books, use the free Adobe Reader from Adobe Systems. If your product installation does not install the reader, you can obtain the reader at http://www.adobe.com.

The software also offers online Help. To access Help, press F1 within any product, or click the Help button in graphical user interfaces (GUIs).

About this book

11

Related publications

Related publications
Table 1 provides links to the web sites of some IT companies, where you can find material to supplement the information contained in this document. Table 2 provides similar information from other sources. Table 1
URL
http://docs.hp.com/hpux/ha/ http://www-1.ibm.com/servers/eserver/clusters/library/ http://docs.sun.com/ http://wwws.sun.com/software/cluster

IT company web sites


Comments
HP documentation web site IBM documentation web site Sun Microsystems documentation web site Search for Cluster, mark Search book titles only Veritas support web site Select product Family: Clustering Technology

http://support.veritas.com/

Table 2
URL

Publication web sites


Comments
Cluster Computing at a Glance, by Mark Baker and Rajkumar Buyya New Architect Magazine

http://www.buyya.com/cluster/v1chap1.pdf http://www.newarchitectmag.com/

Conventions
This book uses the following special conventions:

All syntax, operating system terms, and literal examples are presented in this typeface. Variable text in path names, system messages, or syntax is displayed as italics.

12

CONTROL-M on UNIX Clusters Implementation Manual

Syntax statements

Syntax statements
The following example shows a sample syntax statement:
COMMAND KEYWORD1 [KEYWORD2 | KEYWORD3] KEYWORD4={YES | NO} fileName. . .

The following table explains conventions for syntax statements and provides examples:
Item Items in italics represent variables that you must replace with a name or value. If a variable is represented by two or more words, an uppercase letter distinguishes the second and subsequent words. Brackets indicate a group of optional items. Do not type the brackets when you enter the option. A comma means that you can choose one or more of the listed options. You must use a comma to separate the options if you choose more than one option. Braces indicate that at least one of the enclosed items is required. Do not type the braces when you enter the item. A vertical bar means that you can choose only one of the listed items. In the example, you would choose either commit or cancel. An ellipsis indicates that you can repeat the previous item or items as many times as necessary. Example alias databaseDirectory serverHostName [-FULL, -INCREMENTAL, -LEVEL]

{-A | -C}

{-COMMIT | -CANCEL}

COLUMN_NAME . . .

The following conventions apply to variables in syntax statements:

Variables typically are displayed in uppercase letters and are placed between angle brackets. If a variable is represented by two or more words, underscores distinguish the second and subsequent words (for example, databaseName).

About this book

13

Syntax statements

14

CONTROL-M on UNIX Clusters Implementation Manual

Chapter

UNIX cluster architecture


A cluster is a parallel or distributed processing system that consists of a collection of interconnected stand-alone computers working together as a single, integrated computing resource. A cluster generally has two or more computer nodes connected together. The nodes can exist in a single cabinet or they can be physically separated and connected using a LAN. An interconnected (LAN-based) cluster of computers can appear as a single system to users and applications. Such a system can provide a cost-effective way to gain features and benefits, such as fast and reliable service. To understand this concept, the cluster services can be considered as a layer between the application (CONTROL-M) and the computer (node) operating system and hardware. Typical cluster architecture is shown in Figure 1 Figure 1
Cluster computer architecture

Chapter 1 UNIX cluster architecture

15

Cluster network connections

For the cluster layer to be transparent to the application, a virtual (logical) hostname is defined for each application running on the cluster. This way, a client connecting to the application does not need to know the hostname of the node on which the application is currently running. Although the application runs on a specific node in the cluster, the cluster layer translates the virtual hostname to the name of the node where the application is currently running. This node is called the active node.

Cluster network connections


Each node in a cluster is typically configured with both private network and public network addresses.

Private network addresses are used for node-to-node communications; that is, transferring cluster configuration and administration data between the nodes in the cluster. This is also referred to as an internal network, transport or interconnection. Public network addresses are used for client-to-cluster communications; that is, transferring data between the applications that are configured on the cluster and the application clients. This is also referred to as an external network.

To ensure that both interfaces are highly available, each network interface card is usually included in each node in the cluster. For example, in a two-node cluster, at least four network interfaces are usually installed. Figure 2 illustrates typical two-node cluster architecture.

16

CONTROL-M on UNIX Clusters Implementation Manual

Cluster storage media

Figure 2

Typical two-node cluster architecture

Cluster storage media


The active node and all backup nodes in the cluster are physically connected to shared storage media that can be mounted to any node in the cluster. The file system used by the application is generally mounted only to the active node.

Sharing media between multiple nodes


There are several vendors that provide, with their cluster solutions, the functionality that enables you to access shared media from multiple nodes at the same time. An example is Sun Microsystems, with Sun Cluster 3.0 or later. In Sun Cluster this feature is called Global Devices CONTROL-M provides limited support for global devices. Therefore, the file system where CONTROL-M is installed must not be defined as a global file system.

Chapter 1 UNIX cluster architecture

17

Application failover

Application failover
If the application fails on the active node, or if the entire active node fails, the application can be restarted on an alternative node. The alternative nodes in the cluster are called backup nodes. The process of restarting the application on a backup node, whether done manually or automatically by the cluster, is called failover or switch. The failover process usually involves the following steps:

1 The application is shut down on the active node. 2 The file system is dismounted from the active node and mounted on a backup
node.

3 The virtual hostname now refers to the backup node. 4 The application is brought up on the backup node, which then becomes the new
active node.

Cluster resources
Resource refers to each hardware or software component within the cluster that is managed by the cluster service. Resources are physical or logical entities. Hardware devices such as disk drives and network interfaces are examples of physical cluster resources. Internet Protocol (IP) addresses, applications, and application databases are example of logical cluster resources. Each node in the cluster has its own local resources. However, the nodes in a cluster also share common resources (such as a common data storage array and private cluster network) that are accessible by each node in the cluster. Resources have the following characteristics. They can be

brought online and taken offline managed by a cluster service owned by only one node at a time

18

CONTROL-M on UNIX Clusters Implementation Manual

Resource groups

The following are examples of common cluster resources:


network name (virtual) IP address file system shell script that starts and/or stops an application

A resource is online when it is available and providing service to the cluster.

Resource groups
A resource group is a collection of resources managed by the cluster software as a single, logical unit. Application resources and cluster entities can be more easily managed when logically related resources are defined into a resource group. A cluster software operation performed on a resource group can influence all individual resources in the group. Typically, a resource group is defined to contain all the elements needed by a specific application server and clients for successful use of the application. Resource groups have the following characteristics:

A group is always owned by a single node A resource is always owned by a single group If one resource in the group fails the entire group will be failed over (switched) to an alternative node.

Every group maintains a prioritized list of the nodes that can and should act as its host. If the cluster is composed of more than two nodes, the list is used to prioritize failover points for the application.

Virtual servers
Applications running on a server cluster can appear to users and workstations as running on virtual servers. A virtual server is a resource group that contains at least one Network Name resource and one IP Address resource, in addition to the specific application resources. Multiple virtual servers representing multiple applications can be hosted in a cluster.

Chapter 1 UNIX cluster architecture

19

Monitoring clustered applications

The user or client software that connects to an application that is running in a virtual server does not know which node is actually hosting the virtual server. To users and clients, connecting to an application or service that is running as a clustered virtual server appears no different from connecting to a single, physical server. Services or applications not accessed by user or client applications can run on a cluster node without being managed as a virtual server. In the event of an application or server failure, the cluster service moves the entire virtual server resource group to another node in the cluster. When such a failure occurs, the client detects a failure in its session with the application and attempts to reconnect in exactly the same manner as the original connection. The reconnection is possible because, during recovery operations, the cluster service simply maps the published IP address of the virtual server to a surviving node in the cluster. The client session can reestablish the connection to the application without needing to know that the application is now physically hosted on a different node in the cluster. Each resource in a group may depend on other resources in the cluster. Dependencies are relationships between resources that indicate which resources need to be started and available before another resource can be started. For example, a database application may depend on the availability of a disk, IP address, and network name to be able to start and provide services to other applications and clients. Resource dependencies are identified by using cluster resource properties. They enable the cluster service to control the order in which resources are brought online and offline. The scope of any identified dependency is limited to resources within the same resource group. Most cluster-managed dependencies cannot extend beyond the resource group, because resource groups can be brought online and offline and moved independently. Only some cluster software provides the ability to create dependencies between resource groups.

Monitoring clustered applications


Failover always starts with the detection and identification of the failure condition. It can occur at the hardware level, operating system level, or application level. Failures can be caused by reasons such as a

memory problem system overload database server crash

Failure detection and identification can be performed manually or automatically. The monitoring of the cluster application can be performed by the cluster management software (internally by the cluster) or by a third party application, such as frameworks. The processes and services of each product that should be monitored are described later in this document.
20 CONTROL-M on UNIX Clusters Implementation Manual

Quorum device

Methods for monitoring applications that run on a cluster differ from one vendor to another. Monitoring is usually implemented by running a shell script, a C-program, or a binary that contains procedures to monitor resources and contains policies (rules). These policies determine when and how to start and stop the monitored application and how to perform corrective actions to regain application availability upon failure. Most cluster vendors have different names for those procedures. In this document those procedures are referred to as policy files. The policy file that will control the availability of CONTROL-M should be created according to this document.

Quorum device
Because cluster nodes share data and resources, the cluster must take steps to maintain data and resource integrity. The concept of quorum voting controls cluster membership and whether the cluster is considered up. This concept is explained below. Each node in the cluster is assigned one vote. To form a cluster and for the cluster to remain up, a majority of votes must be available. For this concept to work in the cases where an equal number of votes are available and unavailable, an extra vote is assigned to a quorum device as a tie-breaker. For example, in a two-node cluster, if one node fails, the cluster can remain up because two of the three possible votes are up (one node and the quorum device itself). This mechanism is used in most cluster products, but not all. The quorum device can also be used to store cluster configuration information, and to synchronize configuration changes between the cluster nodes.

Chapter 1 UNIX cluster architecture

21

UNIX cluster products

UNIX cluster products


Table 3 describes some of the UNIX cluster products that are available. Table 3 Available UNIX cluster products

Product Specifications
Product: MC/ServiceGuard Vendor: HP OS Software: HP/UX

Terminology
Package: Set of applications and services that the cluster is managing. Package Control Script: Definitions used by the Package Manager to perform the failover process. Agent/Data Service: Callback methods that start, stop, and monitor the application. Can use C or shell commands Application Server: a cluster resource made highly available by the HACMP software. Resource Group: bundle application servers with the appropriate group resources Service Group: Set of resources that are required for one application to provide service. Failover Group: A type of service group. The cluster engine ensures that the Failure Group is always running.

Product: Sun Cluster Vendor: Sun OS Software: Solaris Product: HACMP Vendor: IBM OS Software: AIX Product: Veritas Cluster Server Vendor: Veritas Software OS Software: Various

There are other cluster products that are suitable for implementation with CONTROL-M. The descriptive terminology should be familiar to anyone interested in implementing CONTROL-M in a cluster environment.

Naming conventions in this document


Table 4 describes the naming conventions used in this document for installing CONTROL-M on a UNIX cluster. Table 4
Component

Naming conventions for CONTROL-M components on UNIX clusters (part 1 of 2)


Naming convention

physical nodes host names hostcla, hostclb CONTROL-M/EM virtual network name vhem620 CONTROL-M/EM home directory /export1/em620

22

CONTROL-M on UNIX Clusters Implementation Manual

Naming conventions in this document

Table 4
Component

Naming conventions for CONTROL-M components on UNIX clusters (part 2 of 2)


Naming convention

CONTROL-M/SERVER virtual network name CONTROL-M/SERVER home directory CONTROL-M/AGENT virtual network name CONTROL-M/AGENT home directory

vhctm620 /export2/ctm620
vhag620

in the example, in an existing file system

NOTE
This document describes installation on a two-node cluster, but the same principles apply to cluster implementations with a larger number of nodes.

Chapter 1 UNIX cluster architecture

23

Naming conventions in this document

24

CONTROL-M on UNIX Clusters Implementation Manual

Chapter

CONTROL-M database configuration


This chapter is comprised of the following phases, all designed to help you work with a CONTROL-M database on UNIX clusters:

Phase 1: Plan the database configuration Phase 2: Install the CONTROL-M database Phase 3: Modify the database network configuration files Phase 4: Configure the policy file Phase 5: Monitor CONTROL-M database processes

Phase 1: Plan the database configuration


Before starting the installation of the CONTROL-M database, you must plan how the CONTROL-M database will be implemented. The following alternatives should be taken into consideration. Among the things you should consider are whether to use

a dedicated or existing CONTROL-M database the same or different file systems for database software and data files the same or a different resource group for database software and CONTROL-M

Using a dedicated or existing CONTROL-M database


In dedicated installation mode, the CONTROL-M license requires that the software that will be used for the installation will be the software that is supplied by BMC Software. In this case, the administration of the database should be performed by the tools provided with CONTROL-M. In existing installation mode, the CONTROL-M database software license must be purchased directly from the database vendor. In this case, the administration of the CONTROL-M database should be performed by external tools.
Chapter 2 CONTROL-M database configuration 25

Using the same or different file systems for database software and data files

Using the same or different file systems for database software and data files
The CONTROL-M database software and data files can be located on the same file system as CONTROL-M, or on different file systems. Use the following guide line when choosing which file system your database software will be located on:

When implementing the CONTROL-M database in dedicated mode, the database software and CONTROL-M must always be running on the same node. When implementing CONTROL-M database in existing mode, the database software can run on the same node as CONTROL-M, or on a different node. In order for them to run on different nodes, a database client software instance must always be available for the CONTROL-M/Server, so it will be able to connect to the database server. When installing Oracle as the CONTROL-M database, using the USE_EXISTING_ORACLE environment variable, the client software can be either located on the same file system as CONTROL-M or on a different file system. If you are using different file systems, the file system on which the Oracle client is located must always be available to CONTROL-M, so it will be able to connect to a remote database server.

Using the same or different resource group for database software and CONTROL-M
The database software can (but need not) be implemented into the same resource group as CONTROL-M. Use the following guide line when choosing which resource group your database resources will be member of:

When the database software and CONTROL-M are implemented into the same resource group, they can either be installed on the same file system or on different file systems that are both members in the relevant resource group. When the implementation is done using a single resource group, the database software can be implemented in either a dedicated or existing mode. When the database software is implemented into a different resource group than CONTROL-M and the implementation is done in an existing mode (in this case the database software and CONTROL-M must be located on different file systems), the two resource groups can run on the same or different nodes.

26

CONTROL-M on UNIX Clusters Implementation Manual

Making additional database preference choices

When the database software is implemented into a different resource group than CONTROL-M and the implementation is done in a dedicated mode (in this case the database software and CONTROL-M should be located on different file systems), there must be a dependency between the two resource groups, so they will always be running on the same node.

Making additional database preference choices


CONTROL-M can be implemented on a UNIX cluster that uses a database platform that is configured with additional database high availability features such as Oracle Data Guard (formerly known as Oracle Standby Server), which enables replicating the Oracle database content to a remote location, or Oracle Real Application Cluster (formerly known as Oracle Parallel Server), which enables you to install multiple instances of Oracle to access the same database simultaneously. These features should be configured and administered by an experienced DBA. When using these solutions the CONTROL-M home directory path should be identical on the primary and standby hosts. Some cluster vendors supply policy files for popular applications, such as Oracle, Siebel, Sybase, SAP. Application vendors often certify these policy files. For the implementation of CONTROL-M, BMC Software recommends that you use certified policy files, if available, to monitor the CONTROL-M database. Sun Cluster data service for Oracle/Sybase and Veritas cluster Oracle agent for AIX, are examples of certified products.

Phase 2: Install the CONTROL-M database


Once the planning of the CONTROL-M database configuration is complete, the installation of the database software should be performed according to the CONTROL-M Installation Guide. Follow the relevant instructions according to the implementation method you chose in the previous phase. If you are installing the CONTROL-M databases in a dedicated mode, perform phase 1 of the relevant product components at this stage.

Chapter 2 CONTROL-M database configuration

27

Phase 3: Modify the database network configuration files

Phase 3: Modify the database network configuration files


To modify the database network configuration files, follow the instructions according to the database platform in use at your site. Sybase and Oracle instructions are shown below.

Sybase (interfaces file)


1 Use the following command to set the DISPLAY parameter to the local terminal
address (the IP address of the PC running the telnet/x-window session, or the IP address of the UNIX console that is being used):
setenv DISPLAY terminalAddress:0

2 Run the dsedit application. 3 Replace the CONTROL-M database server host name with the virtual host name of
the database resource group, for example, vhctm620 (CONTROL-M/Server) or vhem620 (CONTROL-M/EM)

4 Replace the SYB_BACKUP database server host name with the virtual host name
of the database resource group, for example, vhctm620 (CONTROL-M/Server) or vhem620 (CONTROL-M/EM)

Oracle
1 Ensure that the Oracle server is stopped and that none of the Oracle processes are
running.

2 Replace the host name with the virtual host name of the database resource group,
for example, vhctm620 (CONTROL-M/Server) or vhem620 (CONTROL-M/EM) in the following files:

$ORACLE_HOME/network/admin/tnsnames.ora $ORACLE_HOME/network/admin/listener.ora

28

CONTROL-M on UNIX Clusters Implementation Manual

Phase 4: Configure the policy file

Phase 4: Configure the policy file


When CONTROL-M/Server and CONTROL-M/EM share the same database server (or Oracle instance), the following actions may be required:

If CONTROL-M/Server is installed with a dedicated database, run the startdb script in the CONTROL-M/Server account but do not run the start_server script in the CONTROL-M/EM account. If CONTROL-M/EM is installed with a dedicated database, run the start_server script in the CONTROL-M/EM account but do not run the startdb script in the CONTROL-M/Server account.

The CONTROL-M database server can be shut down or started using the following scripts only if it is installed in a dedicated mode. If the cluster vendor has a certified database policy file, BMC Software recommends that you use that file. To start the CONTROL-M database, use the following commands:
# CONTROL-M/EM EM_ACC=em620 EM_DIR=/export1/em620 su - $EM_ACC -c $EM_DIR/scripts/start_server # CONTROL-M/Server CTM_ACC=ctm620 CTM_DIR=/export2/ctm620 su - $CTM_ACC -c $CTM_DIR/ctm/scripts/startdb

To stop the CONTROL-M database, use the following commands:


# CONTROL-M/EM EM_ACC=em620 EM_DIR=/export1/em620 su - $EM_ACC -c $EM_DIR/scripts/stop_server # CONTROL-M/Server CTM_ACC=ctm620 CTM_DIR=/export2/ctm620 su - $CTM_ACC -c $CTM_DIR/ctm/scripts/shutdb

Chapter 2 CONTROL-M database configuration

29

Phase 5: Monitor CONTROL-M database processes

Phase 5: Monitor CONTROL-M database processes


Oracle processes
Oracle architecture (beginning with version 8.0.x) is built on the mandatory processes shown in Table 5, and one optional process, called listener. The cluster application should monitor all these processes and should switch the database application if any of them fails. Failover action decisions should be made together with the database administrator. Table 5 Oracle processes
Process name ora_dbw0_instanceName ora_lgwr_instanceName ora_ckpt_instanceName ora_smon_instanceName ora_pmon_instanceName ora_recd_instanceName

Oracle process Database Writer Log Writer Checkpoint System Monitor Process Monitor Recoverer

Sybase processes
Sybase ASE (Adaptive Server Enterprise) software has two running processes. The first is the dataserver process, which functions as the database engine. The second is the backup server, which enables you to make online backups of Sybase data while transactions are running on the database. These processes are shown in Table 6: Table 6
Dataserver

Sybase processes
Process name HomeDir/sybase/platform/ASE-12_0/bin/dataserver -sdatabaseName-dHomeDir/database/master.dat -eHomeDir/log/error_log HomeDir/sybase/ASE-12_0/bin/backupserver -eHomeDir/log/backup.log -N25 -C20 -MHomeDir/sybase/platform/ASE-12_0/bin/sybmultbuf -SSYB_BACKUP

Sybase process

Backup

30

CONTROL-M on UNIX Clusters Implementation Manual

Sybase processes

The cluster should monitor both these processes and should switch the database package if either process fails. Failover decisions should be made together with the database administrator.

Chapter 2 CONTROL-M database configuration

31

Sybase processes

32

CONTROL-M on UNIX Clusters Implementation Manual

Chapter

CONTROL-M/Enterprise Manager Installation


3

This chapter is comprised of the following phases, all designed to help you install CONTROL-M/Enterprise Manager to work on UNIX clusters:

Phase 1: Create CONTROL-M/EM UNIX accounts Phase 2: Install CONTROL-M/EM server components Phase 3: Reconfigure TAO Phase 4: Managing CONTROL-M/EM server components Phase 5: Global Condition Server Phase 6: Install and log onto the CONTROL-M/EM client components (Windows) Phase 7: Configure the policy file Phase 8: Install BMC Batch Impact Manager Phase 9: Install CONTROL-M/Forecast Phase 10: Monitor CONTROL-M/EM processes Phase 11: Troubleshooting

Chapter 3

CONTROL-M/Enterprise Manager Installation

33

Phase 1: Create CONTROL-M/EM UNIX accounts

Phase 1: Create CONTROL-M/EM UNIX accounts


If the CONTROL-M/EM database is installed in a dedicated mode this phase should be performed before starting the database installation.

1 Create two user accounts as shown in the following example, one on each node. EXAMPLE
useradd -g controlm -s /bin/tcsh -m -d /export1/em620 em620

This command should be invoked by a user with administrative privileges

2 Both users must have identical names (referred to in the example as: em620) and
identical user IDs (UID).

3 Both users home directories should point to the same location on a shared disk
(referred to in the example as: /export1/em620).

Phase 2: Install CONTROL-M/EM server components


1 Before starting the installation ensure to run the prerequiste check script
(check_req) on each of the cluster nodes.

2 Install CONTROL-M/EM according to the instructions provided in the


CONTROL-M Installation Guide.

NOTE
As the home directory of the CONTROL-M/EM account is located on the shared disk, the CONTROL-M/EM software will be installed on the shared disk rather than on the local disk node.

3 Install the latest fix pack to apply the most recent software updates.

34

CONTROL-M on UNIX Clusters Implementation Manual

Phase 3: Reconfigure TAO

Phase 3: Reconfigure TAO


TAO is an implantation of the CORBA protocol that manages the communication between the CONTROL-M/EM client and server components. The Naming Service is the TAO component that manages the registered servers and their connection parameters. The following procedure configures the Naming Service to use (publish) the virtual host name of CONTROL-M/EM, so that client components can access the product without knowing which node it is running on.

To reconfigure TAO: 1 On the first node, hostcla, ensure the following servers are not running:

CONTROL-M/EM GUI Server Global Alerts Server BMC Batch Impact Manager Server (BIM) CONTROL-M/Forecast

2 Stop the CORBA Naming Service from the root_menu or by entering the following
command: stop_ns_daemon The following message is displayed: Naming Service was successfully stopped.

3 Set the JAVA_HOME environment variable to the JRE installation directory.


Ensure JRE version 1.4.x or later is used (for example: setenv JAVA_HOME /usr/java/jre).

4 Set the DISPLAY environment parameter by entering the following command:


setenv DISPLAY terminalAddress:0

5 Activate the Domain Configuration wizard by running the orbconfigure java


utility. The Domain Settings wizard page is displayed, as shown in Figure 3.

Chapter 3

CONTROL-M/Enterprise Manager Installation

35

Phase 3: Reconfigure TAO

Figure 3

Domain Settings wizard page

6 Modify the published address to virtual host name and enter the CONTROL-M/EM
virtual host name (as referred to in the example: vhem620) and then click Next. The Naming Service wizard page is displayed, as shown in Figure 4. Figure 4 Naming Service wizard page

7 Enter the CONTROL-M/EM virtual host name (as referred to in the example:
vhem620) and the relevant port (default: 13075), and then click Next.

36

CONTROL-M on UNIX Clusters Implementation Manual

Phase 3: Reconfigure TAO

Ensure that the port is available and is not used by other programs on any of the cluster nodes. The Ports wizard page is displayed, as shown in Figure 5. Figure 5 Ports wizard page

8 Set CONTROL-M/EM ports or leave default settings, and then click Next.
The Summary wizard page is displayed, as shown in Figure 6.

Chapter 3

CONTROL-M/Enterprise Manager Installation

37

Phase 4: Managing CONTROL-M/EM server components

Figure 6

Summary wizard page

9 Ensure all relevant entries in the summary reports refer to the virtual host name,
and then click Finish.

10 Start the TAO Naming Server by running the following command:


start_ns_daemon

Phase 4: Managing CONTROL-M/EM server components


Choose between the following methods of controlling and monitoring of the CONTROL-M/EM servers components:

Managing CONTROL-M/EM components using the Administration facility In the CONTROL-M/EM Administration facility, define each of the CONTROLM/EM server components and modify the server component host name during failover.

Letting the cluster manage CONTROL-M/EM components Do not use the CONTROL-M/EM Administration facility and Administration Agent. Instead, let the cluster start, stop, and monitor each of the server components.

38

CONTROL-M on UNIX Clusters Implementation Manual

Managing CONTROL-M/EM components using the Administration facility

Managing CONTROL-M/EM components using the Administration facility


When managing CONTROL-M/EM components by the Administration facility, each CONTROL-M/EM server component is defined in the Administration facility using the physical host name of the active node. During the failover process, before starting the CONTROL-M/EM Administration Agent, the host name of the components is modified in the CONTROL-M/EM database, to reflect the name of the new host. In the CONTROL-M/EM database, each component is registered in the CONFREG table. The MACHINE_NAME field in the CONFREG table refers to the name of the host the component is running on. Since this field only supports the use of a physical host name, the field values should be altered during the CONTROL-M/EM resource group failover to reflect the new active node host name.

NOTE
When defining the CONTROL-M/EM server components in the Administration facility, ensure to use the virtual host name (in the example, vhem620) as the component name for the servers that uses CORBA communication protocol. These components are the GUI Server and Global Alerts Server. This name differs from the machine name value which should be the physical host name of the active node.

Execute the database update every time the CONTROL-M/EM resource group is moved from one node to another, before starting the Administration Agent. For additional details, see Administration facility administration mode on page 43. For additional information about monitoring of the CONTROL-M/EM server components, and a complete list of the processes names, see Phase 10: Monitor CONTROL-M/EM processes on page 49.

Letting the cluster manage CONTROL-M/EM components


When managing CONTROL-M/EM components by the cluster, the cluster software is used to start, stop, and monitor the CONTROL-M/EM server components. In this administration mode, the CONTROL-M/EM server components should be defined as resources, according to the commands described in Cluster administration mode on page 44 in Phase 7: Configure the policy file.

NOTE
For viewing the processes states from the CONTROL-M/EM Administration facility, define all components in desire state ignored and start the Administration Agent.

Chapter 3

CONTROL-M/Enterprise Manager Installation

39

Phase 5: Global Condition Server

For additional information about monitoring of the CONTROL-M/EM server components, and a complete list of the processes names, see Phase 10: Monitor CONTROL-M/EM processes on page 49.

Phase 5: Global Condition Server


By default, the Global Condition Server (GCS) configuration file specifies the physical host name parameter of the active node. For the GCS to ignore this value, the parameter should be deleted from the GCS configuration file. The configuration file is located in the following path:
emHome/site/resource/Defaults.rsc

EXAMPLE
/export1/em620/site/resource/Defaults.rsc

Back up the Default.rsc file and delete the line that specifies the host name. Shown below is an example for the Default.rsc file. The last line in the file should be deleted.

EXAMPLE
# @(#) Tool.rsc #################################################################### #################################################################### LogDir * Name /export1/em620/log MpmDir * Name /export1/em620 Database * Name vhem620 GCS * hostname hostcla

Phase 6: Install and log onto the CONTROLM/EM client components (Windows)
1 Ensure that the virtual host name (in the example: vhem620) is properly defined in
the DNS or local hosts file so it can be accessed by name or IP address from Microsoft Windows.

2 During installation, enter the virtual host name as the location for the

GUI Server CORBA Naming Service Database Server Name (when Installing client administration components)

40

CONTROL-M on UNIX Clusters Implementation Manual

Phase 6: Install and log onto the CONTROL-M/EM client components (Windows)

3 When logging on to the CONTROL-M/EM GUI, as shown in Figure 7, use the


virtual host name configured by the orbconfigure utility (in Phase 3: Reconfigure TAO on page 35) to connect to the CORBA Naming repository. The virtual host name should be entered in the Host Name field, which is displayed after clicking Advanced. This name is resolved by the network and should be identical to the name configured by the orbconfigure utility. Figure 7 Logging in to CONTROL-M/EM

4 The server name is automatically displayed once the connection to the CORBA
Naming service is established. This name represents the registered name of the CONTROL-M/EM Server component in the CORBA naming service repository. By default, the name is the physical host name of the computer. To register the CONTROL-M/EM Server components using the virtual host name (as referred to in the example: vhem620) use one of the following options:

When managing CONTROL-M/EM Server components using the Administration facility, define the server components in the Administration facility using the virtual host name as the component name. This name differs from the machine name value, which should be the physical host name of the active node. When managing the CONTROL-M/EM Server components by cluster, start the server component that uses CORBA using the -name parameter, where the value of the parameter should be the virtual host name (as referred to in the example: vhem620).

For additional information and examples, see Phase 7: Configure the policy file.

5 By default, CONTROL-M/EM client components are configured to reconnect to


the GUI Server upon disconnection. If the failover time of the resource group is longer than the reconnection retries and timeout, change these parameters in the CONTROL-M/EM reconnection panel of the options menu, shown in Figure 8.

Chapter 3

CONTROL-M/Enterprise Manager Installation

41

Phase 7: Configure the policy file

Figure 8

Options menu reconnection panel

Phase 7: Configure the policy file


Configure the policy file according to the instructions provided below for either the Administration facility administration mode or the Cluster administration mode.

42

CONTROL-M on UNIX Clusters Implementation Manual

Administration facility administration mode

Administration facility administration mode


To start the CONTROL-M/EM components, enter the following commands.
EM_ACC='em620' EM_DIR='/export1/em620' EM_OWNER='emuser' EM_OWPD='empass' EMDB_VNAME='vhem620' HOSTNAME=`hostname` # Update CONTROL-M/EM database to reflect the active node hostname (Oracle) ORACLE_DIR=$EM_DIR/oracle/product/9.2.0.1 su - $EM_ACC -c "$ORACLE_DIR/bin/sqlplus $EM_OWNER/$EM_OWPD@$EMDB_VNAME" <<endit update CONFREG set MACHINE_NAME = '${HOSTNAME}'; commit; endit # Update CONTROL-M/EM database to reflect the active node hostname (Sybase) SYBASE_DIR=$EM_DIR/sybase/OCS-12_5 su - $EM_ACC -c "$SYBASE_DIR/bin/isql -U $EM_OWNER -P $EM_OWPD -S $EMDB_VNAME" <<endit update CONFREG set MACHINE_NAME='${HOSTNAME}' go endit # Start CONTROL-M EM CORBA Naming Service su - $EM_ACC -c $EM_DIR/scripts/start_ns_daemon # Start the Administration Agent and the CONTROL-M/EM Server components su - $EM_ACC -c $EM_DIR/scripts/start_admin_agent

To stop the CONTROL-M/EM components, enter the following commands.


EM_ACC='em620' EM_OWNER='emuser' EM_OWPD='empass' EMDB_VNAME='vhem620' EM_DIR='/export1/em620' PLATFORM='<platform>' # Stop the Administration Agent and all components su - $EM_ACC -c "$EM_DIR/appl/bin/ecs ctl -U $EM_OWNER -P $EM_OWPD -C Maint_Agent -all -cmd shutdown" # Stop CONTROL-M EM CORBA Naming Service su - $EM_ACC -c "$EM_DIR/appl/ecs/bin.$PLATFORM/orbadmin ns stop -local" sleep 5

NOTE
Modify <platform> to the correct folder name according to the relevant platform vendor.

Chapter 3

CONTROL-M/Enterprise Manager Installation

43

Cluster administration mode

Cluster administration mode


To start the CONTROL-M/EM components, enter the following commands.
EM_ACC='em620' EM_VNAME='vhem620' CTM_VNAME='vhctm620' EM_DIR='/export1/em620' # Start CONTROL-M EM CORBA Naming Service su - $EM_ACC -c $EM_DIR/scripts/start_ns_daemon # Start CONTROL-M EM GUI Server su - $EM_ACC -c "$EM_DIR/appl/bin/ecs guisrv -name $EM_VNAME" & # Start CONTROL-M EM Global Alerts Server su - $EM_ACC -c "$EM_DIR/appl/bin/ecs gasrv -name $EM_VNAME" & # Start CONTROL-M EM Global Conditions Server su - $EM_ACC -c "$EM_DIR/appl/bin/ecs gcsrv" & # Start CONTROL-M EM Gateway su - $EM_ACC -c "$EM_DIR/appl/bin/ecs gtw -dc $CTM_VNAME" &

To stop the CONTROL-M/EM components, enter the following commands.


EM_ACC='em620' EM_OWNER='emuser' EM_OWPD='empass' EM_VNAME='vhem620' HOSTNAME=`hostname` EM_DIR='/export1/em620' CTM_VNAME='vhctm620' PLATFORM='<platform>' # Stop CONTROL-M EM GUI Server su - $EM_ACC -c "$EM_DIR/appl/bin/ecs ctl -U $EM_OWNER -P $EM_OWPD -C GUI_Server -name $EM_VNAME -cmd stop" sleep 3 # Stop CONTROL-M EM Global Alerts Server su - $EM_ACC -c "$EM_DIR/appl/bin/ecs ctl -U $EM_OWNER -P $EM_OWPD -C GAS -name $EM_VNAME -cmd stop" sleep 3 # Stop CONTROL-M EM Global Conditions Server su - $EM_ACC -c "$EM_DIR/appl/bin/ecs ctl -U $EM_OWNER -P $EM_OWPD -C GCS -M $HOSTNAME -cmd stop" sleep 3 # Stop CONTROL-M EM Gateway su - $EM_ACC -c "$EM_DIR/appl/bin/ecs ctl -U $EM_OWNER -P $EM_OWPD -C Gateway -dc $CTM_VNAME -cmd stop" sleep 3 # Stop CONTROL-M EM CORBA Naming Service su - $EM_ACC -c "$EM_DIR/appl/ecs/bin.$PLATFORM/orbadmin ns stop -local" sleep 5

44

CONTROL-M on UNIX Clusters Implementation Manual

Phase 8: Install BMC Batch Impact Manager

NOTE

Modify <platform> to the correct folder name according to the relevant platform vendor. Before defining the CONTROL-M/EM Gateway cluster resource, the CONTROLM/Server should be defined using the Communication Administration window in CONTROL-M/EM. Create a Gateway resource for each CONTROL-M/Server that is connected to CONTROL-M/EM. In the resource command line, indicate the CONTROL-M/Server name as a value to the -dc parameter.

Phase 8: Install BMC Batch Impact Manager


This phase is optional. It is intended only for customers who are licensed to use BMC Batch Impact Manager (BIM).

1 Log in to the CONTROL-M/EM account on the active node (as referred to in the
example: hostcla).

2 Install BMC Batch Impact Manager according to the instructions provided in the
BMC Batch Impact Manager User Guide.

3 The target installation folder should be on the folder on which CONTROL-M/EM


was installed, on the shared disk (as referred to in the example: /export1/em620).

4 Using the CONTROL-M/EM Administration facility, modify the value of the


following system parameters to the CONTROL-M/EM virtual host name (as referred to in the example: vhem620):

BIMHostName BMCImpactCellID BackupGUIServers

5 Deploy the BMC Batch Impact Manager web application according to the
instructions provided in the BMC Batch Impact Manager User Guide.

Chapter 3

CONTROL-M/Enterprise Manager Installation

45

Phase 8: Install BMC Batch Impact Manager

6 Follow the instructions below according to the method used to manage the
CONTROL-M/EM components:

When using the Administration Facility method, use the following parameter values: Type: BIM Machine Type: Unix GUI Server Name: VirtualHostName (as referred to in the example: vhem620) Machine Name: ActiveHostName (as referred to in the example: hostcla) Additional parameters: -name=VirtualHostName (as referred to in the example: -name=vhem620)

When using the cluster method, add the following commands to the policy file of CONTROL-M/EM: For starting the BMC Batch Impact Manager Server component, enter the following commands:

EM_ACC='em620' EM_VNAME='vhem620' EM_DIR='/export1/em620' # Start the Web Application Server # Enter here the commands to start your web application server # Start Batch Impact Manager Server su - $EM_ACC -c "$EM_DIR/appl/bin/ecs bimsrv -name=$EM_VNAME -gsr=$EM_VNAME" &

For stopping the BMC Batch Impact Manager Server component, enter the following commands:
EM_ACC='em620' EM_OWNER='emuser' EM_OWPD='empass' EM_VNAME='vhem620' EM_DIR='/export1/em620' # Stop Batch Impact Manager Server su - $EM_ACC -c "$EM_DIR/appl/bin/ecs ctl -U $EM_OWNER -P $EM_OWPD -C BIM -name $EM_VNAME -cmd stop" sleep 3 # Stop the Web Application Server # Enter here the commands to stop your web application server

7 After starting the CONTROL-M/EM resource group and the BMC Batch Impact
Manager Web Application Server resources are up and running, open a web browser in Microsoft Windows and enter the following URL: http://virtualName:8080/bim

46

CONTROL-M on UNIX Clusters Implementation Manual

Phase 8: Install BMC Batch Impact Manager

EXAMPLE
http://vhem620:8080/bim A page similar to the following is displayed.

8 Click Advanced. 9 Modify the following field values:


Field Value

Host

By default, the host name indicates the physical host name of the cluster node. Replace the default host name with the CONTROL-M/EM virtual host name (in this example: vhem620).

10 Click Add Host. 11 Click Login to log on to BMC Batch Impact Manager.

Chapter 3

CONTROL-M/Enterprise Manager Installation

47

Phase 9: Install CONTROL-M/Forecast

Phase 9: Install CONTROL-M/Forecast


This phase is optional. It is intended only for customers who are licensed to use CONTROL-M/Forecast.

1 Log into the CONTROL-M/EM account on the active node (as referred to in the
example, hostcla).

2 Install CONTROL-M/Forecast according to the instructions provided in the


Release Notes for CONTROL M/Forecast version 6.2.01.

3 The target installation folder should be on the folder on which CONTROL-M/EM


was installed, on the shared disk (as referred to in the example: /export1/em620).

4 Follow the instructions below according to the method used to manage the
CONTROL-M/EM components:

When using the Administration Facility method, use the following parameter values: Type: Forecast Server Machine Type: Unix GUI Server Name: VirtualHostName (as referred to in the example: vhem620) Machine Name: ActiveHostName (as referred to in the example: hostcla) Additional parameters: -n VirtualHostName (as referred to in the example: -n vhem620)

When using the cluster method, add the following commands to the policy file of CONTROL-M/EM: For starting the CONTROL-M/Forecast Server component, enter the following commands:

# Start CONTROL-M Forecast Server EM_ACC='em620' EM_VNAME='vhem620' EM_DIR='/export1/em620' su - $EM_ACC -c "$EM_DIR/appl/bin/ecs forecastserver -n $EM_VNAME -gsr=$EM_VNAME" &

For stopping the CONTROL-M/Forecast Server component, enter the following commands:
# Stop CONTROL-M EM Forecast Server EM_ACC='em620' EM_OWNER='emuser' EM_OWPD='empass' EM_VNAME='vhem620' EM_DIR='/export1/em620' su - $EM_ACC -c "$EM_DIR/appl/bin/ecs ctl -U $EM_OWNER -P $EM_OWPD -C Forecast_Server -name $EM_VNAME -cmd stop" sleep 3

48

CONTROL-M on UNIX Clusters Implementation Manual

Phase 10: Monitor CONTROL-M/EM processes

Phase 10: Monitor CONTROL-M/EM processes


CONTROL-M/EM components can be monitored by the Administration Agent or by the cluster. If the components are monitored by the cluster, use the process names for the cluster monitoring definitions as specified in Table 7. Table 7
GUI Server Global Alert Server (GAS) Gateway Global Condition Server (GCS) Administration Agent TAO Naming Service BMC Batch Impact Manager Server CONTROL-M/Forecast

CONTROL-M/EM processes
Process name ecs.guisrv ecs.gasrv ecs.gtw -dc dcName ecs.gcsrv ecs.maintag Naming_Service -ORBDaemon -ORBListenEndpoints iiop://hostName:port -p emHome ecs.bimsrv ecs.forecastserver

CONTROL-M/EM component

Phase 11: Troubleshooting


This phase presents troubleshooting information for the following components:

The Administration Agent The Global Condition Server

The Administration Agent


In some cases, after setting a virtual host name online, the physical host name of the cluster node becomes unavailable. In such case, additional configuration is required for the Administration Agent. This configuration provides the ability to send life check communications to the Administration Agent using the virtual host name. This configuration should only be used if the Administration Agent appears in the Administration facility as Not Responding after starting the Administration Agent.

Chapter 3

CONTROL-M/Enterprise Manager Installation

49

The Administration Agent

To add a HostPort system parameter for the Administration Agent: 1 Stop the Administration Agent from root_menu or by running the command
stop_admin_agent from the shell prompt (this requires the CONTROL-M/EM DBO password).

2 In the Administration facility, choose Tools => System Parameters. 3 In the general section, select the HostPort parameter and click New.
The Add New Parameter dialog box is displayed:

4 Click Advanced. 5 Modify the following field values:


Field Value General (the default value) HostPort (the default value)

Type Name Value

vhem620:0, which is specified in the format:


virtualHostName:port By specifying 0 (zero) as the port number, the Administration Agent will listen on a random port that is chosen when the Administration Agent starts. If a port other than zero is specified, the Administration Agent will use the specified port number.

Advanced field: Type Advanced field: Name Advanced field: Host

Maint Agent * (the default value) * (the default value)

50

CONTROL-M on UNIX Clusters Implementation Manual

The Global Condition Server

NOTE
If the installation of the CONTROL-M/EM components is distributed between additional hosts other then the cluster nodes, a separate condition must be specified with the physical host names of each cluster node specified in the Host Name parameter.

6 Click Save. 7 Start the Administration Agent.

The Global Condition Server


In some cases, after setting a virtual host name online, the physical host name of the cluster node becomes unavailable. In such case, additional configuration is required for the GCS. This configuration should only be used if the GCS failed to start after a failover.

To add a HostPort system parameter for the GCS: 1 Start the Administration facility and change the desired state of the GCS to DOWN. 2 Ensure that the GCS is not running on any of the cluster nodes (ps -ef | grep gcs). 3 In the Administration facility, choose Tools => System Parameters. 4 In the General section, select the HostPort parameter and click New.
The Add New Parameter dialog box is displayed:

Chapter 3

CONTROL-M/Enterprise Manager Installation

51

The Global Condition Server

5 Click Advanced. 6 Modify the following field values:


Field Value General (the default value) HostPort (the default value) vhem620:55443, which is specified in the format: virtualHostName:port Do not specify 0 as the port number. Advanced field: Type Advanced field: Name Advanced field: Host GCS * (the default value) * (the default value)

Type Name Value

7 Click Save. 8 Start the GCS using the Administration facility.

52

CONTROL-M on UNIX Clusters Implementation Manual

Chapter

CONTROL-M/Server Installation
This chapter is comprised of the following phases, all designed to help you install CONTROL-M/Server to work on UNIX clusters:

Phase 1: Create CONTROL-M/Server UNIX accounts Phase 2: Install CONTROL-M/Server Phase 3: Configure the policy file Phase 4: Monitor CONTROL-M/Server processes

Phase 1: Create CONTROL-M/Server UNIX accounts


If the CONTROL-M/Server database is installed in a dedicated mode this phase should be performed before starting the database installation.

1 Create two user accounts as shown in the following example, one on each node. EXAMPLE
useradd -g controlm -s /bin/tcsh -m -d /export2/ctm620 ctm620

This command should be invoked by a user with administrative privileges

2 Both users must have identical names (referred to in the example as: ctm620) and
identical user IDs (UID).

3 Both user home directories should point to the same location on a shared disk
(referred to in the example as: /export2/ctm620).

Chapter 4

CONTROL-M/Server Installation

53

Phase 2: Install CONTROL-M/Server

Phase 2: Install CONTROL-M/Server


1 Before starting the installation ensure to run the prerequiste check script
(check_req) on each of the cluster nodes.

2 Install CONTROL-M/Server according to the instructions provided in the


CONTROL-M Installation Guide. When prompted to provide the Local IP Host Interface Name, enter the virtual host name (referred to in the example as: vhctm620).

NOTE
As the home directory of the CONTROL-M/Server account is located on the shared disk, the CONTROL-M/Server software will be installed on the shared disk rather than on the node local disk. If CONTROL-M/Server is already installed on the shared disk, change the Local IP Host Interface Name to the virtual host name (vhctm620) in ctm_menu.

A from ctm_menu select option 6 - Parameter Customization B select option 1 - Communication and Operational Parameters
These changes will set the OS_PRM_HOSTNAME parameter of the CONTROL-M/Server configuration file ($HOME/ctm/data/config.dat), to the virtual host name as the parameter value. This change will not take effect until CONTROL-M/Server is shut down and restarted.

3 Install the latest Fix Pack to apply the most recent software updates.

54

CONTROL-M on UNIX Clusters Implementation Manual

Phase 3: Configure the policy file

Phase 3: Configure the policy file


To start CONTROL-M/Server, use the following commands:
CTM_ACC=ctm620 CTM_DIR=/export2/ctm620 su - $CTM_ACC -c $CTM_DIR/ctm/scripts/start_ctm

To stop CONTROL-M/Server, use the following commands:


CTM_ACC=ctm620 CTM_DIR=/export2/ctm620 su - $CTM_ACC -c $CTM_DIR/ctm/scripts/shut_ctm

Phase 4: Monitor CONTROL-M/Server processes


CONTROL-M/Server monitors its own processes internally. Failover of the CONTROL M/Server resource group is not required if only one process fails. The Supervisor process (SU) performs heartbeat monitoring of all other processes. If another process fails, SU will try several times to bring up the process. If these attempts are unsuccessful, SU will shut down CONTROL-M/Server. Therefore, it is sufficient if you only monitor the SU process (/export2/ctm620/ctm/exe_Solaris/p_ctmsu). To ensure that the SU processes do not become a single point of monitoring, one additional process, such as the RT (p_ctmrt), CO (p_ctmco), NS (p_ctmns), LG (p_ctmlg), or the TR (p_ctmtr) process, can be monitored.

Chapter 4

CONTROL-M/Server Installation

55

Phase 4: Monitor CONTROL-M/Server processes

56

CONTROL-M on UNIX Clusters Implementation Manual

Chapter

CONTROL-M/Agent Installation
This chapter is comprised of the following phases, all designed to help you install CONTROL-M/Agent to work on UNIX clusters:

Phase 1: Plan CONTROL-M/Agent configuration Phase 2: Create CONTROL-M/Agent UNIX accounts Phase 3: Install CONTROL-M/Agent Phase 4: Configure the policy file Phase 5: Monitor CONTROL-M/Agent processes

Phase 1: Plan CONTROL-M/Agent configuration


When you implement CONTROL-M/Agent on a UNIX cluster, a dedicated CONTROL-M Agent is installed within each resource group where CONTROL-M should submit jobs. When a single application is running on the cluster, a single Agent should be installed. When multiple applications are running on the cluster, CONTROL-M submits jobs to those applications using different Agents. The file system on which CONTROL-M/Agent is installed on should be located on the shared disk. This file system should always be mounted to the same node as the application where CONTROL-M submits jobs. This file system can be

the same file system as the application file system a different file system, as long as both file systems are always active on the same node (if they are not members in the same application resource group)

Each Agent should be configured to use the application virtual host name for the communication with CONTROL-M/Server. When submitting jobs to this Agent, the NODEID parameter value for the jobs should be the virtual host name.

Chapter 5 CONTROL-M/Agent Installation

57

Phase 2: Create CONTROL-M/Agent UNIX accounts

Before starting the implementation of CONTROL-M/Agent on a UNIX cluster, first identify the file system where the Agent should be installed, and determine the resource group where the agent should be installed. In the procedure described in Phase 2: Create CONTROL-M/Agent UNIX accounts, CONTROL-M/Agent is installed into the same file system as CONTROLM/Server (referred to in the example: /export2), and uses the same virtual network name as CONTROL-M/Server (referred to in the example: vhctm620). The same procedure can be used if CONTROL-M/Agent is installed for any other external application.

NOTE
The solution described above is relevant only for Agents installed on clusters. If only CONTROL-M/Server is installed on a cluster but the Agents are running on non-clustered hosts, no change in the architecture is necessary.

Phase 2: Create CONTROL-M/Agent UNIX accounts


1 Create two user accounts as shown in the following example, one on each node. EXAMPLE
useradd -g controlm -s /bin/tcsh -m -d /export2/ag620ctm ag620ctm

This command should be invoked by a user with administrative privileges

2 Both users must have identical names (referred to in the example as: ag620ctm) and
identical user IDs (UID).

3 Both user home directories should point to the same location on a shared disk
(referred to in the example as: /export2/ag620ctm).

58

CONTROL-M on UNIX Clusters Implementation Manual

Phase 3: Install CONTROL-M/Agent

Phase 3: Install CONTROL-M/Agent


1 Install CONTROL-M/Agent on the relevant file system on the shared disk
according to the instructions provided in the CONTROL-M Installation Guide.

NOTE
If multiple CONTROL-M/Agents are installed on the cluster, use different server-to-agent ports for each CONTROL-M/Agent installation. If all CONTROL-M/Agents are connected to the same CONTROL-M/Server, all CONTROL-M/Agents should be configured to use the same agent-to-server port.

2 Install the latest Fix Pack to apply the most recent software updates. 3 Edit the $HOME/ctm/data/CONFIG.dat file and modify the LOCALHOST parameter
to the virtual host name (referred to in the example: vhctm620)

4 In the CONTROL-M/Agent configuration menu, define the CONTROL-M/Server


host name as authorized to submit jobs to this CONTROL-M/Agent. If CONTROL-M/Server is installed on a cluster, only the virtual network of CONTROL-M/Server (referred to in the example: vhctm620) should be specified.

Job disappeared reasons


Every time a job is submitted, a process is created to monitor the job and report about its completion. This process is called Agent Monitor (AM). When the AM is started (with every job), it creates two files for the job: a status file and a procid file. In a normal scenario, the AM detects the job completion, updates the procid file and sends a trigger to the Agent Tracker (AT) about the completion. The AT then sends the update to CONTROL-M/Server. In a failover scenario, while the job is still executing, the agent process is stopped and the agent file system is unmounted from the first host. In this case the job can keep running, but the procid file will not be updated when the job completes (the agent file system will be mounted to the backup node). Therefore, when the agent is started on the backup node, and the next AT Track-All time arrives, it will find the original procid file but it will not find the actual process. This is why the job is marked as disappeared.

Chapter 5 CONTROL-M/Agent Installation

59

Job disappeared workaround

Job disappeared workaround


As an optional workaround, you can define a JLOST ON statement for the jobs that run on the clustered agent (Statement=*, Code=JLOST) and execute a DO RERUN command. In this case the jobs will be automatically restarted (rerun) on the backup server when CONTROL-M/Server determines that they have disappeared.

NOTE
You must enter value greater than 0 in the MAX RERUN parameter in order for the job to be resubmitted.

Phase 4: Configure the policy file


To start CONTROL-M/Agent processes, use the following commands, which are executed by the root account:
# Start CONTROL-M Agent AG_ACC=ag620ctm AG_DIR=/export2/ag620ctm $AG_DIR/ctm/scripts/start-ag -u $AG_ACC -p ALL

To stop CONTROL-M/Agent and Tracker processes, use the following commands, which should be executed by the root account:
# Stop CONTROL-M Agent AG_ACC=ag620ctm AG_DIR=/export2/ag620ctm $AG_DIR/ctm/scripts/shut-ag -u $AG_ACC -p ALL

Phase 5: Monitor CONTROL-M/Agent processes


When monitoring CONTROL-M/Agent processes by the cluster, use the following process names for the cluster monitoring definitions:

60

CONTROL-M on UNIX Clusters Implementation Manual

Phase 5: Monitor CONTROL-M/Agent processes

Table 8

CONTROL-M/Agent processes
Process name p_ctmag p_ctmat p_ctmra

CONTROL-M/Agent component CONTROL-M/Agent Listener CONTROL-M/Agent Tracker CONTROL-M/Agent Router

NOTE
The CONTROL-M/Agent Router (p_ctmar) is only active when working in a persistent connection mode. When working in transit connection mode, only the CONTROL M/Agent Listener (p_ctmag) and Tracer (p_ctmat) are active.

Chapter 5 CONTROL-M/Agent Installation

61

Phase 5: Monitor CONTROL-M/Agent processes

62

CONTROL-M on UNIX Clusters Implementation Manual

Appendix

Example of installing Apache Tomcat on UNIX clusters


A

This appendix provides an example of installing Apache Tomcat version 5.0.28 on a UNIX cluster. The installation is performed in the same file system as CONTROLM/EM.

NOTE
You can download Apache Tomcat from http://jakarta.apache.org/tomcat.

1 Log into the CONTROL-M/EM account on the active node (as referred to in the
example: hostcla).

2 Extract Apache Tomcat into a subfolder in the CONTROL-M/EM home directory


(as referred to in the example: /export1/em620/tomcat).

3 Set the JAVA_HOME environment variable to the JAVA SDK installation


directory. Ensure JAVA SDK version 1.4.x or later is used.

4 Add the following commands to the Policy file of CONTROL-M/EM:


Commands for starting Apache Tomcat:
# Start Apache Tomcat Application Server EM_ACC='em620' EM_DIR='/export1/em620' TOMCAT_DIR=$EM_DIR/tomcat su - $EM_ACC -c "$TOMCAT_DIR/bin/startup.sh" &

Appendix A

Example of installing Apache Tomcat on UNIX clusters

63

Commands for stopping Apache Tomcat:


# Stop Apache Tomcat Application Server EM_ACC='em620' EM_DIR='/export1/em620' TOMCAT_DIR=$EM_DIR/tomcat su - $EM_ACC -c "$TOMCAT_DIR/bin/shutdown.sh" sleep 3

64

CONTROL-M on UNIX Clusters Implementation Manual

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Index
A
additional database preference choices 27

E
electronic documentation 11 existing CONTROL-M database using 25

B
BMC Batch Impact Manager Server 35 BMC Software, contacting 2

F
file systems using different for database software and data files 26 using same for database software and data files 26

C
Cluster network connections 16 Cluster resources 18 clusters typical two-node architecture 17 configure policy file 29 CONTROL-M database installing 27 CONTROL-M database processes monitoring 30 CONTROL-M/EM GUI Server 35 CONTROL-M/Forecast 35 CONTROL-M/Server processes monitoring 55 conventions documentation 12 customer support 3

G
Global Alerts Server 35

H
Help, online 11

I
installing CONTROL-M database 27

D
database configuration planning 25 database network configuration Oracle 28 Sybase interfaces file 28 database network configuration files modifying 28 database preference choices additional 27 dedicated CONTROL-M database using 25 documentation information 11 special conventions 12

M
modifying database network configuration files 28 monitoring CONTROL-M database processes 30 CONTROL-M/Server processes 55 Oracle processes 30 Sybase processes 30 Monitoring clustered applications 20 Monitoring HA applications 20 multiple nodes sharing media 17

Index

65

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

O
online Help 11 Oracle database network configuration 28 Oracle processes monitoring 30

two-node cluster 17

U
UNIX cluster products 22 using dedicated CONTROL-M database 25 different file systems for database software and data files 26 existing CONTROL-M database 25 same file system for database software and data files 26 same or different resource group for database software and CONTROL-M 26 using a dedicated or existing CONTROL-M database 25

P
Phase 1 Basic CONTROL-M/Agent implementation 59 Create CONTROL-M/Server UNIX accounts 53, 58 Phase 2 Install CONTROL-M/Server 54 Phase 3 Configure the policy file 55 planning database configuration 25 policy file configure 29 product support 3 publications, related 12

Q
Quorum device 21

R
related documentation 11 related publications 12 Resource groups 19 resource groups using same or different for database software and CONTROL-M 26

S
sharing media multiple nodes 17 Sharing media between multiple nodes 17 support, customer 3 Sybase interfaces file database network configuration 28 Sybase processes monitoring 30 syntax statement conventions 13 Syntax statements 13

T
technical support 3

66

CONTROL-M on UNIX Clusters Implementation Manual

Notes

*56190* *56190* *56190* *56190*


*56190*