Overview 2
Introduction to Exchange 2000 Clustering 3
Reviewing Key Concepts of Microsoft
Windows 2000 Clustering 8
Examining Key Concepts of
Exchange 2000 Clustering 16
Exchange 2000 Clustering Best Practices 23
Appendix
Windows 2000 Clustering Best Practices 52
References 61
Exchange 2000
Clustering Best
Practices
-1-
Overview
IntroductiontoExchange2000Clustering
ReviewingKeyConcepts of MicrosoftWindows 2000
Clustering
ExaminingKeyConcepts of Exchange2000
Clustering
Exchange2000ClusteringBestPractices
Appendix: Windows 2000ClusteringBestPractices
One important decision that you must make when designing your Microsoft®
Exchange 2000 organization is the desired level of availability of your
Exchange 2000 messaging system. If your goal is to maximize uptime for your
messaging system, you need to consider using Exchange 2000 server clusters.
Clustering offers server and application redundancy that effectively reduces the
downtime of your messaging system.
To design an appropriate Exchange 2000 clustering strategy for your company,
you need a solid understanding of server clustering architecture. Because
Exchange 2000 server clusters are built on Microsoft Windows® 2000 server
clusters, you must first understand how Windows 2000 server clusters work.
You must also understand how Exchange 2000 server clusters work in the
clustering environment. Finally, you must choose both a clustering model and a
data storage strategy that help to ensure the high availability of your
Exchange 2000 messaging system.
An appropriate Exchange 2000 clustering strategy ensures that you will have
the desired server and application redundancy to achieve high availability for
your Exchange 2000 messaging system. However, before you design an
Exchange 2000 clustering strategy, you must understand the architecture of
server clustering, and the benefits that server clustering offers. It is also
imperative that you ensure that your decision to use a clustering solution is
justified by the business needs of your company.
-2-
Architecture of Server Clustering
A server cluster is a group of servers and storage devices that can be accessed
by clients as a single system. The individual servers in the cluster are referred to
as nodes, which function together to provide automatic recovery when clustered
services and applications fail.
An active node is a node that is actively supporting the clustered services and
applications. A passive node is a node that remains idle until it takes over the
clustered services and applications from a failed node. Depending on the
number of nodes it has, a server cluster can have one of several configurations
such as active/passive, active/active, 2 active/1 passive, or 3 active/1 passive.
There are two types of network communications in a server cluster: private and
public. The nodes communicate with each other over a high performance,
reliable, private network, and share one or more common storage devices.
Clients communicate to logical servers, referred to as virtual servers, through a
public network to gain access to grouped resources, services, and applications.
When a client connects to a virtual server, the virtual server routes the request
to the node controlling the requested resource, service, or application. If the
controlling node fails, all clustered services or applications running on the
failed node will restart on an alternate designated node.
-3-
Implementing Exchange 2000 server clusters involves significant infrastructure
investment such as hardware and data storage devices. As a result, you must
make sure that your clustering solution can be justified by the business needs of
your company.
-4-
The following table shows some of the commonly encountered points of failure
and their possible solutions. It is clear that clustering is not always a solution to
prevent a point of failure.
Point of failure Cluster service solution Possible other solutions
When you create Exchange 2000 server clusters, you use the services that are
provided by Windows 2000 Cluster service. In fact, you run Microsoft
Exchange 2000 Server Setup on a node in the Windows 2000 server cluster to
install the cluster-aware version of Exchange 2000.
Because Exchange 2000 server clusters are built on Windows 2000 server
clusters, it is imperative that you understand the key concepts of Windows 2000
server clusters before designing your Exchange 2000 server clusters.
Clustering Components
Windows 2000 server clustering has such components as nodes, cluster disks, a
quorum resource, virtual servers, resources, and groups or resource groups.
Cluster service refers to the collection of components on each node that
performs cluster-specific activities.
-5-
Nodes
Nodes are the individual servers that comprise a server cluster. Nodes are units
of management for the server cluster. A node can be online or offline,
depending on whether it is currently in communication with the other nodes in
the cluster.
Cluster Disks
Cluster disks are shared hard drives to which all server cluster nodes attach by
means of a shared bus or storage area network (SAN). You store data,
applications, resources, and services on the shared disks.
Cluster disks are configured by using RAID, with redundant power supplies and
network interface cards. Cluster disks come in several packaged solutions such
as shared small computer system interface (SCSI), SANs, and Network Area
Storage appliances.
Quorum Resource
The vital function of the quorum resource is allowing a node to form a cluster
and maintaining consistency of the cluster configuration for all nodes. The
quorum resource holds the cluster management data and recovery log, and is
used to arbitrate among nodes to determine which node controls the cluster.
The quorum resource resides on a shared cluster disk. It is best to use a
dedicated cluster disk for the quorum resource so that it will not be affected by
failover policies of other resources, or by the space that other applications
require. It is recommended that the quorum resource be on a disk partition of at
least 500 megabytes (MB).
Virtual Servers
Cluster service uses a physical server, or a cluster node, to host one or more
virtual servers. Virtual servers have server names and network configurations
that appear as physical servers to clients. Important facts about a virtual server
are:
Each virtual server has an Internet Protocol (IP) address and a network
name that are published to clients on the network. This information allows
network clients to interact with the virtual server as if it were a physical
server.
Clients access applications or services on a virtual server in the same way
that they would if the application or service were on a physical server. They
do not know which node is actually hosting the virtual server.
Because clients connect to the virtual server directly and are not concerned
about the node that hosts the virtual server, you can move the virtual server
from one node to node without affecting your clients.
-6-
Note Virtual servers in a clustering environment should not be confused
with Exchange 2000 virtual servers such as SMTP virtual servers. Although
they are both called virtual servers, they represent two different concepts. For
more information about Exchange 2000 virtual servers, see Course 1572C,
Implementing and Managing Microsoft Exchange 2000.
Resources
Resources are the basic management and failure units of Cluster service.
Examples of resources are physical hardware devices, such as disk drives, or
logical items, such as IP addresses, network names, applications, and services.
A cluster resource can only run on a single node at any given time and is
identified as online when it is available for a client to use. Under the control of
Cluster service, resources may migrate to another node as part of a group
failover. For example, when Cluster service detects that a single resource has
failed on a node, it then moves the whole group to another surviving node in the
cluster.
Cluster service uses resource monitors to track the status of the resources.
Cluster service will attempt to restart or migrate resources when they fail or
when one of the resources that they depend on fails.
Resource Groups
Groups or resource groups are logical collections of cluster resources that
Cluster service manages as single units for configuration purposes. Typically, a
resource group is made up of logically related resources such as applications
and their associated peripherals and data. However, resource groups can contain
cluster entities that are related only by administrative needs, such as an
administrative collection of virtual server names and IP addresses. A resource
group can be owned by only one node at any given time. Individual resources
within a group must exist on the node that currently owns the group. At any
given instance, different servers in the cluster cannot own different resources in
the same resource group.
Each resource group has an associated cluster-wide policy that specifies which
server or node the group prefers to run on, and which server or node the group
should move to in case of a failure. Each group also has a network service name
and address to enable network clients to bind to the services provided by the
resource group. In the event of a failure, resource groups can be failed over or
moved from the failed node to another available node in the cluster. However,
clients on the network may still access the same resources by using the same
network name and IP address.
Cluster Communications
-7-
The mixed network can be used either for intra-cluster communications or
for client access to the cluster.
Failover
Failover can occur automatically because of an unplanned hardware or
application failure, or can be triggered manually by the person who administers
the cluster for maintenance. The algorithm for both situations is identical; when
a resource fails, all other resources in the resource group attempt to shut down
safely before the failover takes place.
When an entire node in the cluster fails, its resource groups are moved to one or
more available servers in the cluster. Automatic failover is similar to planned
administrative reassignment of resource ownership. It is, however, more
complicated because the normal shutdown process is not safely performed on
the failed node.
Automatic failover requires determining what groups were running on the failed
node and which nodes should take ownership of the various resource groups.
The node preference list, which is part of the resource group properties, is used
to assign a resource group to a node. After the assignment is complete, all nodes
in the cluster update their databases and keep track of which node owns the
resource group. Clusters that are configured as active/passive clusters use N+1
failover when setting the node preference list of all resource groups.
N+1 Failover
N+1 failover sets the node preference list of all resource groups to a passive
node that is not the primary node for any resource group. The node preference
list identifies the passive cluster node to which resources should be moved
during first failover. The standby node is a node in the cluster that is mostly
idle.
N+1 failover typically provides the fastest failover time. This is because the
passive node does not support any virtual server during normal operations. The
-8-
passive node remains idle, waiting to take over a virtual server if one of the
active nodes fails.
Failback
When a node comes back online, some resource groups are moved back to the
recovered node. This move is referred to as failback. The properties of a
resource group must have a preferred owner defined to failback to a recovered
or restarted node. Resource groups for which the recovered or restarted node is
the preferred owner will be moved from the current owner to the recovered or
restarted node.
Cluster service provides protection against failback of resource groups at peak
processing times, and also protects nodes that have not been correctly recovered
or restarted. Failback properties of a resource group may include the hours of
the day during which failback is allowed, plus a limit on the number of times
that failback is attempted.
Cluster Administrator
Note For more information about developing extensions, see the Microsoft
Platform Software Development Kit (SDK).
When you run Exchange 2000 Server Setup on a node in the Windows 2000
server cluster, Exchange 2000 Server installs the cluster-aware version of
Exchange 2000 with all the necessary custom files and resources that are
required for the Exchange 2000 server cluster to function.
-9-
This presentation provides an overview of the key concepts of Exchange 2000
clustering.
To create an Exchange 2000 server cluster, you must meet the hardware and
software requirements that are described in the following sections.
Hardware Requirements
All hardware used for Exchange 2000 clustering must be on the Hardware
Compatibility List (HCL). You can find the most recent version at
http://www.microsoft.com/hcl/default.asp.
Software Requirements
Specific versions of Windows 2000 and Exchange 2000 Server are required to
create Exchange 2000 server clusters. These version requirements are described
in the following table.
Exchange clusters
Windows 2000 Exchange 2000 available
- 10 -
Note It is recommended that Exchange 2000 SP1 or greater always be used
when using Exchange 2000 server clusters. SP1 contains several enhancements
that improve performance and availability of Exchange 2000. In addition, you
should install the latest service pack for your copies of Windows 2000
Advanced Server or Windows 2000 Datacenter Server.
When you create an Exchange 2000 server cluster, you create a Windows 2000
cluster group and then add specific resources to it. Exchange 2000 server
clusters are referred to as Exchange Virtual Servers. Unlike the physical
computer running Exchange 2000 Server, an Exchange Virtual Server is a
cluster group that can be failed over if the physical server itself fails.
(continued)
Resource name Functionality Description
- 11 -
Resource name Functionality Description
Post Office Active/Active Provides connections to client computers and
Protocol version 3 is a dependency of the Exchange Information
(POP3) Store.
Hypertext Transfer Active/Active Provides connections to client computers and
Protocol (HTTP) is a dependency of the Exchange Information
Store.
Content Indexing Active/Active The MSSearch resource provides content
indexing for the Exchange Virtual Server and
is a dependency of the Exchange Information
Store.
Message Transfer Active/Passive The MTA resource is active/passive; there can
Agent (MTA) be only one MTA per cluster. The MTA is
created on the first Exchange Virtual Server. If
the Exchange Virtual Server with the MTA is
not the last Exchange Virtual Server in the
cluster and is deleted, the MTA will be moved
to another Exchange Virtual Server in the
cluster. The MTA serves all Exchange Virtual
Servers in the cluster as long as it is online.
The MTA is a dependency of the System
Attendant.
Routing Service Active/Active Builds the link state tables and is a
dependency of the System Attendant.
Unsupported Resources
Exchange 2000 server clusters do not support the following Exchange 2000
Server components as resources:
Active Directory® Connector (ADC)
Chat Service
Microsoft Exchange 2000 Conferencing Server
Instant Messaging
Key Management Service
Exchange Calendar Connector
Exchange Connector for Lotus cc:Mail
Exchange Connector for Lotus Notes
Exchange Connector for Microsoft Mail
Exchange Connector for Novell GroupWise
Event Service
Network News Transfer Protocol (NNTP)
Site Replication Service (SRS)
- 12 -
Exchange 2000 Clustering Best
Practices
DesignAndDeploymentBestPractices
ExaminingAvailableClusteringModels
ChoosinganAppropriateClusteringModel
DesignConsiderations
ChoosingAppropriateDataStorage
StoringExchangeData
PerformanceMonitoring
ImprovingPerformance
Backupand Recovery BestPractices
Exchange 2000 best practices start right from installations. You must follow
design guidelines to design and deploy cluster servers.
Choose the appropriate clustering model. You must also be aware of such issues
as storage group limitations, server roles, location of databases and storage
group log files, multi-processor support limitations, memory limitations, and
drive letter limitations. All these facts will inevitably influence your design
decision.
Choose the appropriate data storage and store the Exchange databases and files
appropriately to optimize performance.
Once the server is installed and operational you should monitor its performance
take regular backups. Have a disaster recovery plan and practice restores.
There are three clustering models available: two-node, three-node, and four-
node. The two-node model is based on Windows 2000 Advanced Server, while
the three-node and four-node clustering models are based on Windows 2000
Datacenter Server.
- 13 -
Two-Node Active/Passive Configuration
In this configuration, the primary node of the Exchange 2000 server cluster
supports a single Exchange Virtual Server and services all client computers
while the secondary node is idle. The secondary node is a dedicated server that
is ready to be used whenever a failover occurs on the primary node. If the
primary node fails, the secondary node picks up all operations and continues to
service client computers at a rate of performance that is close or equal to that of
the primary node.
Because this configuration has a dedicated secondary node, it ensures that your
Exchange 2000 server is minimally affected by a failover. As a result, you are
able to achieve the maximum availability and performance of your
Exchange 2000 messaging system.
Choosing an Appropriate
Clustering Model
- 14 -
Choosing a clustering model involves making two decisions: the number of
nodes in your cluster and the specific configuration of the nodes in the cluster.
- 15 -
Choosing a Three-Node or Four-Node
Model
Three-node and four-node clusters are for companies that have a large number
of mailboxes in a centralized location that require high availability. If the
number of mailboxes that you need to support in a single location exceeds what
can be supported on a single server running Exchange 2000, you will not be
able to use a two-node cluster. In this case, you should use the three-node or
four-node model so that you can distribute the mailboxes across two or three
active nodes with one passive node available for failover.
Whether to choose a three-node or four-node cluster depends on two factors:
the number of total mailboxes that the cluster will support and the capacity of
the hardware that you plan to use in the cluster. For example, you have 3,000
mailboxes that you need to host in a cluster. You have determined that the class
of server that you plan to use in your cluster can support 2,000 mailboxes while
still providing your required level of performance. You can then divide the
3,000 mailboxes between two active nodes (1,500 mailboxes per node), with a
third node configured as a passive node. This three-node 2 active/1 passive
cluster can grow and support up to 4,000 mailboxes (2,000 mailboxes per
node). If with the growth of your company, you must support more than 4,000
mailboxes, you can add an additional active node to your existing three-node
cluster, making it a four-node 3 active/1 passive cluster and dividing the
mailboxes evenly among the three active nodes.
Note Most hardware vendors that sell cluster capable hardware have tools
that can help you to determine the servers that are right for your clustering
requirements.
Design Considerations
The following are aspects that you must consider when designing the
architecture for your Exchange 2000 server cluster.
If EVS2 on node 1 fails over to node 2, one of the storage groups from node 1
will fail to mount on node 2. This failure occurs because node 2 will have
exceeded the storage group limit for a single server and a cluster node.
- 16 -
Dedicated Server Roles
If the purpose of your clusters is to provide Exchange 2000 services to your
users, it is recommended that your servers only run Exchange 2000. In addition,
the servers in your Exchange 2000 server clusters should be member servers of
a domain and not domain controllers or global catalog servers.
Memory Limitations
Exchange 2000 Server does not support instancing, which is the ability to run
multiple instances of an application as separate processes on the same
computer, or Physical Address Extension (PAE), which limits Exchange 2000
Server to about 3 gigabytes (GB) of usable memory. Therefore, you should not
install more than 3 GB of physical memory on a computer running
Exchange 2000 Server. In addition, Exchange 2000 requires that the /3GB
switch be used on your servers that have more than 1 GB of physical RAM
installed.
Note For more information about PAE, see the Microsoft Web site at
http://www.microsoft.com/hwdev/PAE
- 17 -
Designing an Appropriate Data
Storage Strategy
There are several data storage methods that are available when you use
Exchange 2000 server clusters. The two most popular choices are external
storage arrays and SANs. Which data storage method you choose will have a
potential impact on the performance and availability of your Exchange 2000
server cluster.
- 18 -
unparalleled data-access performance. By using RAID 0+1 arrays, SANs
provide high levels of fault tolerance and the ability to withstand disk failures.
In addition, the advent of fiber channel switches provides much higher levels of
throughput and allows administrators to design SANs with no single points of
failure.
External storage arrays that are physically attached to your servers running
Exchange 2000 slow down application and I/O processing. In contrast, SANs
improve your Exchange 2000 performance by shifting data transactions and I/O
away from your servers running Exchange 2000. As a result, SANs offer the
most efficient way to add and manage data storage capacity while ensuring
continuous availability.
Although SANs have traditionally been a more expensive solution than a
simple external storage array, the price for SANs has recently become more
affordable for most clustering budgets.
Planning Considerations
- 19 -
Planning external storage arrays and planning SANs require different
considerations.
Availability Levels
Most high-end SANs are designed to offer 99.999 percent availability. Such
high availability is accomplished through redundant components, the ability to
“hot swap” components that fail, and the ability to upgrade firmware and
- 20 -
software without any downtime. Therefore, the total availability of your
Exchange 2000 messaging system should not be compromised by your SAN.
Performance
The performance of your SAN affects the performance of your Exchange 2000
cluster. When selecting a SAN solution, you should consider the performance
ratings of each vendor’s storage solution. Ask each vendor how performance of
their storage solutions, such as maximum I/O, compares with their competitors’
storage solutions when configured with the number of disks you envision for
your WAN solution.
SMTPQueuefolder
ExchangeDatabasefiles
TransactionLogfiles
IndexingFiles
- 21 -
performance before capacity and reliability. However, in some situations when
downstream processes fail, the SMTP queue could be required to store a large
amount of data. For that reason, do not assume that a RAID-0 array is the best
storage solution for SMTP queues. Generally, RAID-0 is acceptable only if
mail loss is acceptable. RAID-1 is a good solution because it gives some
measure of reliability while providing adequate throughput.
Indexing Files
- 22 -
- 23 -
Performance Monitoring
Memory
Virtual Memory
MSExchangeIS\VMLargest BlockSize
MSExchangeIS\VMTotal 16MB FreeBlocks
MSEXchangeIS\VMTotal FreeBlocks
MSExchangeIS\VMTotal LargeFreeBlockBytes
/3GB Switch
Ensuring that your Exchange 2000 Server clusters perform well involves setup
steps and proactive monitoring of your clusters. The following sections provide
steps to improve performance, monitor performance, and test the performance
of your Exchange 2000 clusters.
Memory
By default, Exchange 2000 uses all of the physical memory available on your
computer; however, you can restrict the amount of memory used by Exchange.
The biggest individual consumer of memory in Exchange 2000 is the Store.exe
process. On an active, production Exchange 2000 Server, it is not uncommon to
notice that the Store.exe process consumes nearly all of the server memory.
Like Exchange Server version 5.5, the Store.exe process uses a unique cache
mechanism called Dynamic Buffer Allocation (DBA). This process self-
governs how much memory it uses, and balances that with other applications
running on the server. If Exchange is the only application running, DBA
allocates more memory to itself.
The amount of memory you need in your server depends on the number of
databases, size of the databases, and number of transactions. As you create
more Exchange databases on the server, your memory requirements increase.
Database configuration achieves a certain amount of memory preservation. For
example, the first database in a storage group consumes the greatest amount of
virtual memory; whereas, if you add a new database to an existing storage
group, the memory consumption of this database will be less.
Exchange 2000 can handle up to 20 databases per server; the total storage space
is made up of a maximum of 4 storage groups and 5 databases per storage
group. Wherever possible, fill out your storage groups to the maximum number
of databases before you create a new storage group.
- 24 -
The advantages to filling out your storage group are:
• Reduced memory consumption
• Reduced disk overhead
• However, there are a few disadvantages:
• Circular logging can only be controlled at the storage group level and
is not recommended.
• Only one backup process can take place in a single storage group at a
time. Backing up one database in a storage group will halt the online
maintenance of all other databases in the storage group.
Virtual Memory
Windows 2000 implements a virtual memory system based on a flat (linear) 32-
bit address space. Thirty-two bits of address space translates into 4 GB of
virtual memory. On most systems, Windows 2000 allocates half this address
space (the lower half of the 4-GB virtual address space from x00000000
through x7FFFFFFF) to processes for its unique private storage and the other
half (the upper half, addresses x80000000 through xFFFFFFFF) to its own
protected operating system memory usage.
It is important to monitor the virtual memory on your Exchange 2000 clusters.
For more information about monitoring virtual memory, see the “Performance
Monitoring” section in this document.
For more information about virtual memory, see your Windows 2000 online
documentation, Microsoft Windows 2000 Server Resource Kit, and Inside
Microsoft Windows 2000.
/3GB Switch
If your computer running Exchange 2000 has 1 GB of physical memory or
more, it is very important to add the /3GB switch to the Boot.ini file on the
server so that 3 GB are available for user-mode applications. By default,
Microsoft Windows 2000 Advanced Server reserves 2 GB of virtual address
space for the kernel and allows user mode processes, such as the Exchange
2000 Store.exe process, to use 2 GB of virtual address space.
Virtual address space for a specific process is allocated at startup and increases
as more memory is used during run time. It is normal for the actual memory
usage of a process to be much less than the address space the process was
allocated.
Important If your computer running Exchange 2000 has 1 GB of memory or
more, it is very important that the Store.exe process does not run out of virtual
address space. If it runs out of virtual address space, memory allocation fails,
even if there is plenty of physical RAM left, and you must restart the
Information Store service.
Example
A server with 2 GB of physical RAM without the /3GB switch in the Boot.ini
file will run out of memory when the Store.exe process’s virtual address space
reaches 2 GB. Windows Task Manager will show that only about 1.5 GB is
actually being used when actually the server is out of memory.
- 25 -
- 26 -
Exchange 2000 Server Cluster
Failover Performance
Many factors determine the speed of Exchange 2000 Server cluster failovers. If
an Exchange Virtual Server must fail over, certain tasks must be accomplished
to complete the failover. Understanding these tasks helps you configure your
Exchange 2000 Server clusters for the fastest possible failover.
- 27 -
Backup and Recovery
BackingUpData
RecoveringaSingleLostServer inCluster
RecoveringaLostCluster Quorum
It is important to back up all of the important data for your company. This
includes the contents of your users’ mailboxes and the configuration data that is
needed to operate the servers running Exchange 2000. Make sure that you have
backups of your static data, such as all software applications and management
scripts. In addition, it is advisable to make regular backups of your dynamic
data, such as all Exchange 2000 configuration data and your Exchange
databases.
- 28 -
Backing Up cluster Server
Types of
Backing Up Data on an Exchange 2000 Server
Cluster Node
Windows
Window
Use Windows 2000 Backup to back up a cluster node in which the Cluster
service is operational. On the What to Back Up screen of the wizard, select
Back up everything on my computer.
Exchang
Be sure the node you back up is the owner of the cluster quorum disk. To check
Exchang
the ownership, stop the Cluster service on all other nodes except the node
running Windows 2000 Backup. Then chose one of the following options:
• Select Only back up the System State data to back up the system
state, which includes the quorum.
To back up all cluster disks owned by a node, perform the backup from that
Supporti
node.
Support
Note: During backup, Windows 2000 Backup might report the following error:
“Completed with Skipped Files.” If you examine the Windows 2000 Backup
log, notice that both CLUSDB and ClusDB.log failed to be backed up. You
should ignore this error. The quorum logs from the cluster quorum drive are
successfully backed up.
After you back up the cluster quorum disk on one node, you do not need to
User
UserApp
Ap
back up the quorum on the remaining cluster nodes. As an option, you can also
back up the cluster software, cluster administrative software, system state data,
and other cluster disks on the remaining cluster nodes.
- 29 -
Managem
Manage
Recovering Cluster Servers
Recoverin
Recovering a Single Lost Server in a Cluster
Have
Haverep
rep
If a single server in a cluster fails, Exchange resources running on the server
move to another available node in the cluster. Exchange databases remain intact
on shared storage and accessible by the Exchange Virtual Server from other
nodes in the cluster. This is a feature of clustering, and it provides reliability
when a disaster occurs on a single server in a cluster. After you move resources
to an available node in the cluster, use the following procedures to remove the
nonfunctioning node and replace it with a new node.
Have
HaveWin
Wi
When a server suffers a disaster in a cluster and needs to be replaced by a new
node, follow these steps:
• Use Cluster Administrator to evict the lost node from the cluster.
•
available
available
Use Cluster Administrator to verify for each cluster group and resource
that the evicted node no longer appears as a possible or preferred
owner.
• Physically remove the damaged node from the cluster and shared
storage.
Have
Havefull
You do not have to rebuild the lost node to replicate the original lost node. You
ful
can build an entirely new node (new computer name, new IP address, and so
on) and then join the cluster.
- 30 -
• Join the same domain as before with the same administrative permissions
given to the previous Exchange administrator account.
• Set up the new computer to access the same shared storage of the original
node.
- 31 -
Restore Cluster Quorum from Backup
Before the cluster can be restarted on any nodes in the cluster, the quorum
needs to be restored.
• Use the DumpConfig tool to restore the signature of the quorum disk if the
signature has changed since it was backed up. You can find the
DumpConfig tool in Microsoft Windows 2000 Server Resource Kit.
• If the Cluster service is running, stop the service on all cluster nodes.
• Use Windows 2000 Backup to restore the system state data (which contains
the contents of the cluster quorum disk). Windows 2000 Backup puts the
contents of the cluster quorum disk in the
systemroot\cluster\cluster_backup subdirectory.
• After you restore the cluster quorum, you are prompted to restart. Instead of
restarting, run Clusrest.exe to restore the contents of the
systemroot\cluster\cluster_backup subdirectory to the cluster quorum disk.
The Clusrest.exe tool can be found in Microsoft Windows 2000 Server
Resource Kit.
• Restart the computer.
- 32 -
Recovering Databases
Recoverin
ActiveDire
If the server running Exchange 2000 is still functional after a disaster,
recovering a database is a straightforward process. You can use the
Forest
Windows 2000 Backup utility to restore the databases that you want to recover.
For step-by-step procedures for recovering databases, see the Exchange 2000
online documentation.
To recover one or more databases, first make sure that the Exchange
Information Store is running. In addition, make sure that the database or
databases that you want to restore are dismounted.
Because Exchange 2000 supports multiple storage groups and multiple
databases, it is only necessary to dismount the specific database or databases
that you want to restore. This allows users to continue to access all of the other
databases in the information store. Note that each storage group has a log file
signature value. Each log file in the sequence has the signature stamped on it.
The corresponding .edb file stores this signature value in its header information,
which prevents you from accidentally replaying log files from a different
storage group against the database you are restoring.
- 33 -
or a single attachment, it is possible that you could lose a whole folder, a whole
mailbox, or even the entire store.
Note When you are restoring from a backup set, your current database is
overwritten as soon as the process begins. Rename the database that you are
restoring before you begin the restore process. If you do not leave your database
drive at least half empty, you will not be able to restore from backup because
you will not have enough space left for the restore.
- 34 -
Either log on to the production domain as the original user and copy the
information from the .pst file to the original users mailbox, or give the .pst
file to the original user and direct them to recover their data.
- 35 -
Recovering Mailboxes
Recoverin
You can recover a deleted mailbox by reconnecting it to a new user account.
You can recover a damaged mailbox by restoring it from a backup.
Recov
enabling the Recover Deleted Items feature on the server running
Exchange 2000.
Recov
30-day retention period, then you can recover and reconnect that mailbox
without restoring the entire database because of the GUID associated with each
mailbox.
Every mailbox has a GUID that never changes. Users in Active Directory also
Expire
have an attribute called msExchMailboxGuid. When a user is connected to a
mailbox, their GUID attribute matches the msExchMailboxGUID associated
with a mailbox in the store.
When you delete a user account from Active Directory, although their
Exchange mailbox is not deleted, there is no longer a user account in Active
Directory with a GUID that points to that mailbox. The next time the Cleanup
Wizard or Information Store (IS) maintenance runs, the mailbox is marked as
De
disconnected.
Reconnecting a mailbox takes the msExchMailboxGUID value that is on the
mailbox in the store and associates it with the user account that you specify.
- 36 -
The Cleanup Wizard reads the msExchMailboxGUID associated with each of
the mailboxes in the store and searches Active Direcotry for a user account with
that GUID. If the wizard cannot find a matching user account, then it marks the
mailbox as “disconnected.”
If the retention period has expired, you can recover the mailbox by performing
the following tasks:
Install the recovery server in a different Active Directory forest than that in
which the original server is located, because only one Exchange 2000
organization can exist in any one Active Directory forest.
You are not required to match the name of the recovery forest to that of the
original forest. Recovery servers running Exchange 2000 can exist in the
same physical network as the original organization without interfering either
with Active Directory or with the Exchange 2000 organization.
Install Exchange 2000 on the recovery server by using the same
organization name that was used in the original organization.
Recover the database to an administrative group in which the
legacyExchangeDN values match the legacyExchangeDN values in the
administrative group from which the database was originally located.
Name the restore storage group and the restore logical database so that their
names match the original storage group and logical database names.
Create a .pst file, and move all data that you need to recover into the .pst
file. Open the .pst file on the production server, and move the data back to
the appropriate location.
- 37 -
Creating a Disaster Recovery Plan
Creating
Keep
Keepaacop
An effective disaster recovery plan documents standard operating procedures, is
co
known and understood by all administrators, and is periodically tested in a lab
environment to verify its accuracy.
informatio
When you develop a disaster recovery plan for your Exchange 2000 data, you
informatio
must address several system prerequisites and logistic considerations.
you
youmay
maynn
Prerequisites
The first steps in creating a disaster recovery plan are to make sure that you
have performed the following:
Installed the correct tape drivers on each server that you want to back up.
Supplied each server that you want to back up with an appropriate number
of backup tape sets. Depending on the amount of data that you intend to
back up, each tape set may require multiple tapes.
Verify
Verifythat
Configured each server that you want to back up not to reboot automatically
tha
to prevent the MEMORY.DMP file from being overwritten by another,
immediate system malfunction.
of
oflog
logfiles
Configured each server that you want to back up to write an event to the
file
system log, to send an administrative alert, to write all debugging
information to %SystemRoot%\MEMORY.DMP, and to overwrite any
existing file.
Logistics
In addition to addressing all of the prerequisites in the preceding list, you must
also settle the following logistic issues:
Maintain a copy of your backup procedures, of your configuration
information, and of all appropriate repair disks in the same room with each
Disable
Disablecir
server that you might need to back up. Configuration information should
ci
include details about your operating system and about your server running
- 38 -
restore to
Exchange 2000, in addition to any hardware-specific data, such as
information about your Redundant Array of Independent Disks (RAID)
configuration and disk partitioning.
Make sure that you have enough capacity on your hard disk or disks to
restore both the database and the log files. Remember that a full weekly
backup plus one week of transaction log files might be more than your
server can store. This depends partly on how many log files are generated
during each week. For example, if your server generates 2,000 log files each
week, then this will amount to 10 GB of log file space, in addition to the
space required by the database itself.
Consider the location of your tape drives. If a tape drive is located on a
server, consider the effect that the additional load will have on the other
services on that server. Depending on the number of available tape drives,
tape backups of most servers may be preferable to backing up data across
the network.
Remember that circular logging automatically overwrites transaction log
files after the data that those files contain has been fully committed to the
database. Although circular logging reduces disk storage space
requirements, when it is enabled you cannot perform either differential or
incremental backups, and you cannot recover to the point of failure.
Plan to back up mailbox stores as often as possible. Ideally, you should back
up mailbox stores once each business day.
Plan to replicate or back up critical public folders. Ideally, you should either
replicate these folders at least once per business day, or back up the public
folder store once each business day.
Plan to keep a copy of your data backups at an off site location.
- 39 -
Appendix
Contents
Overview..........................................................................................................2
Introduction to Exchange 2000 Clustering......................................................2
Architecture of Server Clustering....................................................................3
Advantages of Server Clustering ....................................................................3
Identifying Business Needs for Using Exchange 2000 Clustering.................3
Reviewing Key Concepts of Microsoft Windows 2000 Clustering................5
Clustering Components....................................................................................5
Cluster Communications..................................................................................7
Failover and Failback.......................................................................................8
Cluster Administrator.......................................................................................9
Examining Key Concepts of Exchange 2000 Clustering................................9
Multimedia: Exchange 2000 Clustering..........................................................9
Hardware and Software Requirements..........................................................10
Exchange 2000 Virtual Servers.....................................................................11
Exchange 2000 Clustering Best Practices......................................................13
Examining Available Clustering Models.......................................................13
Choosing an Appropriate Clustering Model..................................................14
Design Considerations...................................................................................16
Designing an Appropriate Data Storage Strategy..........................................18
Examining Available Data Storage Models..................................................18
Choosing an Appropriate Data Storage Model..............................................19
Planning Considerations................................................................................19
Storing Exchange Data.............................................................................21
Performance Monitoring................................................................................24
Exchange 2000 Server Cluster Failover Performance...................................27
Backup and Recovery....................................................................................28
Backing Up cluster Server.............................................................................29
Recovering Cluster Servers............................................................................30
Recovering Databases....................................................................................33
Recovering Mailboxes...................................................................................36
Creating a Disaster Recovery Plan................................................................38
Appendix........................................................................................................40
Cluster Networking Requirements...................42
General Requirements.........................................................................................42
Geographically Dispersed Clusters.....................................................................43
Cluster Networking Best Practices...................45
Hardware Planning Recommendations..........................................................45
Network Interface Controller Configuration Recommendations..................45
Cluster Service Configuration Recommendations........................................47
Procedures for Implementing Best Practices......................................................48
Configuring Network Interface Controllers Prior to Configuring Cluster
Service............................................................................................................48
Configuring Cluster Network Properties after Configuring Cluster Service.....49
Windows 2000...............................................................................................49
- 40 -
Windows .NET Server...................................................................................49
References.....................................................51
White Papers........................................................................................................51
- 41 -
Cluster Networking Requirements
This section describes the requirements that server cluster puts on the network
infrastructure. These requirements must be met for the server cluster solution to
function correctly.
General Requirements
This section describes the requirements for any server cluster deployment.
• All of the adapters used to attach nodes to the same cluster network
must use the same communication settings – e.g. the same Speed,
Duplex Mode, Flow Control, and Media Type. If the adapters are
connected to a switch, the port settings of the switch must match those
of the adapters.
(Windows 2000, .NET server)
- 42 -
255.255.0.0. Addresses may be assigned to the nodes dynamically by
DHCP, but manual configuration with static addresses is recommended
(see section Cluster Networking Best Practices). The use of Automatic
Private IP Addressing (APIPA) to configure cluster networks is not
supported. APIPA is not designed for use with computers that are
attached to multiple networks.
(Windows 2000, .NET server)
- 43 -
• The nodes in a cluster may be on different physical networks; however,
the private and public network connections between cluster nodes must
appear as a single, non-routed LAN using technologies such as virtual
LANs (VLANs).
(Windows 2000, .NET server)
• As with LANs, each VLAN must fail independently of all other cluster
networks.
(Windows 2000, .NET server)
- 44 -
Cluster Networking Best Practices
This section describes network best practices for deploying a server cluster.
Hardware Planning
Recommendations
• Use identical NICs in all cluster nodes; that is, each adapter should be
the same make, model, and firmware version.
(Windows 2000, .NET server)
• Use static IP addresses for all nodes on the private network. Choose the
addresses from one of the following ranges.
• 10.0.0.0 - 10.255.255.255 (Class A network)
• 172.16.0.0 - 172.31.255.255 (Class B network)
• 192.168.0.0 - 192.168.255.255 (Class C network)
(Windows 2000, .NET server)
• Use static IP addresses for all nodes on all public networks. The use of
dynamic configuration via DHCP is not recommended. Failure to
renew a lease could disrupt cluster operation.
(Windows 2000, .NET server)
- 45 -
(Windows 2000, .NET server)
• The private LAN should be isolated. Only nodes that are part of the
cluster should be connected to the private subnet. Where there are
several clusters, using the same subnet for the private network for all of
the clusters is reasonable. You should not, however, put other network
infrastructure such as domain controllers, WINS server, DHCP servers
etc. on the private subnet.
(Windows 2000, .NET server)
• You should disable the default media sense policy for TCP/IP to ensure
that, if cables are disconnected or media sense is lost, the TCP/IP
configuration and corresponding cluster network configuration are not
torn down. Add the following registry value to each node:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\
Tcpip\Parameters
Value Name: DisableDHCPMediaSense
Data Type: REG_DWORD
Data: 1
(Windows 2000)
- 46 -
Cluster Service Configuration
Recommendations
• Set the private network role to Internal Cluster Communications Only.
Verify that the role for each public network is set to “All
Communications” (this is the default value).
(Windows 2000, .NET server)
- 47 -
Procedures for Implementing Best
Practices
Configuring Network Interface
Controllers Prior to
Configuring Cluster Service
• Configure the speed of each NIC as follows:
o Open Control Panel. Open Network Connections. Right-click on
the appropriate connection object and select Properties. Click
Configure, and then Advanced.
o Set the desired network speed using the drop-down list.
o Ensure that other settings, such as Duplex Mode, are also the same
for all adapters on a network.
• Configure the Internet Protocol settings of the private NIC as follows:
o Return to Network Connections. Open the Properties of the
appropriate connection object.
o Ensure that the Internet Protocol (TCP/IP) check box is selected.
o Highlight Internet Protocol and select Properties.
o Click the radio-button for Use the following IP address and enter a
static address.
o Ensure that there is no default gateway configured for the private
network.
o Verify that there are no values defined in the Use the following
DNS server addresses box. Click Advanced. On the DNS tab,
verify that there are no values defined. Make sure that the Register
this connection's addresses in DNS and Use this connection's DNS
suffix in DNS registration check boxes are cleared. Note that if the
cluster node is a DNS server, then IP address 127.0.0.1 will appear
in the list and should remain there.
• Configure the network connection order as follows:
o Return to Network Connections. Select Advanced. Select
Advanced Settings. In the Connections box, order the available
network connections as follows:
Public network(s)
Private network
Remote access connections
• Change the default names of the network connections as follows:
o Return to Network Connections. Right click on a network
connection object. Select Rename. Edit the name value.
o The name used for the connection object that represents a network,
such as the private network, should be the same on all nodes. If the
names of the connection objects are not the same, Cluster Service
will choose one and change the others to match it.
- 48 -
Configuring Cluster Network Properties
after Configuring Cluster Service
Windows 2000
When installing the cluster software, the Configuring Cluster Networks dialog
box will be presented for each network (in arbitrary order). For public network,
make sure name and IP address match network interface for public network.
Check the box "Enable this network for cluster use". Select the option "All
communications (mixed network)". For private network, make sure name and
IP address match network interface for private network. Check the box "Enable
this network for cluster use". Select the option "Internal cluster communications
only".
- 49 -
IPSec
Although it is possible to use Internet Protocol security (IPSec) for applications
that can failover in a server cluster, IPSec was not designed for failover
situations and we recommend that you do NOT use IPSec for applications in a
server cluster.
The primary issue is that Internet Key Exchange (IKE) Security Associations
(SAs) are not transferred from one server to the other if a failover occurs
because they are stored in a local database on each node.
The default time-out for the Security Association Idle Timer is 5 minutes, in the
event of a failover, clients will not be able to reestablish connections until at
least 5 minutes after all resources are online, using IPSec.
NetBIOS
In Windows .NET Server, the cluster service does not require NetBIOS,
however a number of services are affected if NetBIOS is disabled. You should
be aware of the following:
- 50 -
References
White Papers
• Deploying Microsoft Exchange 2000 Server Service Pack 2
Clusters
http://www.microsoft.com/exchange/techinfo/deployment/200
0/E2KSP2_Cluster.asp
- 51 -