Larry M. Carpenter
Distinguished Product Manager
Page 2 of 76
Data Guard
Hands On Lab
Oracle Database 11g Release 2
Maximum
Availability
Architecture
(MAA)
Oracle Best Practices For High Availability
Page 3 of 76
Page 4 of 76
INTRODUCTION.................................................................................................................................................... 7
OVERVIEW OF THE EXERCISES ..................................................................................................................... 9
SETUP AND CONFIGURATION ....................................................................................................................... 11
ORACLE DATA GUARD .................................................................................................................................... 13
CREATING A PHYSICAL STANDBY ......................................................................................................................... 14
Preparing the Primary Database..................................................................................................................... 14
Enabling Archiving ...................................................................................................................................... 14
Enable Force Logging .................................................................................................................................. 15
The Password File ........................................................................................................................................ 15
Adding Standby Redo Log Files .................................................................................................................. 16
Configuring OracleNet .................................................................................................................................... 17
Configure the Standby System ......................................................................................................................... 20
Creating the required directories .................................................................................................................. 20
Creating the Password File .......................................................................................................................... 21
Creating the Parameter File ......................................................................................................................... 21
Starting up the Standby Instance.................................................................................................................. 22
Creating the Physical Standby Database ........................................................................................................ 22
THE DATA GUARD BROKER ........................................................................................................................... 25
CONFIGURING THE BROKER .................................................................................................................................. 26
Broker Prerequisites ........................................................................................................................................ 26
Configuring the Listener .................................................................................................................................. 28
Create the Broker Configuration ..................................................................................................................... 31
Verifying the Broker Configuration ................................................................................................................. 33
Managing a Broker Configuration .................................................................................................................. 36
TRANSPORT SERVICES .................................................................................................................................... 43
MODIFYING THE TRANSPORT METHOD ................................................................................................................. 46
AUTOMATIC REDO GAP RESOLUTION ................................................................................................................... 48
PROTECTION MODES ....................................................................................................................................... 51
MAXIMUM PERFORMANCE .................................................................................................................................... 51
MAXIMUM AVAILABILITY..................................................................................................................................... 51
MAXIMUM PROTECTION ....................................................................................................................................... 51
CHANGING THE PROTECTION MODE ..................................................................................................................... 52
ROLE MANAGEMENT SERVICES .................................................................................................................. 53
SWITCHOVER ........................................................................................................................................................ 53
Performing a Switchover ................................................................................................................................. 53
FAILOVER ............................................................................................................................................................. 56
Enabling Flashback Database ......................................................................................................................... 56
Performing a Manual Failover ........................................................................................................................ 58
Performing a Manual Reinstate ....................................................................................................................... 60
FAST-START FAILOVER (FSFO)............................................................................................................................ 63
Enabling Fast-Start Failover ........................................................................................................................... 64
Executing an Automatic Failover .................................................................................................................... 67
CONCLUSION ...................................................................................................................................................... 71
Page 5 of 76
Page 6 of 76
Introduction
Oracle Data Guard ensures high availability, data protection, and disaster recovery for enterprise data. Data
Guard provides a comprehensive set of services that create, maintain, manage, and monitor one or more standby
databases to enable production Oracle databases to survive disasters and data corruptions. These standby
databases are maintained as transaction consistent copies of the production database.
If the production database becomes unavailable because of a planned or an unplanned outage, Data Guard can
switch any standby database to the production role, minimizing the downtime associated with the outage. Data
Guard can be used with traditional backup, restoration, and cluster techniques to provide a high level of data
protection and data availability.
A Data Guard configuration consists of one production database and one or more standby databases. The
databases in a Data Guard configuration are connected by Oracle Net and may be dispersed geographically.
There are no restrictions on where the databases are located, provided they can communicate with each other.
For example, you can have a standby database on the same system as the production database, along with two
standby databases on other systems at remote locations.
A standby database can be:
This handbook is your guide to the capabilities of Data Guard in Oracle Database 11g Release 2.
Page 7 of 76
Page 8 of 76
Note: While these exercises are run on a single system for simplicity this is NOT the best practice when
implementing Data Guard.
As you go through these exercises you will find the commands that must be done for each step at each line
beginning with the word TASK.
The exercises assume that you have multiple terminal windows set up for SQL*Plus command, DGMGRL
commands or Linux commands. Each TASK line in the book list the commands (SQL*Plus, RMAN, DGMGRL
or Linux) that are to be executed in one of the windows, depending on the type of command. Any DGMGRL
commands are to be executed in your DGMGRL terminal window, SQL commands in your SQL*Plus terminal
window, etc.
For the first exercise, Creating a Standby, the RMAN command to actually create the standby database at the
end of the section is long so there is one script in the appendix of this book called CreateNYC that you will be
directed to execute at the appropriate time. All of the tasks leading up to this step have to be executed first.
Page 9 of 76
Page 10 of 76
Page 11 of 76
Page 12 of 76
Administrators can chose either manual or automatic failover of production to a standby system if the primary
fails in order to maintain high availability for mission critical applications. For the purposes of this lab, we will
concentrate on Physical Standby databases (Redo Apply).
A Data Guard configuration includes a production database, referred to as the primary database, and up to 30
standby databases. Primary and standby databases connect over TCP/IP using Oracle Net Services. There are no
restrictions on where the databases are located provided that they can communicate with each other. A standby
database is initially created from a backup copy of the primary database. Data Guard automatically synchronizes
the primary database and all of its standby databases by transmitting primary database redo - the information used
by Oracle to recover transactions - and applying it to the standby database.
Page 13 of 76
Archive Mode
Enabled
USE_DB_RECOVERY_FILE_DEST
30
32
32
TYPE
----------string
big integer
VALUE
---------------/u01/app/fast_recovery_area
8G
TASK:
In the SFO SQL*Plus window execute:
o sql
o archive log list
o show parameter recovery
Page 14 of 76
TASK:
In the SFO SQL*Plus window execute:
o select force_logging from v$database;
o alter database force logging;
o select force_logging from v$database;
The Password File
Data Guard uses Oracle Net sessions to transport redo data and control messages between the members of a Data
Guard configuration. These redo transport sessions are authenticated using either the Secure Sockets Layer (SSL)
protocol or a remote login password file. Your Data Guard configuration is configured to use a remote login
password file. Every physical standby database in the configuration must have an up-to-date copy of the
password file from the primary database which will be copied to the standby database during the creation process
later in this exercise.
Note: Whenever you grant or revoke the SYSDBA or SYSOPER privilege or change the login password of a
user who has these privileges, you must replace the password file at each physical or snapshot standby database
in the configuration with a fresh copy of the password file from the primary database.
TASK:
No task as the Primary already has a password file.
Page 15 of 76
TASK:
In the SFO SQL*Plus window execute:
o alter database add standby logfile size 50M;
o alter database add standby logfile size 50M;
o alter database add standby logfile size 50M;
o alter database add standby logfile size 50M;
This will create 4 standby redo log files (SRL) at the Primary SFO in the directory specified by the parameter
create_file_dest, which in our case is the oradata area. In addition a second copy will automatically be created
in the Fast Recovery Area (FRA).
You can examine the standby redo log files in the v$logfile view where the TYPE is STANDBY.
Page 16 of 76
TASK:
In the SFO SQL*Plus window execute:
o select member from v$logfile where type=STANDBY;
Configuring OracleNet
To enable your Primary database to talk to your standby database you must define a tnsnames entry on the
Primary system that connects to the standby database. In addition, to enable your standby database to talk to your
Primary database you must define a tnsnames entry on the standby system that connects to the Primary database.
These tnsnames connection descriptors will be used by Data Guards Redo Transport mechanism to send redo
from the Primary and to resolve any gaps in the redo when there is a connection failure.
To accomplish this you would first run netmgr on your Primary system and add a tnsnames entry called NYC
pointing to your Standby system. You would then go to the Standby system and, again using netmgr, add a
tnsnames entry called SFO pointing to your Primary system. Since both of your databases will reside on the
same system in this lab you would only need to add the entry for NYC as SFO already exists.
Since we are using the LOCAL_LISTENER parameter in our databases it is also necessary to create tnsname
entries for our local listeners. These are called SFO_LISTENER and NYC_LISTENER.
Page 17 of 76
TASK:
At the Linux prompt execute the following command
o cat $ORACLE_HOME/network/admin/tnsnames.ora
NOTE: You will need to ensure that your TNSNAMES.ORA is setup correctly for your system.
Page 18 of 76
TASK:
At the Linux prompt execute the following command
o cat $ORACLE_HOME/network/admin/listener.ora
NOTE: You will need to ensure that your LISTENER.ORA is setup correctly for your system as above.
Page 19 of 76
mkdir
mkdir
mkdir
mkdir
/u01/app/oradata/NYC
/u01/app/fast_recovery_area/NYC
/u01/app/fast_recovery_area/NYC/archivelog
/u01/app/fast_recovery_area/NYC/onlinelog
TASK:
At the Linux prompt execute the following commands
o mkdir /u01/app/oradata/NYC
o mkdir /u01/app/fast_recovery_area/NYC
o mkdir /u01/app/fast_recovery_area/NYC/archivelog
o mkdir /u01/app/fast_recovery_area/NYC/onlinelog
Page 20 of 76
TASK:
Execute orapwd file=$ORACLE_HOME/dbs/orapwNYC password=oracle
Creating the Parameter File
Normally you would need to obtain a copy of the initialization parameter file from the Primary database and edit
it yourself before you could create the physical standby database. With the RMAN procedure you are going to
use this is no longer necessary. RMAN will obtain a copy of the Primary databases initialization parameter file
copy it over and edit it for you. You can also perform any additional parameter editing in the RMAN script
without the need for a text initialization parameter file. The resulting spfile will be placed in the default
$ORACLE_HOME/dbs directory.
To get your standby database created you do need to create a text initialization parameter file for the standby but
it only needs to contain the DB_NAME parameter and it does not matter what value that parameter has. For
example, you could cat a value for db_name into a temporary init.ora file.
cat > /u01/app/oracle/product/11.1.0/db_1/dbs/tempini.ora <<EOF
db_name=anyname
EOF
As with the password file, the script you will execute does this task for you.
TASK:
Execute the following:
cat > /u01/app/oracle/product/11.1.0/db_1/dbs/tempini.ora <<EOF
db_name=anyname
EOF
As you will see when you execute the script, very few parameters actually need to be set as the bulk of the Data
Guard specific parameters will be set for you in the next exercise Configuring the Data Guard Broker.
However it is possible to set all the necessary parameters in the single RMAN script if you so choose. An
example of this is in Appendix A at the end of this book.
Page 21 of 76
TASK:
Set your environment to NYC and execute the following:
sqlplus '/ as sysdba'
STARTUP PFILE='/u01/app/oracle/product/11.1.0/db_1/dbs/tempini.ora' NOMOUNT;
Page 22 of 76
TASK:
Execute the above script from the Linux prompt
You will notice that the only parameters that are expressly set in the standby database are the bare minimum
required parameters and that there are no Data Guard specific parameters being set. This is done so that all of the
Data Guard parameters get created when you create the Data Guard Broker configuration in the next exercise.
One thing to note is that the RMAN FROM ACTIVE DATABASE duplication method will place the SPFILE
of the standby in the normal $ORACLE_HOME/dbs directory. If you wish you can move it into ASM with the
CREATE SPFILE command in SQL*Plus if you are using ASM.
Page 23 of 76
At this point you have a standby database but it is not receiving any redo from the Primary nor is it ready to apply
any redo. This is due to the fact that we did not set any of the Data Guard parameters in the Standby or in the
Primary yet. You can verify that the Physical Standby database is in fact running as a standby with SQL*Plus.
bash-3.2$ . oraenv
ORACLE_SID = [NYC] ? NYC
The Oracle base remains unchanged with value /u01/app
bash-3.2$ sql
SQL*Plus: Release 11.2.0.2.0 Production on Thu Mar 24 16:48:40 2011
Copyright (c) 1982, 2010, Oracle.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit
With the Partitioning, OLAP, Data Mining and Real Application
You are logged in as sys with sysdba privileges!
Unique Name
Current Role
Open Mode
Protection Mode
---------------- ---------------- ------------ -------------------NYC
PHYSICAL STANDBY MOUNTED
MAXIMUM PERFORMANCE
TASK:
Execute the following commands in your NYC SQL*Plus window
o . oraenv (Enter NYC if it is not already the default)
o sql
To complete the process manually you could now log into the Standby as above and set all the Data Guard
parameters and start Redo Apply. After that you would return to the Primary and setup the same parameters
including the redo transport parameters. Once complete you would switch logs on the Primary and redo would
begin being sent to the Physical standby database. However, in our case we will be using the Broker so well
leave things as they are and let the Broker define everything for us in the next exercise.
You have successfully completed the Creating a Physical Standby Database exercise. The next step is to add in
a Data Guard Broker configuration and watch how the Broker takes care of all those parameters for you. No
need for more and more scripts!
Page 24 of 76
Page 25 of 76
Enterprise Manager enables historical trend analysis on the Data Guard metrics that it monitors - for example,
how the metrics performance has been in the last 24 hrs, or last 5 days, etc. Also, through Enterprise Manager, it
is possible to set up notification-alarms such that administrators may be notified in case the metric crosses the
configured threshold value. Use of Enterprise Manager for Data Guard requires that you enable the Broker.
For the rest of the exercises we will be using the Data Guard Broker.
Broker Prerequisites
The following conditions must be met before you can use the Broker:
1.
2.
3.
4.
5.
The primary and standby databases must be using the same version of Oracle Database
You must use a server parameter file (SPFILE)
Oracle Net Services network files must be set up on the primary database and on the standby database.
The value of the DG_BROKER_START initialization parameter must be set to TRUE
A service with a specific name must be statically registered with the local listener of each instance.
More information on these prerequisites can be found in Chapter 2 of the Data Guard Broker manual.
Page 26 of 76
The first 3 of the above prerequisites have already been met. The other two 2 prerequisites need to be met before
you can execute these commands and create your Data Guard Broker configuration. These two need to be met
for all Broker configurations.
To enable the Broker to create and manage the Data Guard configuration you must first enable the Data Guard
Broker background processes on each database in the configuration. This is done by setting the parameter
dg_broker_start to TRUE on both databases (SFO and NYC).
Before you enable the Broker you first need to check the placement of the Broker configuration files. These are
controlled by the parameters dg_config_file1 and dg_config_file2. These files default to the
$ORACLE_HOME/dbs directory and we will leave them there. On our standby they are as follows.
SQL> show parameter broker
NAME
TYPE
VALUE
---------------------- ----------- -----------------------------dg_broker_config_file1 string
/u01/app/oracle/product/11.2.0
/db_1/dbs/dr1NYC.dat
dg_broker_config_file2 string
/u01/app/oracle/product/11.2.0
/db_1/dbs/dr2NYC.dat
dg_broker_start
boolean
TRUE
For the Primary database SFO they would be in the same location but with SFO in the filename.
SQL> show parameter broker
NAME
TYPE
VALUE
---------------------- ----------- -----------------------------dg_broker_config_file1 string
/u01/app/oracle/product/11.2.0
/db_1/dbs/dr1SFO.dat
dg_broker_config_file2 string
/u01/app/oracle/product/11.2.0
/db_1/dbs/dr2SFO.dat
dg_broker_start
boolean
TRUE
Normally you would modify these parameters so that both copies of the files on each system in the configuration
reside in different directories for redundancy. And if the database is a Real Application Cluster (RAC) they then
must be modified so that the files are visible across the entire cluster as there can only be one copy of each file
for the entire RAC.
TASK: (Optional)
Execute the following commands in each SQL*Plus window
o show parameter broker
Page 27 of 76
TASK:
Execute the following command in the SQL*Plus window of SFO and NYC
o alter system set dg_broker_start=TRUE scope=both;
TASK:
Expand the configuration and select the LISTENER.
Page 28 of 76
TASK:
Select the Database Service drop down menu at the right.
TASK:
Click the Add Database button.
TASK:
Add Database 2 with SFO_DGMGRL as the name and SFO as the SID.
Page 29 of 76
TASK:
Click Add Database again
Add Database 3 with NYC_DGMGRL as the name and NYC as the SID
o Remember, if you were using two systems you would do this step on the Standby system
TASK:
Select File from the menu and click Exit.
TASK:
And save the configuration changes by clicking the Save button.
Page 30 of 76
1 instance(s).
UNKNOWN,
1 instance(s).
UNKNOWN,
TASK:
Execute the following command at the Linux prompt
o lsnrctl status
TASK:
Execute the following commands in the DGMGRL window
o dgmgrl sys/oracle@SFO
You must connect to the Primary database to create a configuration.
Page 31 of 76
TASK:
Execute the following commands in the DGMGRL window
o create configuration DG2011 as primary database is SFO connect identifier is SFO;
Next you add the standby database that we just created by specifying the tnsname for the standby. As with our
Primary you specify a tnsnames descriptor that includes all of the Standby instances. In our case the database is a
single instance database so our tnsname points to one host.
DGMGRL> add database NYC as connect identifier is NYC
> maintained as physical;
Database "nyc" added
DGMGRL>
TASK:
Execute the following commands in the DGMGRL window
o add database NYC as connect identifier is NYC maintained as physical;
At this point the configuration is complete, a Primary database with our one standby database. But up to now all
we have done is update the Broker configuration files at the Primary. Until we enable the configuration, nothing
will be written to the Standby Broker configuration files nor will any Data Guard parameters be created. So the
next step is to enable the configuration. Once this step is complete Data Guard will be in full operation.
TASK:
Execute the following command in the DGMGRL window
o show configuration;
At this point the Broker has defined all the parameters that you would normally have to do manually. In addition
it has started redo transport from the Primary to the Standby in asynchronous mode and it has started the Redo
Apply process at the standby.
You can verify that the redo is being shipped and applied by using SQL*Plus on the standby to select data from
the V$MANAGED_STANDBY view and display it, Primary first, standby second. In the output you will see
that there is an LNS process on the Primary database that is sending the current redo Sequence to our standby.
There would be multiple of LNS lines if there were multiple standbys.
SQL> select client_process,process,sequence#,status
2 from v$managed_standby;
CLIENT_P
-------ARCH
ARCH
ARCH
ARCH
LNS
PROCESS
SEQUENCE# STATUS
--------- ---------- -----------ARCH
43 CLOSING
ARCH
0 CONNECTED
ARCH
46 CLOSING
ARCH
47 CLOSING
LNS
48 WRITING
TASK:
Execute the following command in the SFO SQL*Plus window
o select client_process, process, sequence#, status from v$managed_standby;
Page 33 of 76
PROCESS
SEQUENCE# STATUS
--------- ---------- -----------ARCH
47 CLOSING
ARCH
46 CLOSING
ARCH
0 CONNECTED
ARCH
44 CLOSING
MRP0
48 APPLYING_LOG
RFS
48 IDLE
RFS
0 IDLE
RFS
0 IDLE
RFS
0 IDLE
TASK:
Execute the following command in the NYC SQL*Plus window
o select client_process, process, sequence#, status from v$managed_standby;
From this output you can see that the Primary is currently shipping sequence #48 and the standby is applying that
same sequence in real time apply mode. Real time apply is enabled because we created standby redo log files on
the primary database before we created the standby. Since they existed on the Primary, RMAN created them for
us on the Standby and the Broker automatically uses Real Time Apply when they exist.
You can examine the standby redo log files on NYC in the v$logfile view where the TYPE is STANDBY.
SQL> select member from v$logfile where type = 'STANDBY';
MEMBER
--------------------------------------------------------------/u01/app/oradata/NYC/onlinelog/o1_mf_4_6rqcggh4_.log
/u01/app/fast_recovery_area/NYC/onlinelog/o1_mf_4_6rqcgglm_.log
/u01/app/oradata/NYC/onlinelog/o1_mf_5_6rqcgnd7_.log
/u01/app/fast_recovery_area/NYC/onlinelog/o1_mf_5_6rqcgnhs_.log
/u01/app/oradata/NYC/onlinelog/o1_mf_6_6rqcgvmn_.log
/u01/app/fast_recovery_area/NYC/onlinelog/o1_mf_6_6rqcgvr3_.log
/u01/app/oradata/NYC/onlinelog/o1_mf_7_6rqch4d9_.log
/u01/app/fast_recovery_area/NYC/onlinelog/o1_mf_7_6rqch4hx_.log
8 rows selected.
TASK:
In the NYC SQL*Plus window execute:
o select member from v$logfile where type=STANDBY;
Page 34 of 76
TASK:
In the NYC SQL*Plus window execute:
o select group#, sequence#, status from v$standby_log;
So from the previous mixture of SQL*Plus and DGMGRL exercises you can see how the Broker simplifies
things as you only had to log in once and use the same window to examine both databases in your Data Guard
configuration. We will discuss Redo Transport and the various processes associated with it later.
Page 35 of 76
PRIMARY
TRANSPORT-ON
Properties:
DGConnectIdentifier
ObserverConnectIdentifier
LogXptMode
DelayMins
=
=
=
=
'sfo'
''
'ASYNC'
'0'
ReopenSecs
NetTimeout
RedoCompression
LogShipping
PreferredApplyInstance
ApplyInstanceTimeout
ApplyParallel
StandbyFileManagement
ArchiveLagTarget
LogArchiveMaxProcesses
LogArchiveMinSucceedDest
DbFileNameConvert
LogFileNameConvert
FastStartFailoverTarget
InconsistentProperties
InconsistentLogXptProps
SendQEntries
LogXptStatus
RecvQEntries
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
'300'
'30'
'DISABLE'
'ON'
''
'0'
'AUTO'
'MANUAL'
'0'
'4'
'1'
''
''
''
'(monitor)'
'(monitor)'
'(monitor)'
'(monitor)'
'(monitor)'
TopWaitEvents
= '(monitor)'
Database Status:
SUCCESS
TASK:
In the DGMGRL window execute:
o show database verbose sfo;
Page 36 of 76
PHYSICAL STANDBY
APPLY-ON
0 seconds
0 seconds
OFF
Properties:
DGConnectIdentifier
ObserverConnectIdentifier
LogXptMode
DelayMins
=
=
=
=
'nyc'
''
'ASYNC'
'0'
ReopenSecs
NetTimeout
RedoCompression
LogShipping
PreferredApplyInstance
ApplyInstanceTimeout
ApplyParallel
StandbyFileManagement
ArchiveLagTarget
LogArchiveMaxProcesses
LogArchiveMinSucceedDest
DbFileNameConvert
LogFileNameConvert
FastStartFailoverTarget
InconsistentProperties
InconsistentLogXptProps
SendQEntries
LogXptStatus
RecvQEntries
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
=
'300'
'30'
'DISABLE'
'ON'
''
'0'
'AUTO'
'MANUAL'
'0'
'4'
'1'
''
''
''
'(monitor)'
'(monitor)'
'(monitor)'
'(monitor)'
'(monitor)'
TopWaitEvents
= '(monitor)'
Database Status:
SUCCESS
TASK:
In the same DGMGRL window execute:
o show database verbose nyc;
Page 37 of 76
At this point, when the Broker performs the next health check it will report a parameter inconsistency between
what it think the value should be and what it actually is in the database.
TASK:
In the DGMGRL window execute:
o show configuration
Page 38 of 76
PROPERTY_NAME
LogArchiveMaxProcesses
SPFILE_VALUE
6
BROKER_VALUE
4
DGMGRL>
TASK:
In the DGMGRL window execute:
o show database sfo InconsistentProperties;
To clear the error you can either edit the Broker configuration to set the Broker property value to 6 or you can tell
the Broker to set it back by using an enable database command.
DGMGRL> enable database sfo;
Enabled.
DGMGRL> show configuration;
Configuration - dg2011
Protection Mode: MaxPerformance
Databases:
sfo - Primary database
nyc - Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS
TASK:
In the DGMGRL window execute:
o enable database sfo;
o show configuration;
Page 39 of 76
TASK:
Examine the SFO alert log output.
Likewise, any changes to the state of the transport or apply services must be performed with DGMGRL otherwise
you will see errors when you monitor your configuration. For example, stopping the redo apply services using
SQL*Plus instead of the Broker will result in an error state.
SQL> alter database recover managed standby database cancel;
Database Altered
TASK:
In the NYC SQL*Plus window execute:
o alter database recover managed standby database cancel;
Page 40 of 76
TASK:
In the DGMGRL window execute:
o show configuration;
A simple ENABLE DATABASE command will put things back the way they were supposed to be just like it
did with the parameter we set earlier. Or you can use the EDIT command to restart the apply services.
DGMGRL> edit database nyc set state=apply-on;
Succeeded.
DGMGRL>
TASK:
In the DGMGRL window execute:
o edit database nyc set state=apply-on;
Page 41 of 76
TASK:
Examine the NYC alert log output.
To recap, you should never make any changes to a Broker controlled Data Guard configuration using SQL*Plus.
These are called Out of Band changes and will always be flagged by the Broker as errors. To change a
parameter or the state of anything in the configuration you must always use the EDIT command of DGMGRL
or Enterprise manager. In the above case, to stop the redo apply services on NYC you would execute edit
database nyc set state=apply-off; from DGMGRL.
Now that you have created, managed and monitored your Data Guard configuration it is time to move on to
modifying the Redo Transport method and Protection Modes.
Page 42 of 76
Transport Services
Data Guard Redo Transport Services coordinate the transmission of redo from a primary database to the standby
database. At the same time that the primary database LGWR process is writing redo to its ORL, a separate Data
Guard process called the Network Server (NSS or NSA) is reading from the redo buffer in SGA and passing redo
to Oracle Net Services for transmission to the standby database. Data Guards flexible architecture allows a
primary database to transmit redo directly to a maximum of 30 standby databases. Data Guard is also well
integrated with Oracle RAC. Each primary instance that is active generates its own thread of redo and has its
own NSS/NSA process per standby database to transmit redo to that standby database.
Data Guard transport services transmit the redo to a standby database either synchronously or asynchronously,
where it is written to a standby redo log file (step one in the figure below).
The processes involved in redo transport are outlined in the next figure. Redo records transmitted by the
NSS/NSA are received at the standby database by another Data Guard process called the Remote File Server
(RFS). The RFS receives the redo at the standby database and writes it to a sequential file called a standby redo
log file (SRL). In a multi-standby configuration, the primary database has a separate NSS/NSA process that
manages redo transmissions for each standby database. In a configuration with three standby databases, for
example, three NSS/NSA processes are active on each primary database instance. As users commit transactions
at a primary database, Oracle generates redo records and writes them to a local online log file.
Page 43 of 76
Synchronous redo transport (SYNC) causes the primary database to wait for confirmation from the standby
database that redo has been hardened to disk before it will acknowledge commit success to the application providing zero data loss protection. Primary database performance is impacted by the sum of the time required
for the standby redo log file I/O to complete and network round-trip time.
Data Guard 11g Release 2 is designed to reduce the impact to primary performance of synchronous transport.
Redo is now transmitted to the remote standby in parallel with the local online log file I/O on the primary
database, effectively eliminating standby I/O from impacting total round trip time. This is shown in the figure
below as Step 1 where the NSS (Network Server Synchronous) process sends the redo at the same time the
LGWR is queuing the I/O to the Online Redo Log (ORL) file. Once the LGWR has the acknowledgement from
the disk I/O and the Standby round trip, Step 2, it will release the commit to the user, Step 3.
Page 44 of 76
Note: The asynchronous transport method is able to stream redo to a standby database so efficiently with no
impact to the Primary database that the old ARCH method of shipping redo was deprecated in Oracle Database
11g and the default transport mode for Data Guard is ASYNC.
Page 45 of 76
TASK:
In the DGMGRL window execute:
o show database sfo logxptmode;
o show database nyc logxptmode;
Since SFO is the Primary this property just shows how redo will be sent to SFO when it becomes a Standby. And
the property for NYC, our standby database, shows how the redo is currently being sent to NYC from SFO. You
can verify this by looking at the LOG_ARCHIVE_DEST_n parameters as they are currently defined on SFO.
SQL> show parameter 'log_archive_dest_2'
log_archive_dest_2 string
TASK:
In the SFO SQL*Plus window execute:
o show parameter log_archive_dest_2
Page 46 of 76
TASK:
In the DGMGRL window execute:
o edit database sfo set property logxptmode=SYNC;
o edit database nyc set property logxptmode=SYNC;
This change to NYC will become immediately visible on the Primary SFO.
SQL> show parameter 'log_archive_dest_2'
log_archive_dest_2 string
You can see the results of the action by examining the alert log file of the Primary SFO.
Thu Mar 31 09:48:16 2011
ALTER SYSTEM SET log_archive_dest_2='service="nyc"','LGWR SYNC
AFFIRM delay=0 optional compression=disable max_failure=0
max_connections=1 reopen=300 db_unique_name="nyc"
net_timeout=30','valid_for=(all_logfiles,primary_role)' SCOPE=BOTH;
ALTER SYSTEM SWITCH ALL LOGFILE start (SFO)
Thu Mar 31 09:48:16 2011
Destination LOG_ARCHIVE_DEST_2 is SYNCHRONIZED
TASK:
Examine the SFO alert log output.
Changing the transport mode to SYNC is the first step to configuring a Zero Data Loss (ZDL) Data Guard
configuration but by itself will not provide ZDL. You must also modify the Protection Mode, something we will
do right after we discuss the last part of Transport Services, Automatic Redo Gap Resolution.
Page 47 of 76
Once the standby becomes available again, the ARCH process that has been designated as the Standby Ping
Process will discover it and, as long as the specified reopen time (specified by the ReopenSecs property) has
passed, immediately begin resolving any missing archive log files that were not sent to that standby. In addition
it will force a log switch which will get the NSS/NSA process for that standby restarted and connected to the
standby so that the current redo stream can also begin to be shipped to the standby.
Page 48 of 76
TASK:
In the DGMGRL window execute:
o edit database sfo set property reopensecs=15;
o edit database nyc set property reopensecs=15;
Now, to create the gap we will abort our standby NYC and switch logs twice at the Primary SFO.
SQL> shutdown abort
ORACLE instance shut down.
TASK:
In the NYC SQL*Plus window execute:
o shutdown abort
Then switch logs twice at the Primary database.
SQL> select sequence# from v$log where status like 'CURRENT%';
SEQUENCE#
---------86
SQL> alter system switch logfile;
System altered.
SQL> alter system switch logfile;
System altered.
SQL>
SEQUENCE#
---------88
SQL>
Page 49 of 76
TASK:
Examine the NYC alert log output.
Now that you understand how Redo transport works we will finish our modifications and change the Protection
Mode of our configuration to a Zero Data Loss mode.
Page 50 of 76
Protection Modes
Data Guard provides three modes of data protection to balance cost, availability, performance, and data
protection. Each mode uses a specific redo transport method, and establishes rules that govern the behavior of the
Data Guard configuration should the primary database ever lose contact with its standby. The following table
outlines the characteristics of each mode.
DATA GUARD PROTECTION MODES
MODE
TRANSPORT
IF NO
ACKNOWLEDGEMENT
FROM THE STANDBY
DATABASE, THEN:
Maximum Protection
SYNC
Maximum Availability
SYNC
Maximum Performance
ASYNC
Data Guard protection modes implement rules that govern how the configuration will respond to failures,
enabling you to achieve your specific objectives for data protection, availability, and performance. Data Guard
can support multiple standby databases in a single Data Guard configuration, and they may all have the same, or
different, protection mode setting, depending on your requirements. The different Data Guard protection modes
are Maximum Performance, Maximum Availability, and Maximum Protection.
Maximum Performance
This mode emphasizes primary database performance over data protection. It requires ASYNC redo transport so
that the LGWR process never waits for acknowledgement from the standby database. Primary database
performance and availability are not impacted by redo transport, by the status of the network connection between
primary and standby, or by the availability of the standby database.
Maximum Availability
This mode emphasizes availability as its first priority and zero data loss protection as a very close second priority.
It requires SYNC redo transport, thus primary database performance may be impacted by the amount of time
required to receive an acknowledgement from the standby that redo has been written to disk. SYNC transport,
however, guarantees 100-percent data protection during normal operation in the event that the primary database
fails. However, events that have no impact on the availability of the primary database can impact its ability to
transmit redo to the standby.
Maximum Protection
As its name implies, this mode places utmost priority on data protection. It has the same requirements as
Maximum Availability but the primary will not acknowledge a commit to the application unless it receives
acknowledgement from at least one standby database in the configuration that the data needed to recover that
transaction is safely on disk. If the primary does not receive acknowledgement from a SYNC standby database, it
will stall and eventually abort, preventing any unprotected commits from occurring.
Page 51 of 76
Page 52 of 76
Switchover
A switchover is a planned operation used to reduce downtime during planned maintenance, such as operating
system or hardware upgrades, rolling upgrades of the Oracle database, and other database maintenance.
Regardless of the transport service (SYNC or ASYNC) or protection mode utilized, a switchover is always a zero
data loss operation.
Performing a Switchover
If you were not using the Broker then a switchover would be a combination of several commands on both
databases and a manual database restart of the original primary. The order of the process would be as follows:
1. On the Primary database
a. Shutdown all but one instance of the Primary
b. alter database commit to switchover to standby with session shutdown;
c. shutdown immediate
2. On the Standby Database
a. alter database commit to switchover to primary;
b. alter database open;
3. On the original Primary (now a standby)
a. startup mount
b. alter database recover managed standby database using current logfile disconnect;
c. Startup the other instances
The Broker simplifies all of that process for you down into on command switchover to <database name>;
DGMGRL> show configuration;
Configuration - dg2011
Protection Mode: MaxAvailability
Databases:
sfo - Primary database
nyc - Physical standby database
Fast-Start Failover: DISABLED
Configuration Status:
SUCCESS
Page 53 of 76
TASK:
In the SFO SQL*Plus window execute:
o select db_unique_name, database_role from v$database;
o connect sys/oracle as sysdba
o select db_unique_name, database_role from v$database;
To return to our original configuration, perform a 2nd switchover and make SFO the Primary database again.
DGMGRL> switchover to sfo;
Performing switchover NOW, please wait...
New primary database "sfo" is opening...
Operation requires shutdown of instance "NYC" on database "nyc"
Shutting down instance "NYC"...
ORACLE instance shut down.
Operation requires startup of instance "NYC" on database "nyc"
Starting instance "NYC"...
ORACLE instance started.
Database mounted.
Switchover succeeded, new primary is "sfo"
TASK:
In the DGMGRL window execute:
o switchover to sfo
Note: Dont forget to reconnect your NYC SQL*Plus window!
Page 55 of 76
Failover
A failover brings a standby database online as the new primary database during an unplanned outage of the
primary database. As with a switchover, a failover operation does not require the standby database to be restarted
in order to assume the primary role. Also, as long as the database files on the original primary database are intact
and the database can be mounted, the original primary can be reinstated and resynchronized as a standby database
for the new primary using Flashback Database it does not have to be restored from a backup.
Manual failover is initiated by the administrator using the Oracle Enterprise Manager GUI interface, the Data
Guard Brokers command line interface, or directly through SQL*Plus. Optionally, Data Guard can perform
automatic failover in a very controlled manner using Fast-Start Failover (FSFO).
TASK:
In the both the SFO and NYC SQL*Plus windows execute:
o show parameter db_flashback_retention_target
o alter system set db_flashback_retention_target=60 scope=both;
Page 56 of 76
TASK:
In the SFO SQL*Plus window execute:
o select flashback_on from v$database;
o alter database flashback on;
o select flashback_on from v$database;
However, the standby redo apply services must be stopped first on the standby before you can enable flashback
database. DGMGRL has a SQL command capability but it requires that your session is connected to the database
where you wish to execute the SQL command. Since we want to change the standby we will reconnect to NYC.
DGMGRL> connect sys/oracle@NYC
Connected.
DGMGRL> edit database nyc set state=apply-off;
Succeeded.
DGMGRL> sql "alter database flashback on";
Succeeded.
DGMGRL> edit database nyc set state=apply-on;
Succeeded.
DGMGRL>
TASK:
In the DGMGRL window execute:
o connect sys/oracle@NYC
o edit database nyc set state=apply-off;
o sql alter database flashback on;
o edit database nyc set state=apply-on;
Page 57 of 76
Now we are ready to perform a failover and force NYC to become the Primary database.
Panic ensues! Production is down! Never fear, Data Guard is here! You can examine the state of your
configuration with the usual show configuration command. The error will be a TNS error because the Broker
has lost its connection with the Primary database. Remember, in the last exercise we reconnected our DGMGRL
session to the standby NYC. If you forgot to do that then you will not be able to examine the state of the
configuration since SFO is down!
Page 58 of 76
TASK:
In the NYC SQL*Plus window execute:
o select db_unique_name,database_role from v$database;
Page 59 of 76
Page 62 of 76
Page 63 of 76
TASK:
In the DGMGRL window execute:
o edit database sfo set property FastStartFailoverTarget = nyc;
o edit database nyc set property FastStartFailoverTarget = sfo;
We will also set the FastStartFailoverThreshold to enable our failover to happen faster. The actual setting for
this property depends on your configuration (if the Primary is a RAC you have to add cluster miscount and
reconfiguration time to the threshold) and how long you want the Primary to stall before aborting and a failover
occurring. There are other properties that relate to FSFO but well just accept the defaults for them.
TASK:
In the DGMGRL window execute:
o edit configuration set property FastStartFailoverThreshold=30;
Page 64 of 76
30 seconds
nyc
(none)
30 seconds (not in use)
TRUE
TRUE
YES
YES
NO
NO
YES
Page 65 of 76
30 seconds
nyc
ip-10-124-233-12
30 seconds (not in use)
TRUE
TRUE
YES
YES
NO
NO
YES
TASK:
In the DGMGRL window execute:
o show fast_start failover;
As you can see there are other properties you can set for a FSFO configuration that we have not explicitly set.
Refer to the Data Guard Broker manual for more information on these properties.
We are now ready for the last exercise. An Automatic Failover!
Page 66 of 76
That will trigger a failover to NYC since the Broker (the Observer) recognizes immediately that our last
surviving Primary instance has crashed. It will wait until our threshold has passed and then it will make NYC the
Primary database automatically!
Page 67 of 76
TASK:
In the OBSERVER window
o Watch the Observer performing the failover.
Returning to the DGMGRL window you can see the current state of the configuration, NYC as the Primary and
SFO needing reinstatement.
DGMGRL> show configuration
Configuration - dg2011
Protection Mode: MaxAvailability
Databases:
nyc - Primary database
Warning: ORA-16817: unsynchronized fast-start failover
configuration
sfo - (*) Physical standby database (disabled)
ORA-16661: the standby database needs to be reinstated
Fast-Start Failover: ENABLED
Configuration Status:
WARNING
TASK:
In the DGMGRL window execute:
o show configuration;
This is exactly the same state we were in when we performed our manual failover, NYC is the Primary and SFO
needs to be reinstated. The difference is (and it is a big one) that the Primary cannot open read write no matter
how it is started and the Observer will automatically reinstate it as a standby when it does restart.
Page 68 of 76
TASK:
In the OBSERVER window
o Watch the Observer performing the reinstate.
Page 69 of 76
TASK:
In the DGMGRL window execute:
o show configuration;
Success! You can leave things like they are and move production back to SFO when NYC fails or you can
perform a switchover to move production from NYC back to SFO. Of course you could also do a SHUTDOWN
ABORT on NYC and let the Observer move production to SFO.
This ends our exercises with Data Guard but feel free to redo or play around with your configuration.
Page 70 of 76
Conclusion
This concludes the exercises for the Oracle Database 11g Data Guard Hands On Lab. During this session you
have explored and exercised Data Guards unique capabilities for protecting your data. You were able to create,
manage and use Data Guard standby databases in a variety of situations.
You have completed the following exercises.
It is our sincere hope that these exercises have helped to guide you on your path to using Data Guard and
understanding how to use the features that it brings to the Oracle Database world to provide you with Disaster
Protection for your Oracle data.
Thank you for your participation.
Larry M. Carpenter
Page 71 of 76
Page 72 of 76
Resources
The following documentation is available for Data Guard
Oracle Data Guard Concepts and Administration 11g Release 1 (11.1)
o http://download.oracle.com/docs/cd/B28359_01/server.111/b28294/toc.htm
Oracle Data Guard Broker 11g Release 1 (11.1)
o http://download.oracle.com/docs/cd/B28359_01/server.111/b28295/toc.htm
Oracle Data Guard Concepts and Administration 11g Release 2 (11.2)
o http://download.oracle.com/docs/cd/E11882_01/server.112/e10700/toc.htm
Oracle Data Guard Broker 11g Release 2 (11.2)
o http://download.oracle.com/docs/cd/E11882_01/server.112/e10702/toc.htm
The Maximum Availability Architecture Best Practices papers are on Oracle OTN.
Best Practices for High Availability -- Maximum Availability Architecture (MAA)
o http://www.oracle.com/goto/maa
Best Practices for Data Guard
o http://www.oracle.com/goto/dataguard
Page 73 of 76
Page 74 of 76
Page 75 of 76
Page 76 of 76