Do you know how your Oracle RMAN backup works?

Recovery Manager or better known as RMAN, is an Oracle client utility that is installed with the
Enterprise or Standard edition, you can also find it with the Admin option when installing the Oracle
Client.

In it's most basic form, the RMAN client connects, an it needs to be with a "sysdba" privileged user, to
the database that is being backed up, called a TARGETdatabase, this client, is an executable that is
normally found in $ORACLE_HOME/bin.

By being a client side utility, it allows you to use one RMAN executable version to backup current and
previous versions of the Oracle Database, there are some restrictions to this though, which can be
verified in MOS document RMAN Compatibility Matrix [ID 73431.1].

This RMAN executable uses a file called recover.bsq , this file is located
in $ORACLE_HOME/rdbms/admin , basically what the executable does, is to interpret the commands you give
it , direct server sessions to execute those commands, and record its activity in the TARGET database
control file that is being backed up.

There are two main SYS packages that do the work of backup and recovery, which are DBMS_RCVMAN, this has
the procedures which list your database incarnations, the set until time recovery window, list your
backups, to name a few, and DBMS_BACKUP_RESTORE , which as you might have guessed is the one who does
the backup and recovery operations, like create the control file snapshot , backup the datafiles ,
backup the spfile to name some.

As mentioned above, the way that the RMAN client directs the server sessions to execute the commands are
through channels , a channel represents one stream of data to a device, and corresponds to one database
server session. The channel reads data into PGA memory, processes it, and writes it to the output
device.

The work of each channel, whether of type disk or System Backup Tape (SBT), is subdivided into the
following distinct phases:
1. Read Phase
A channel reads blocks from disk into input I/O buffers. The allocation of these buffers depend on the
number of datafiles being read simultaneously from disk and written to the same backup piece. One way to
control the numbers of files is the backup parameter FILESPERSET
2. Copy Phase
A channel copies blocks from input buffers to output buffers and performs additional processing on the
blocks, like the validation of the data blocks, as it verifies that it's not backing up corrupt data
blocks, it's also the phase where it does the binary compression and the backup encryption
3. Write Phase
A channel writes the blocks from output buffers to storage media. The write phase can be either to SBT
or to disk, and these are mutually exclusive, meaning you write to one or the other, not both.
As you can see by the phases above, and what distinguishes RMAN from any other method, is that the
backup is at the block level, as to the user managed backups, it brings great advantages , as it wont
have to backup empty blocks.

One great way to see how your backup works, is to do a trace, here you will see everything that has been
mentioned above

oracle@servidor1.oracleenespanol.blogspot.com [TESTDB] /home/oracle/bin

oracle $ rman target / debug trace=backup.trc
Recovery Manager: Release 11.2.0.3.0 - Production on Thu Dec 13 09:42:01 2012

Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

RMAN-06005: connected to target database: TESTDB (DBID=2580834116)

RMAN> backup database plus archivelog;

Once the backup finishes, you can search the trace file and see what it is actually doing, here is an example of the RCVMAN
fetching the incarnation information:

DBGRCVMAN: Fetching incarnation records

DBGRCVMAN: incarnation record id=0
DBGRCVMAN: icprs=0,icprc=
DBGRCVMAN: icrls=1,icrlc=Oct 24 2012 09:26:28
DBGRCVMAN: icrlc_i=797505988
DBGRCVMAN: icpinc=0
DBGRCVMAN: icflg=CURRENT
DBGRCVMAN: number of incarnation=1
DBGRCVMAN: canApplyAnyRedo is set to TRUE
DBGRCVMAN: canConvert_Cf is set to TRUE
DBGRCVMAN: canHandleTransportableTbs is set to TRUE

Below you can see the RPC calls being done by DBMS_BACKUP_RESTORE and the IO information by the INPUT buffers and the
OUTPUT buffers.

DBGRPC: krmxr - channel ORA_DISK_1 calling peicnt

DBGRPC: krmxrpc - channel ORA_DISK_1 kpurpc2 err=0 db=target proc=SYS.DBMS_BACKUP_RESTORE.BACKUPPIECECREATE excl: 0

DBGRPC: krmxrpc - channel ORA_DISK_1 kpurpc2 err=0 db=target proc=SYS.DBMS_BACKUP_RESTORE.PIECECONTEXTGETNUMBER excl: 0

DBGRPC: krmxrpc - channel ORA_DISK_1 kpurpc2 err=0 db=target proc=SYS.DBMS_BACKUP_RESTORE.BACKUPCANCEL excl: 0

DBGIO: Type %Comp Blocks Tot Blocks Blksize ElpTime(s) IO Rt(b/s) Name [09:43:14.946] (krmkqio)

DBGIO: ---- ----- ---------- ---------- ---------- ---------- ---------- ----- [09:43:14.946] (krmkqio)

DBGIO: IN 3.6 2304 64000 8192 0 0 /mount/u01/oracle/TESTDB/data/undotbs1_01.dbf [09:43:14.946]

(krmkqio)

DBGIO: IN 90.0 11520 12800 8192 0 0 /mount/u01/oracle/TESTDB/data/sysaux01.dbf [09:43:14.947] (krmkqio)

DBGIO: IN 51.9 33216 64000 8192 2 136052736 /mount/u01/oracle/TESTDB/data/system01.dbf [09:43:14.947] (krmkqio)

DBGIO: IN 59.7 38208 64000 8192 2 156499968 /mount/u01/oracle/TESTDB/data/users01.dbf [09:43:14.947] (krmkqio)

DBGIO: OUT 39224 8192 2 160661504

/mount/oracle/copy01/flash_recovery_area/TESTDB/TESTDB/backupset/2012_12_13/o1_mf_nnndf_TAG20121213T094259_8dmtcn0w_.bkp [09:43:14.947] (krmkqio)

DBGIO: AGG 85248 8192 2 349175808 [09:43:14.947] (krmkqio)

DBGMISC: ENTERED krmzlog [09:43:14.948]

RMAN-08044: channel ORA_DISK_1: finished piece 1 at Dec 13 2012 09:43:14

I hope this helps you understand more how your backup works, and this is just the super condensed version, there is a lot of
information missing, like how RMAN allocates the IN and OUT buffers, or how the disk writes are differentiated from the SBT
writes, so go out there and see it yourself.

Also some of the information for this entry came from these two great books:
 how rman works internally
RMAN backup and recovery operation for a target database are managed by RMAN client. RMAN uses the target
database control file to gather metadata about the target database and to store information about its own
operations. The RMAN client itself does not perform backup, restore, or recovery operations. When you connect
the RMAN client to a target database, RMAN allocates server sessions on the target instance and directs them
to perform the operations.The work of backup and recovery is performed by server sessions running on the
target database. A channel establishes a connection from the RMAN client to a target or auxiliary database
instance by starting a server session on theinstance.The channel reads data into memory, processes it, and
writes it to the output device.
When you take a database backup using RMAN, you need to connect to the target database using
RMAN Client.The RMAN client can use Oracle Net to connect to a target database, so it can be located on any
host that is connected to the target host through Oracle Net. For backup you need to allocate explicit or
implicit channel to the target database. An RMAN channel represents one stream of data to a device, and
corresponds to one database server session. This session dynamically collect information of the files from
the target database control file before taking the backup or while restoring.
For example If you give Backup database from RMAN, it will first get all the datafiles information from
the controlfile. Then it will divide all the datafiles among the allocated channels. ( roughly equal size of
work as per the datafile size ). Then it takes the backup in 2 steps. In the first step the channel will read
all the Blocks of the entire datafile to find out all the formatted blocks to backup. Note : RMAN do not take
backup of the un formatted blocks. In the second step it take backup of the formatted blocks. This is the
best advantage of using RMAN as it only take backup of the required blocks. Lets say in a datafile of 100 MB
size, there may be only 10 MB of use full data and rest 90 MB is free then RMAN will only take backup of those
10 MB

Thursday, 16 February 2017

How RMAN utility works internally ?

Recovery Manager or better known as RMAN, is an Oracle client utility that comes pre installed with the Enterprise or
Standard edition.
This RMAN executable uses a file called recover.bsq , this file is located in $ORACLE_HOME/rdbms/admin , basically
what the executable does, is to interpret the commands you give it , direct server sessions to execute those commands,
and record its activity in the TARGET database control file that is being backed up.

The way that the RMAN client directs the server sessions to execute the commands are through channels ,
a channel represents one stream of data to a device, and corresponds to one database server session. The channel
reads data into PGA memory, processes it, and writes it to the output device.
The work of each channel, whether of type disk or System Backup Tape (SBT), is subdivided into the following distinct
phases:

1. Read Phase
A channel reads blocks from disk into input I/O buffers. The allocation of these buffers depend on the number of data files
being read simultaneously from disk and written to the same backup piece. One way to control the numbers of files is the
backup parameter FILESPERSET
2. Copy Phase
A channel copies blocks from input buffers to output buffers and performs additional processing on the blocks, like the
validation of the data blocks, as it verifies that it's not backing up corrupt data blocks, it's also the phase where it does the
binary compression and the backup encryption

3. Write Phase
A channel writes the blocks from output buffers to storage media. The write phase can be either to SBT or to disk, and these
are mutually exclusive, meaning you write to one or the other, not both.

Architecture:

Tuesday, November 11, 2008

What Happens during a hot backup?

There are a couple of myths around the hot backup or online backup process:

Myth #1: The hot backup generates "a lot" of redo information
Myth #2: The archivelog mode "dramatically slows down" the database
Myth #3: When a hot backup is in progress the target datafile is frozen.

There are two ways to generate a hot backup, the first one is by a user managed backup and the second one with recovery
manager. The database is required to be in archivelog mode for it to be able to perform an online backup. Both ways to perform
an online backup work in a similar way, as I will further explain later, rman is more efficient than the user managed backup.

User managed backup

alter tablespace ts_name begin backup;

When the command is issued, at this point in time a checkpoint is performed against the target tablespace; then the datafile
header is frozen, so no more updates are allowed on it (the datafile header), this is for the database to know which was the
last time the tablespace had a consistent image of the data.

The datafiles with the backup in progress will still allow read/write operations just as a regular datafile, I/O activity is not
frozen.

Each time a row is modified, not only the row, but the complete block is recorded to the redo log file, this will only happen
the first time the block is modified, subsequent transactions on the block will only record the transaction just as normal.

During the user managed backup process the "fractured block" event may be present. Let's remember that the oracle block is
the minimum IO unit, and an oracle block is made out of several OS blocks; let's assume a block size of 8K and an OS block of
512b, this will give 16 OS blocks. If during the backup process of a block there is a write operation on the block then the backup
will contain a before image and an after image of the oracle block, the complete block in the backup media will be corrupt.
This is normal, consistency is not guaranteed on the backup, that is why the header must be frozen to mark the point where
the recovery process will have to start, and that is why oracle record a complete block image on the redo log file.

At the time the alter tablespace ts_name end backup; command is issued then the backup process is finished and the
datafile header resumes its regular IO activity.

Recovery manager backup

The same process happens when a rman backup takes place, the only difference is that rman better handles the fractured
block issue, it doesn't write block fragments or partial blocks to the backup, it writes the complete consistent block image to
the backup media. So recovery manager doesn't need to record the complete block to the redo log file.

Some further comments on the rman case, rman doesn't freeze the datafile header, it continues to checkpoint just as regular,
but it does perform a checkpoint to the tablespace.

From my perspective, the user managed backup (UMB) is a backup method that is less frequently seen on production
environments, since Oracle 9i Rel 2 most DBA's considered rman as part of the regular backup/recover strategy, it performs
better that the UMB, it is able to perform a block level backup, meanwhile in the UMB the whole datafile must be backup even
if a lot of clean blocks are present.

Some advices for the people who still use UMB, don't let the BEGIN BACKUP run for long periods of time, it is very likely that
the more time it takes to perform the backup, the more blocks are likely to change, which may generate more blocks contents
to be written to the redo log files.
Publi par Hector R. Madrid l'adresse 7:20 PM
Libells : Oracle Internals

6 comments:

Tom said...

I agree with some of your points and they could just be written wrong.

As to Myth#1. The Hot Backup itself does not generate a lot of redo. But if you have a lot of blocks that are
changed while the tablespace is in hot backup mode, each block will be copied the first time it is changed. This
is what can generate a lot of redo.

Myth#2. This can go back to Myth #1. If a lot of blocks are copied into the redo log files, they can fill up faster.
If they cannot be archived fast enough, it can cause things to "hang".

I agree with Myth #3. The data blocks are still marked dirty and flushed out by the DB Writer.

The SCN in the data file header is not updated until after the tablespace that those datafiles belong to come
out of hot backup mode. The current SCN's are still recorded in the redo. When you recover the tablespaces
datafiles, it knows what SCN was "safe" and it looks and recovers any blocks past that point that need recovery
using the archive log files that have the complete data blocks. You were mostly right and it could just be the
way it was written.

Great blog by the way!

Nov 11, 2008, 10:59:00 PM

Dan Norris said...

I don't think I agree with your last comment about RMAN. It doesn't use the same mechanism as user-managed
backup and the datafile header is not frozen and full blocks are not written to the redo logs during backup. You
might want to verify that and post some sources (even if it is your own testing) so your claims can be validated.

Nov 12, 2008, 7:51:00 AM

Hector Rivera Madrid said...

Thank you Dan for taking the time to leave a comment.

That is right, the datafile header is not frozen during the backup time with rman, I will emphasize this on the
original post. On the other way I didn't mention that rman writes the block contents to the redo log, all the
opposite, "... So recovery manager doesn't need to record the complete block to the redo log file". rman better
handles the fractured block issue, it writes a consistent block image to the backup media, so it doesn't require
to write the complete image to the backup media.

Nov 12, 2008, 6:08:00 PM

Hector Rivera Madrid said...

Thanks Tom for your comment.

Yes, the hotbackup in a user managed backup will generate additional redo information due to the block
information it has to record to the redo log files, but the amount of redo depends on the amount of blocks
being modified, and since the block is only written once during the hot backup session, in a worst case scenario
the amount of block dumped to the redo will be the same as the number of used blocks in a tablespace. So, it
depends ...

Regarding Mith#2, I once met a DBA who was able to convince the company of not using the archive log mode
because it caused a 'deadly slowdown' in database performance. When I took over the database I realize that
the root problem was a badly tuned redo mechanism.

Nov 12, 2008, 6:18:00 PM