Training Guide
R01.1
ii
Copyright
1999-2009 CommVault Systems, Inc. All rights reserved.
CommVault, CommVault and logo, the CV logo, CommVault Systems, Solving Forward,
Management, QiNetix, Quick Recovery, QR, CommNet, GridStor, Vault Tracker, InnerVault,
QuickSnap, QSnap ,Recovery Director, CommServe, CommCell and ROMS are trademarks or
registered trademarks of CommVault Systems, Inc. All other third party brands, products, service
names, trademarks, or registered service marks are the property of and used to identify the
products or services of their respective owners. All specifications are subject to change without
notice.
TABLE OF CONTENTS
Introduction Module..................................................................................................................... 7
Agenda ........................................................................................................................................ 9
Certification Program................................................................................................................ 10
ROMS ....................................................................................................................................... 12
Standard Terminology .............................................................................................................. 13
CommServe............................................................................................................................... 14
CommCell Console ................................................................................................................... 15
Media Agent.............................................................................................................................. 16
Index ......................................................................................................................................... 17
Data Paths ................................................................................................................................. 18
Libraries .................................................................................................................................... 19
Client Agent .............................................................................................................................. 20
CommCell ................................................................................................................................. 22
Training Environment ............................................................................................................... 23
Introduction
www.commvault.com/training
Introduction Module
User Administration
Library Management
Disaster Recovery
Media Management Disaster Recovery Planning
Storage Policies Risk Reduction
Clients CommCell Recovery
Job Management Client Data Recovery
Practicing Disaster Recovery
Restore
Monitoring Archiving & Content
Indexing
Migration Archiving
Compliance Archiving
Content Indexing
Search & Discovery
Agenda
Certification
Practice in Class
Available online after class
Must be registered
Must have signed roster
Certification Program
Certification is a valuable investment for both a company and the IT professional. Certified
personnel can increase a company's productivity, reduce operating costs, and increase potential
for personal career advancement.
Key Points
Certification is integrated with and managed through CommVault's Online Registration system.
Cost of certification registration is included in the associated training course.
Practice assessments are given in class at the end of each module.
Students may take the online certification exam(s) any time after completing the course.
Previous training course students (as validated by the registrar) can also request an opportunity to
take the online assessment exam at no charge.
For those that feel they do not require training, an online assessment opportunity for each
certification level may be purchased separately from the training course.
To access your free certification exam please follow the instructions below:
1. To start the session, log into the CommVault Online Registration system by visiting
www.commvault.com/training. Then click on one of the course offering links located on the
right-hand side of the web page (e.g. US Course Offerings & Registration); then click on the My
Account link.
2. Enter the user name and password that you used when you registered. If you've forgotten your
user name, password, or both, then click on the Forgot username/password link. The system will
automatically email your login information.
3. Once you've logged in you will see your User Profile and all of your current registration where
you'll find the registration record for the online exam. If the link isnt under the registration tab,
try clicking on the past tab (2nd tab in).
4. Click on the Certification Exam link to launch the exam. Note: You must have pop-ups
enabled as the exam is launched in its own browser window.
Note: Allowed a single exam session - You must complete the entire exam at once without
closing your browser.
ROMS
Licensed on the CommServe, Media Agents, and selectively for each client system
Available as an extension to active Maintenance contracts
CommVault ROMS makes it easy to monitor and track alerts across your entire CommCell
deployment, in real-time from a single centralized dashboard. Call home remote alerting
virtually adds the CommVault on-call services team to your staff, for 24x7 help and problem
resolution.
CommVault ROMS provides an immediate health-check of your environment, and points out
where problems are developing before they cause serious disruption. Outages are found and
alerted in real-time, for immediate resolution and resumption of data management activities.
Standard Terminology
CommServe
CommCell Console
Media Agent/Index/Data Paths
Libraries
Client Agent
Standard Terminology
CommServe
Maintains CommCell
configuration info and SQL DB
high level index
Manages user Security
and Module Licensing
Manages all jobs
S e c u r it y
Supported on Windows
Server platforms
J o b C o n t ro l
C o m m C e ll C o n s o le
CommServe
A single Storage Manager, or CommServe (as it is more commonly referred to), provides
administrative and management oversight of all tasks and data movement within a CommCell.
The CommServe functions as the command and control center. All data protection and recovery
activity is managed there. In addition, user security and all Simpana module licensing is handled
by the CommServe.
The CommServe also houses the meta-database catalog. Loosely translated, the meta-database
contains the data about the data. That would include component configuration information,
history, and location of all data backed up within the CommCell.
Three major sub-components within the CommServe are the Job Manager, Scheduler, and the
Event Manager. The Job Manager monitors and controls all the scheduled activity of the
software and provides Galaxy with its restart capabilities. The Scheduler allows users to schedule
tasks which are then initiated by the Job Manager. The Event Manager logs all events, providing
unified notification of important events.
The CommServe can only be installed on a Windows Server platform. It may reside on its own
dedicated system (recommended), or a system that also contains a MediaAgent and/or client
modules. The system requirements are found in the Online Documentation or on the
CommVault Systems Website.
CommCell Console
Summary
Job Controller
CommCell Browser
Event Viewer
CommCell Console
The CommCell Console is the graphical user interface (GUI) used for management of a
CommCell. The Console has a menu, tool, and status bar along with a multi-window component
area for management tools. The four major management tools have their own windows within
the Console. They are the CommCell Browser, Summary, Job Controller, and Event Viewer.
Job Controller
Active Jobs are displayed and managed in the Job Controller window. The administrator can
track job progress, view statistics, events, and logs, and kill, suspend, or resume jobs with the Job
Controller.
Event Viewer
The event viewer provides a real-time display of all relevant events in the CommCell. By
default, 10,000 events are logged and the most recent 200 are displayed. The events of all jobs
can be reviewed, examined for details, or filtered by severity, job id, and time. The administrator
can also do character pattern searches.
Media Agent
Index
Cache
Writes/reads data
to/from data storage
Maintains Index
Cache space
Supported on
Windows, UNIX,
Linux, and Netware
systems
Data Storage
Media Agent
The Media Agent is the workhorse for all data movement. It facilitates the transfer of data
between the data source (i.e. client computer) and the destination storage media. It is supported
on Windows, UNIX, Linux, and Netware systems.
Each Media Agent manages the reading/writing activity of all attached storage devices. Storage
devices can be directly attached (e.g. SCSI) or remotely attached via Storage Area Network
(SAN) or Local Area Network protocols. (e.g. Fibre Channel, iSCSI, TCP/IP, CIFS, NFS)
The Media Agent software is designed to be storage-media independent. This gives the Media
Agent the capability of supporting a wider variety of storage models. This approach allows an
organization to rapidly adapt to changes in storage technology.
By default, the Index Cache is stored on the Media Agent. This is known as a dedicated index
cache configuration. It can also be shared between multiple Media Agents. The Index Cache
is a working space used to read/write object level indexes for granular restore/recall capability.
Index
Dual Level for Scalability and
Performance
Level I
Maintained by CommServe Job summary
Tracks jobs to media data
Level II
Maintained by Media Agent Index
Tracks objects within job Cache
Cached for performance
Copied to media for scalability Media Agent
Index
written
to
Index
The major function of the index cache is to provide quick access to indexed data on the media
during a browse\restore operation. Whenever a new data protection job starts, the Simpana
software will generate a list of all the data objects (e.g. files/subdirectories, database objects,
mailbox objects, etc.) that will be backed up from the source server. This list, along with the
path to the actual archive file that stores the data on the media, is called the index.
The index is stored in the index cache which is a directory accessible by the MediaAgent. Each
new index will generate a new subfolder under the index cache. A copy of each index is also
written (archived) to the storage media as part of the corresponding data protection job.
If a browse or restore operation cannot locate the index in the index cache, it will be
automatically retrieved from the storage media. The index remains in the cache until it exceeds
the user-defined retention criteria or the disk usage limits, configured through the MediaAgent,
are reached. Likewise, the archived index copy will remain on the storage media until it too
exceeds the required data retention criteria.
Two index cache configurations are available. A dedicated index cache is accessible by only
one Media Agent and a shared index cache that is accessible from multiple Media Agents.
Data Paths
SD
0 2 4 6 8 10 ! !
0 1 2 3 4 5 6 7 8 9
1 3 5 7 9 11
PowerVault
650F
Media Agent
Client Magnetic RAID/JBOD
Data Paths
During a data protection operation, or backup, the source server sends data to the Media Agent
for archiving on the media. During a data recovery operation, or restore, the Media Agent
accesses data on the storage media and sends it to the destination server. The connection set up
between the Media Agent, source or destination server, is called a Data Pipe or what is more
commonly referred to as a Data Stream.
A Data Stream is CommVaults patented high performance data mover used to move data back
and forth between a data source and a Media Agent or between two Media Agents. It consists of
a shared memory component on each system for multi-thread processing with multiple
connections between the components. It handles all data encryption, compression, and disk I/O.
When provided with sufficient resources (e.g. bandwidth, CPU, disk I/O), a single data stream
can stream multiple high performance tape drives across a network at optimum speed.
Simultaneous data streams can be run from one or more data sources to one or more Media
Agents at the same time. The data stream will make use of all the available bandwidth unless
throttled back using controls built in to the Client software.
Libraries
Media Container
Magnetic Tape
0
0 2 4
1 3 5
1
6 8 10
7 9 11
2 3 4 5 6 7 8 9
! !
SD
Direct-Attached,
PowerVault
650F
Shared, Dynamic
PowerVault
128T
(SAN), or IP-based
Magnetic RAID/JBOD Tape Library
Tape, Optical (MO,
UDO, DVD), PnP or
Mount Paths Master Drive Pool Magnetic disk
D:\CV_Magnetic Robotic or manual
Drive Pool
\\ServA\CV_Magnetic Barcode-ed or Blind
Writer
(default - 5 per mount
path)
Libraries
Libraries are hardware or software entities that manage a set of storage devices and media (e.g.
tape drives and tapes or magnetic disk storage). Even a single removable media drive or,
standalone, is managed as a library with a tracked set of media.
Access and control of all Libraries in a CommCell is managed by the Media Manager Service on
the CommServe. Local access and control is handled by the Media Agents. Depending on the
storage environment and the data protection requirements, a single Media Agent can be
configured to access multiple libraries. Likewise, multiple Media Agents can be configured to
access a single library.
All protected data, regardless of a backup or restore operations, must pass through a Media
Agent and a library. A list of currently tested devices and configurations is available on
CommVaults Maintenance Advantage website.
Client Agent
Client Agent
iDataAgent
The Intelligent DataAgent (iDataAgent) is the software module that manages the data transfer
to/from the protected storage media through the Media Agent. An iDataAgent is specific to the
data type it manages (e.g. File System, Database, Mailbox, etc.) and is required for each
managed data type per client system. You can have multiple iDataAgents on the same client
system.
Client
A client is the host system that provides the resources necessary for an iDataAgent to access the
data. Clients can be either physical or virtual machines on the network, or virtual machines in a
clustered environment.
Simpana supports multiple client operating systems at various version and service pack levels.
Consult CommVaults latest documentation and release notes for which OS, version, and
service pack level is supported for your data center.
Instance
The Instance level is only present for database iDataAgent s. Many database applications allow
for the existence of multiple instances on the same client. Each instance can be managed
separately to meet any unique requirements.
BackupSet or ArchiveSet
The BackupSet is a logical entity that defines a complete, separate, and recoverable view of the
data. In the case of database iDataAgents, it could be an individual database (SQL) or tablespace
(Oracle). In the case of File System iDataAgents, Simpana creates one default backupset for all
file system data. Normally you would only require one backupset. If redundant access to the
data is required, additional backup sets could be created.
SubClient
A subclient is a logical entity used to divide the data within the backupset. For most (not all)
installed iDataAgents, Simpana creates a default subclient that automatically includes all the data
for that type iDataAgent. Additional subclients may be defined with a distinct range of content
for separate data management and control.
Multiple subclient content within a backupset is mutually exclusive meaning content defined by
one subclient will not be backed up by another subclient. The Default subclient has the unique
characteristic of including all existing and new content not defined by other subclients.
Multiple subclients allow you to distinguish data for different and business appropriate data
protection management
CommCell
CommCell
A CommCell is the management boundary for the CommServe, all Media Agents and
iDataAgents. All command and control functions of the CommServe and other components are
defined within the CommCell. All Simpana products operate within a single CommCell with the
exception of Simpana Monitor which operates at a higher level and can interact with multiple
CommCells. The CommCell recognizes all libraries, devices, and media as unique entities.
Membership of any component within a CommCell, along with its data, can only be changed to
another CommCell via a CommCell migration process.
Training Environment
Training Environment
User Administration
www.commvault.com/training
Overview
Overview
Objectives
List and describe the prescribed means to secure and access the CommCell
Management Interface.
Understand the purpose for CommCell User Group and how they are created and
managed.
Create a CommCell User group with capabilities and associate it to an Object for
administration.
Understand how external Active Directory Users and User Groups are integrated into
a CommCell environment.
List and describe common CommCell Management Interface settings and options for
basic administration tasks.
CommCell Console
Access Options
Using Java Web Start, which allows access to the CommCell console remotely via
any computer running a supported platform with a java-enabled platform. This
allows remote CommCell Console sessions to be active, so multiple users can access
the CommServe system simultaneously.
Web-based
During the installation of the CommServe component - if the Internet Information Server (IIS) is
installed and running on the same host - you are offered an option to configure the CommServe
system for web administration. Note that CommServe web administration is only possible with
IIS.
If the CommServe computer does not have IIS installed, or was not configured for web
administration during installation, you can at anytime enable local web administration or
configure web administration via an alternate IIS host. Note that whatever IIS host is used it
must have the CommCell Console installed. If using an IIS host different than the CommServe
host, the CommCell Consoles e-mail and IIS Server applet must be configured to use the
alternate IIS host. Once configured for web administration, any java enabled browser can be used
to access the Java applet and perform remote administration.
The web-based version of the CommCell Console has the same appearance and functionality as
the installable CommCell Console Java application.
Note that all login sessions where a password is required are done using Secure Socket Layer
(SSL) protection.
User Identification
CommCell User
SMTP Address (for Alerts & Notifications)
Password Aging
External User
Associated Active Directory Group
Single Sign On capable
Outlook User
Advanced message recovery
User Identification
CommCell User
A CommCell User has access to the CommCell Console only. There are no host or network
privileges assigned.
A CommCell users ability to view CommCell Console objects and perform tasks is based on
CommCell User Group membership.
Each CommCell User account should have a SMTP e-mail address which is used for receiving
reports and alerts.
Passwords assigned to the CommCell User account can be aged to provide for periodic password
changes to prevent long-term brute force hacks.
External user
An External user is a member of a Windows Active Directory domain group that has been
associated with a CommCell User Group that has, at a minimum, Browse capability.
External users must enter their name in the CommCell Console Login prompt window in the
following format: <domain name>\<user name>. When a username is entered with a domain
name, the CommServe Server automatically recognizes that the password information must be
authenticated by the external domain server.
When enabled for an associated AD domain. External users can use the Single Sign On (SSO)
feature to automatically login to the CommServe system using their Active Directory user-
account credentials.
To bypass Single Sign On in order to login as a different user, click Cancel at the CommCell
Console Login prompt window and enter the alternate username and password.
Outlook User
CommCell group authentication is required for end-users using the DataArchiver Outlook Add-
In to perform advanced message recovery operations such as find recoveries and browse
recoveries from Outlook. The Single Sign On (SSO) feature allows Exchange administrators to
establish a CommCell User Group for Outlook Add-In end-users to perform these functions
using their existing Windows user accounts and passwords residing in the Active Directory
domain. To configure this feature: Create and enable the nUseCommonSSOUser registry key.
Once the key is enabled, refresh the CommCell Console. This will automatically create a user
group called Common Outlook Add-In Default Group and a user account called Default Outlook
User. This group is automatically created with associations to all CommCell group objects. If
necessary, change this configuration in the User Group Properties (General) dialog box.
Once Single Sign-On has been configured, then Outlook users may perform find and browse
recoveries of archived messages without the need to enter CommCell authentication credentials.
When users select the Outlook Add-In option to Find and Recover Messages, their Windows
user accounts are automatically granted rights to access the CommServe system to perform this
function as part of a CommCell User Group.
Users have access to the resources and features of the CommCell environment based on the
following:
Using this approach, a CommCell administrator can provide users with the exact capabilities
they are required. These requirements can vary, depending on the tasks each user needs to
perform. A CommCell administrator can also restrict the CommCell objects that a user can view,
by restricting the CommCell objects that a user's member user group has an association with.
The focus point of CommCell Security is the CommCell User Group. Each group can be
assigned one or more capabilities necessary to perform needed tasks. The group is then
associated with objects on which these tasks may then be performed. Users derive their ability to
perform tasks on objects by membership in a group.
CommCell Users are created/managed within the CommCell environment and assigned
membership to one or more groups as required. External Users are members of an External
Domain Group that has been enabled and associated with a CommCell User group. Multiple
associations are possible. As such, both an External and CommCell user can be members of
multiple CommCell User groups with different capabilities and managed object association.
Because of its widespread acceptance and level of integration, Microsofts Active Directory was
selected as the integration path for enabling External User access into the CommCell group. As
such, Novell Directory Service and LDAP administrators can use their Active Directory
integration tools to extend their user management/authorization into Active Directory and from
there into the CommCell group.
Understanding Capabilities
& Managed Objects
Must be
Member of
Has
Performs
To Perform
TASK Associated to
To/with
Each CommCell User group must be assigned capabilities and associated with objects so that its
member users can perform functions within the CommCell environment. A CommCell User
group can be assigned all or any subset of capabilities having all or any subset association
needed to manage CommCell objects.
Capabilities are privileges that allow users to perform a variety of functions within a CommCell
environment. These functions can include performing data protection, data recovery, and
administration operations, such as license administration and administering user accounts.
CommCell objects are a hierarchical level of components in the CommCell architecture that
allow user group association. Examples of CommCell Objects are CommServe, Media Agent,
Library, Storage Policy, Client, iDataAgent, Instance, Backup/Archive Set and subclients. The
hierarchical nature of these objects is as viewed from the CommCell Browser. If an object, such
as a client computer or higher level object is associated with a given CommCell User group, then
members of that group can perform any operations involving that client computer and its lower
level objects.
Two default and permanent groups exist. These are the Master groups which have All
Capabilities for All Associations, and the View All group whose members have All Association
level access in the CommCell Console for viewing privileges only. Users who are not members
of the View All group can only see those objects, jobs, and events to which their CommCell User
Group(s) have been associated. By default, new users are automatically added to the View All
group. This default characteristic can be changed in the Security tab of the CommCell group
level properties page.
QCommands provide command line access to many basic functions enabling these functions to
be integrated into your own scripts and scheduling programs. Command line scripts can be
written independently or can be generated using the CommCell Consoles graphical interface.
CommCell Console tasks with the Save as Script option can generate scripts or batch files to
be saved on any CommCell component host. The output of batch or script is dependent upon the
host operating system. Qcommand scripts/batch files can be executed from any CommCell
component host.
Configuration
No special configuration is required to use the command line interface. The commands are
integrated with the Base software, and are therefore available on all computers which have any
CommServe, Media Agent, or Agent software installed.
In order for the commands to function, the CommVault Commands Manager service should be
up and running on the CommServe system. This service is installed by default with the
CommServe system and is responsible for handling command line requests.
Log in Sessions
Using the Qlogin command you can start a single user login session on a component host. This
removes the need to log in for every command window or shell instance. Once logged in, the
authorized session remains valid until you explicitly log out using the Qlogout command.
Encrypted password
Starting a command line session requires an encrypted password. The -p argument of the qlogin
command provides for this purpose. You can obtain this encrypted value by saving any supported
operation (i.e., a backup or restore job) as a script through the CommCell Console. This creates
the qlogin string and encrypted password for the user that is currently logged on to the
CommCell Console. You can then copy and reuse the encrypted password from that script in
other scripts. Be sure you restrict read/execute privileges to scripts containing the qlogin
password.
Scalable Object
Management Tool
Enables consistent
object association
Supports multi-group
membership
Best Used for:
Security/Management
Alerts/Reports
A client computer group is a logical grouping of client computers that serves as a single
CommCell object in which selected options can apply to all member clients. Hence, the need to
configure similar options for multiple individual clients is minimized once those clients are
members of a group.
Once created, client computer groups can be used to perform the following tasks:
A task configured for a client computer group will affect all the clients within the client computer
group, e.g., the View Job History task will display the details for all of the clients within the
client computer group.
A client can be a member of multiple groups. For example, an Exchange server may be a
member of the Chicago site group and also a member of the Exchange Server groups.
On the security side, CommCell User Groups can be associated with a Client Computer Group
for management. All clients within the CommCell User group and any clients added to the group
can be managed by member users of the CommCell User Group.
CommCell User Groups also add another level of Activity Control. Administrators can control
multiple clients in a logical grouping with a single activity control.
The greatest advantage to Client Computer Groups lies in the area of reporting and alerts.
Scheduled reports and alerts can be created for each client group. As clients are added or
removed from the Client Computer group, there is no need to revise the report or alert.
CommCell Administration
Options
E-mail & IIS Configuration
User Preferences
DR Backup Settings
All command and control functions of the CommServe system and other components are defined
within the CommCell group.
E-mail Server
SMTP Capable
IIS
Alternate for Reports
Alternate for Books Online
If reports or alerts are configured to use e-mail as the means of notification, then an SMTP
capable Mail Server is required. The SMTP capable hostname or IP address, port number and e-
mail size limit can be configured in the control panels E-Mail and IIS Configuration applet. An
e-mail size limit of zero (0) means no limit is set. If an e-mail size limit is set, messages
exceeding that limit will be sent using multiple e-mails.
The support for separate IIS hosts for Books Online and Reports is based upon expressed
customer security requirements for distinct access ability and separation.
Web servers other than Microsofts IIS are not supported. Alternate JAVA capable web browsers
for viewing reports are supported.
User Preferences
Job/Event Filters
Window Settings
User Preferences
User preferences are configurable for each user login session and are retained for subsequent
sessions.
Reports
Microsoft Internet Explorer is the default browser used to view reports. You are able to change
the browser to any other preferred browser (i.e., Mozilla, Firefox).
Job Filter
When a job completes, the Job Controller window continues to show the job and its final status
for a configurable number of seconds (300 by default). In an environment where most jobs are
performed off hours and alerts or reports are used for monitoring the results, reducing the display
time can improve GUI performance.
Event Filter
Events can be filtered to display by their severity levels. In an active environment, limiting the
displayed events to major and critical, or just critical can clear the display of unnecessary
information and improve GUI response times.
Updates
Service Packs & Certified Patches
Update Download Site
FTP or HTTP Proxy
Update Cache/Share
Local Supports all OS types
Shared CIFS/NFS locations
Install Updates
Able to Schedule
The Automatic Update feature allows for quick and easy installation of updates in your
CommCell environment, ensuring that the software is up-to-date.
This feature uses a central (Master) cache on the CommServe for holding update packages and
an optional set of distributed caches to localize the updates to their associated clients in order to
minimize LAN traffic for a particular set of components
Update packages can either be downloaded to the central cache via FTP or manually added to the
cache using the CopyToCache tool. The CopytoCache tool is available in the base directory of
the CommVault software installation folder.
If distributed update caches are configured, these can be configured for automatic update from
the CommServe cache. Note that only patches required by components assigned to the
distributed cache will be synchronized
Once update packages are available in the cache, the system provides several options to
seamlessly install the updates to all the computers in the CommCell group.
CommCell Components can be selected for updating individually, as a group, run on demand, or
on a scheduled basis.
If an update requires a reboot of the system, this can be denied or allowed as required. If denied,
the administrator can complete the install by performing a reboot of the system when
appropriate.
FTP Download
The FTP download operation downloads the updates from the FTP source site to the CommServe
cache directory. This server name can be changed if your FTP site for downloading updates
differs from the default location or if you would like to reroute the FTP transfer through an FTP
proxy. If necessary you can also configure an HTTP proxy server to connect to the FTP server to
download the updates.
The FTP download can be scheduled or run on-demand. The software also provides you with the
option to schedule FTP download operations during the install of the CommServe software.
FTP download operation will download updates based on the operating system of the clients
currently available in the CommCell environment.
DR Backup Settings
Protects
CommServe Database
CommServe Registry
Firewall settings
Log Files
Destination
Export Magnetic (Phase I)
Backup Removable Media (Phase II)
DR Backup Settings
The software stores all information for the CommCell environment in a SQL database, and in the
CommServe host registry. It is critical to be able to retrieve this information in the case of a
disaster or system failure. This information is exported and backed up during a Disaster
Recovery (DR) Backup. All or a selected subset of this data can be browsed and then restored
using the CommCell Console or the CommServe Recovery Tool.
A DR Backup operation has two possible phases. Phase I is an export that should be written to a
magnetic share as far away as practical from the CommServe Host. This is your primary source
of recovery and it should be located where a disaster that befalls the CommServe system does
not affect the Phase I backup copy.
When the metadata is backed up and archived, it remains valid (available for restoration) for a
period of time determined by the number of backup sets to retain. The number of Disaster
Recovery backup sets is preset to five full backups during installation. The retention rule for the
export copy is determined by the number of successful full backup cycles to be maintained.
Retention time for copies backed up to media drives during the Backup phase is determined by
the characteristics of the Disaster Recovery storage policy.
DR Backups can be configured to run at user defined times. A best practice for managing your
CommCell environment would be to perform a DR backup before making any major
modifications to the backup environment. This would include modifying or deleting storage
policies, redefining subclient content, and adding/removing libraries.
Module Summary
Key points to
remember
Summary
Creation of Users & User Groups (External Security (AD integration, SSO), Users
with just SMTP address for alerts and reports)
What are Capabilities? (where to apply levels CC, MA, CL, DA, SP,SUB, LIB,
Client Comp Group, BackupSet)
Configuring Auto Updates (Download from FTP, single or multiple cache areas,
client install options, Media Agent update options
Library Management
www.commvault.com/training
Overview
Understanding Libraries
Configuring Media Agents
Configuring Magnetic Libraries
Configuring Removable Media
Libraries
Overview
Objectives
List and describe the types of Media Agents and Libraries configurations available.
Describe the role and configuration options of the index and index cache in data
protection, recovery, storage, and content.
Configure and understand the common settings and options for use of a magnetic
library and associated media.
Configure and understand the common settings and options for use of a robotic tape
library and associated media.
Configure and understand the common settings and options for use of a Standalone or
PnP Library and associated media.
Understand the use and implementation of NDMP or IP-Based libraries.
Understanding Libraries
Media
Connection
Understanding Libraries
A library is a consistent term used by the software to refer to any managed storage device and its
associated attributes. Even standalone drives are referred to as libraries. Libraries are managed
by Media Agents. In some cases a single Media Agent may manage multiple libraries and
multiple Media Agents may have access to the same library. All data written to or read from a
library must transit through a Media Agent.
Media
Magnetic Disk
Read/Write
WORM
Removable Media
Tape
Magnetic Optical
USB Storage Device
Media
Magnetic Disk
Magnetic disk - as a protected storage library is supported on any OS supported file system.
Disks, disk partitions, or logical volumes are addressed as mounted file systems or by network
protocol (CIFS/NFS). Each addressable storage unit is referred to as a Mount Path. A library
can have any number of Mount Path. Access to data stored on Mount Paths can be shared with
other Media Agents.
The primary advantage of Magnetic Disk is its ability to conduct multiple simultaneous
read/write operations. The second advantage is write speed.
Normally, magnetic disk is used as read/write media. However, in archive situations, Write Once
Read Many (WORM) media may be required and is supported.
Removable Media
Removable Media is supported in three forms Tape, Magnetic Optical (R/W and WORM), and
USB storage devices. The most common of the three is tape.
Tape media has its advantages in being a low cost storage media that is easily transportable for
offsite protection and disaster recovery. Tape media has its disadvantages in that it is a sequential
access medium with a potential for wasting capacity if the tape becomes unavailable or removed
before data has been written to the end of the tape. In some cases, a tape is deliberately not filled
in order to meet data protection (e.g. removal) or operational needs (e.g. business boundaries).
Magnetic Optical medias advantage is its durability and reliability. Its primary disadvantages
are cost and capacity.
USB Storage Devices provide a highly compatible and portable storage device at about the same
cost as tape. However, USB Storage Device support is restricted to the equivalent of a
standalone tape library (i.e. no robotics or internal storage device for multiple drives)
Connection
Shared
Static
Dynamic
IP-based
Connection
Dedicated
Direct attached Libraries are dedicated to read/write operations from a single Media Agent. This
is the simplest form of library configuration.
Shared
A Shared library is one where its media devices are accessible by two or more Media Agents.
A Static Shared library is a magnetic library using Shared Disk Devices. These devices can
mounted on the Media Agent and/or shared via network protocol (CIFS) or shared file system
(e.g. GFS Sistina, PolyServe). Shared devices can be enabled for read/write or read-only.
Additionally, a mount path can be cloned (replicated) via hardware or software and made
available for read-only. This is called a Replica Static library.
A Dynamic Shared Library is a library where the read/write devices are available to two or
more Media Agents. Dynamic libraries required a SAN environment. Drive Control is
coordinated/passed from one Media Agent to another as required. Only the controlling Media
Agent can read/write from the device at any one time. Library/Media Changer control is also
managed by one Media Agent at a time. Media Agents having drive control will send requests to
load/unload media from a removable media drive (or reserve/release a magnetic drive) via the
CommServe to the Media Agent with Library/Media Changer control.
Note: A Dynamic Shared Magnetic Library requires a license and drives with Fibre
Channel/SCSi-3 embedded capability.
IP-Based
An IP-Based library is one where the Media Changer/controller is managed by 3rd party
software. Drives within the library can be direct-attached, shared independently or dynamically
shared. IP-based libraries allow the library resources to be shared with multiple software
packages.
Setting a data transfer limit can prevent saturation of resources and optimize the performance of
the Media Agent.
To verify the integrity of the data transferred over the network, the Data Integrity Validation on
Network generates CRC codes at the respective clients before the data transfer which is then
verified with the Data Integrity Validation (CRC) generated at the Media Agent after the data
transfer is complete, and vice versa. This option can be enabled during backup operations, restore
operations, or both.
Indexes located in the cache keep track of where individual objects are written on protected
media. Indexes are created, updated, and written to protected media with each data protection
job. With a shared index cache, a time stamp file is used to synchronize updates from multiple
Media Agents.
A Browse operation will consolidate and read indexes in the Index Cache as required by the
scope of the browse. Indexes required, but not found in the Index Cache are automatically
restored.
Indexes are retained in the Index Cache by management threshold settings on the Media Agent.
For the best performance retain as many indexes as possible in the cache to minimize the number
of times required to retrieve an index from the storage media. Do this by setting the Index
Retention Time in Days to 999999999 thus taking it out of the management role and using only
the Index Cleanup Percent to manage the index cache to its top capacity.
Configuring Magnetic
Libraries
Library Level Settings
Usage pattern
Managed Disk Space
Allocation Policy
Mount Path Usage Options
Mount Paths Settings
Magnetic Libraries are added through the Library and Drive Configuration control applet. Each
Magnetic library has a unique internal ID and an Alias that appears in the CommCell Console.
Naming magnetic libraries is a personal choice, but we recommend using a descriptive name that
identifies the capacity, type, use and of the library. For example; 600GB RAID 5 Spool, 800GB
RAID 3 Archive, etc.
Spill & Fill assigns one writer from each mount path in turn until all mount paths are being used
before adding additional writers to each mount path. This distributes the data transfer operations
across more resources and could yield better performance.
A periodic operation will check for thresholds being exceeded. If the threshold is exceeded the
operation will erase the "Managed" data starting with the oldest until either all managed data is
erased, or the lower threshold level reached. You can increase or decrease the periodicity of this
operation by the Interval (Minutes) between magnetic space updates (default:30 minutes) setting
located in the Service Configuration tab of the Control Panels Media Management control.
If a job exceeds the high watermark and fills the available disk space before managed data can be
erased, the job will go into a waiting state due to "Insufficient media". Once the erase operation
is completed and space becomes available, the job will resume.
Best Practice - Set your high watermark level to leave at least sufficient disk space for what your
hourly throughput rate can fill. This will give some breathing room.
Allocation Policy
Allocation Policies are used to optimize performance and throughput to each storage device by
balancing write demands against available resource. Each mount path has an allocation policy
which defines the maximum number of concurrent writers for that mount path. A magnetic writer
is the equivalent of a tape device. The default maximum number of writers for a mount path is 5.
This can be changed for each mount path via the Allocation Policy tab of the Mount path
Properties window.
Magnetic libraries also have a Library Allocation Policy. This policy is used to determine the
maximum number of writers allowed for the entire library. By default the Library Allocation
Policy is set to the Maximum Allowed Writers which is a summation of writers from all mount
paths. For example; if you have 4 mount paths with 2 writers each, the Maximum Allowed
writers to the library would be 8. You can set the number of writers allowed for a library to a
number less than maximum. You might want to do this if the Media Agent or data path have
insufficient resources or bandwidth to support the maximum number of writers for all mount
paths.
There is no similar control/policy for readers. You can have any number of readers for a mount
path. However, such as for writers, a practical limit bases on available resources should be used.
1
2 Fill & Spill
3 4
5
Media Agent
1
4
2 Spill & Fill
5
Media Agent
3
If no New Volume folder request is made, then the job will attempt to continue to use the active
volume folder for that storage policy copy and stream. If the active volume folder is unavailable,
then a new volume folder request will be made using the previous logic.
If a magnetic librarys Spill & Fill option is selected and a new job request is made on that
library, then the librarys unreserved online, writable mount paths will be searched in order of (a)
Least recently used; (b) Least used; and (c);Oldest created mount path. The first available writer
found will be used. Current active folders are not considered and are always marked full.
If there are no unreserved, online, writable mount paths then the search will be in the order in
which the mount paths were created starting with the oldest and the first available writer found
will be used to create/write the New Job. If writers are multiplexed, then all available writers will
be used first before another multiplex stream is used.
Unbuffered I/O
Space Allocation
Fragmentation
Writer Allocation Policy
Unbuffered I/O
To increase the speed of operations accessing the mount path, you can enable the Media Agent to
bypass the Microsoft Windows file system buffering. Note that this option is only applicable for
Windows Media Agents and disks that are directly mounted (not UNC paths).
Note that this is a licensed feature and requires a feature license to be available on the
CommServe system. However, this option is always enabled for shared magnetic libraries with
dynamic mount paths. Therefore additional licenses are not required for this feature on this
configuration.
Space Allocation
Space allocation allows you to establish the maximum amount of space that must be used by the
mount path. This can be done by specifying the reserve space and selecting either the Use until
the mount path reaches the reserved space or Do not consume more than n GB options.
In some situations the mount path may consume more space and exceed the specified minimum
reserve space. Consider the following example:
If you have established the minimum reserve space as 512 MB, and run a 2 streamed
data protection operation when the disk space is 515 MB, both the streams will use a
minimum of 25 MB before spanning to an alternate mount path. In such a situation
the free space will fall below the specified 512 MB.
Similarly, the system may consume more than the specified maximum space.
Consider the following example:
When a data protection job is initiated, the system checks for the available disk
space and verifies whether it is less than the specified maximum space. For
example; if you have specified 1 GB as the maximum space and if you have used
0.75 GB, the data protection job will be initiated using the mount path. However,
if the size of the data is .5 GB, the system will write the entire data. In such a
situation the space consumed will be more than the specified maximum space.
Note that a data protection operation will generally write to the mount path until the minimum
reserve space is reached, before spanning to an alternate path.
Fragmentation
This option sets the pre-allocation mode meaning that this a reserve ahead feature. It ensures
that an adequate size of contiguous chunk space is available so that all the chunks in a data
protection job can be kept together on a write from a Windows Media Agent. This minimizes
fragmentation caused by multiple simultaneous writes.
If necessary you can also disable the mount path for write operation. This is useful in situations
where you wish to retire or phase-out a mount path.
Configuring Removable
Media Libraries
Auto Cleaning
Auto Discover
Drive Verification
As removable media (tape cartridges and optical disks) can only be accessed by one tape drive
(and consequently one operation) at a time, you must plan carefully to avoid resource contention.
Auto Cleaning
You must clean each drive periodically or when necessary to remove the oxides that accumulate
on the read-write heads. The drive cleaning operation includes the task of mounting the cleaning
media into the drive, cleaning the read-write head and unmounting the cleaning media.
When Auto Cleaning is enabled, the cleaning tape is automatically mounted and a cleaning
operation is initiated on the drive, whenever the selected options indicate that the drive requires
cleaning.
The CommVault system automatically mounts the cleaning tape and cleans the drive if the
Enable Auto-Cleaning options are enabled for the library. This option is not enabled by default.
If you enable this option, verify and ensure that you have a good cleaning media in the Cleaning
Media pool. When a drive is successfully cleaned, a message is displayed in the Event Viewer,
and the drive cleaning parameters in the Odometers tab of Drive Properties are reset.
Keep in mind that if these options are not enabled, the system does not automatically clean the
drive when the hardware indicates that the drive requires cleaning. Hence subsequent mount
operations in the drive may fail.
This option is not available for stand-alone drives and optical libraries.
Auto Discovery
Before using a new media, the Media Agent must collect certain information about it through a
process known as discovery. When a media has been discovered its information is entered into
the CommServe database. The media information is permanently retained; media does not have
to be rediscovered if it is exported from the library and re-imported.
If new media are imported through a librarys mail slot, the import operation triggers a discover
operation. This is dependent on whether you have enabled or disabled the Enable Auto-
Discover option for the library.
If the automatic discovery option is not enabled, the system will prompt you to provide the
necessary details for the media.
If the automatic discovery option is enabled, the system discovers the media during a subsequent
inventory update triggered by a job from the CommCell.
If the automatic discovery option is not enabled for the library and if you have some
undiscovered media from a previous import, or if you import new media by opening the library
door and inserting them, you must initiate a discover operation.
Media can be discovered from both the Library and Drive Configuration window and the
CommCell Browser.
Drive Verification
When selected, the drive serial number and access path is verified before reading or writing to
the media.
It is strongly recommended that this option be enabled at all times to prevent the overwriting of
data, when the drive access path is changed due to hardware configuration changes. This option
is enabled by default.
Module Summary
Key points to
remember
Summary
These libraries can be direct attached or shared among multiple Media Agents.
Support for SAN is easily configured.
Single instancing is supported for libraries within the software as well as specifically
designed hardware.
Media Agents can be multiple platforms and can work together in a single CommCell.
Sharing the Index Cache offers the added convenience of Multiple Data Paths and
Media Agent failover, as well as load balancing.
Media Management
www.commvault.com/training
Overview
Overview
Objectives
List and describe the options available for managing magnetic media.
Describe the role and configuration options for Media pools and removable media.
List and describe the options for importing and exporting media.
Configure and understand the common settings and options for use of a Standalone or
PnP Library and associated media.
Understand the use and implementation of NDMP or IP-Based libraries.
Understanding Media
Management
How data is stored
Basic Media Cycle
Removable Media Pools
Managing Media Content
Job States on Media
Media management involves interaction among the CommServe system, Media Agents, client
programs, storage devices and media.
Media ID
OML
File marker Bytes
offset
chunk
Individual objects within a chunk are identified by offsets from the beginning of the chunk. A
restore job will sort objects by chunk and offset in order to do a single restore pass read of data.
Magnetic Media
Chunks stored on magnetic mount paths are written to uniquely numbered folders (e.g.
VOL_123).
Note: the timing/size of the last two actions can be changed in the Media Management
Parameters list
Exceeds Media
Retention Requested
Criteria
Appendable Active
Active
Appendable
Full
Full
Exceeds
Retention Exceeds
Criteria Usage/Error
Retired
Retired Bad Thresholds
Delete Bad
Undiscovered
Undiscovered media is media not yet stamped or identified by the software. The discovery
operation can be configured to occur automatically upon import or loading of a tape, or can be a
manually initiated operation. Undiscovered media will appear in the CommCell Console as
Foreign media
Spare
Spare media resides logically in a scratch pool and if physically in the library it is available for
assignment by the system. Spare media located outside the library will not be prompted for
import if no spare media exists within the library.
Active
Active media is that media currently being used by the software to write data. Distinct active
media is required for each data stream of each storage policy copy. Hence a 4 stream storage
policy copy would have up to 4 active media in use at one time.
Appendable
Appendable media is that media that has not been written to full capacity and is still eligible for
additional writes. Under normal conditions, a storage policy copy requiring active media will
check for and use any qualifying appendable media.
Full/Bad
Full or bad media is media that can be read from for restores, but is not writable. A full state is
achieved through writing data to the end of the media (EOT) or by a user/job marking the media
full. A bad state is achieved through error or usage levels exceeding set threshold limits or by a
user marking the media bad. Full media will become spare media when all jobs on the media are
logically pruned and the media is not overwrite protected. Bad media will become retired media
when all jobs on the media are logically pruned and the media is not overwrite protected.
Physical
Media In Library
Exported
Logical
Scratch Foreign
Cleaning Retired
Assigned Overwrite Protect
Physical
Each piece of media is a member of a physical location group identified as either Media in
Library or Exported Media. For libraries with internal storage slots, the Media in Library
location would be listed as a slot or drive. Media in the library can be marked as Prevent
Export to ensure it remains in the library.
Exported media is media external to the library and is identified by an assignable (and editable)
location. In the case of a single drive or standalone library, there is no Media in Library
location. All standalone library media that is not in the drive itself is considered to be external
and thus in an export location.
Logical
Media is also tracked logically. Removable media available for re-use are kept in Scratch pools.
There is a default scratch pool created for each removable media library that is configured.
Additional scratch pools can be created for each library to manage use of different spare media
by different drive pools and/or storage policy copies. Since having spare media is critical to
successful data protection operations, each scratch pool has settings for managing the number of
media in the pool and the sequence in which they are used.
The Cleaning media pool is for cleaning media. Simple enough. However, you can have
different drives within a library and thus different cleaning media. Hence, you can have more
than one cleaning media pool. Movement of new cleaning media to this pool can be done
manually or automatically by barcode range. As with spare media, cleaning media is critical to
successful data protection operations. Each cleaning pool has media management settings
similar to scratch pools.
A low watermark for each cleaning and scratch pool can also be established. This parameter
represents the minimum number of media that should be available inside the library for that
media pool at all times. If the number of available media falls below the low watermark, the
system logs a message in the Event Viewer and , if configured, generates an alert.
Non-retired media with retained data can be found in the Assigned media pool. This is the
normal pool for full, appendable, bad, and active media.
Media in the Overwrite Protect Media pool can be placed there manually from the Scratch pool
by an administrator or can be sent there automatically from the Assigned media pool, by enabling
the media option to Prevent Re-use. The Overwrite Protect Media pool status exempts the data
on the media from being re-used in a normal fashion. Media must be manually moved out of the
Overwrite Protect Media Pool by the administrator.
The Foreign media pool is the logical repository for media that has been used by another
application and is not available for use by this CommCell. Media not-recognized or not usable
(e.g. media with different format or from another CommCell that optionally is not used) will be
placed in this pool. The administrator can also manually move media to this pool.
The Retired media pool is for media that has exceeded the vendor recommended threshold
values for usage and errors. Media that has exceeded thresholds but still has retained data will
not be moved to the retired media pool until all such data has been aged off.
Viewing Contents
Deleting Contents
Writing over Contents
Erasing Media
Viewing Contents
You can view the contents of a specific used media. This feature can be used to view a list of
data protection operations residing in the media. All the details associated with the data
protection operation(s) available in the media are displayed. This includes the following:
Deleting Contents
The delete contents option can be used to delete the contents of a media and move it to a
specified scratch pool. This option can be used to make media available to complete an important
data protection job when there are no spare media available in the library. It can also be used to
skip a missing/corrupt tape in a multiple tape restore operation.
Erasing Media
The erase spare media operation ensures that the old data from removable media (tapes and
optical platters) are not recoverable once the media is recycled. This is done as follows:
Only spare, retired and recycled media from tape and optical libraries can be erased.
Note, however, that the erase media operation cannot be performed on magnetic and
stand-alone drives. Once a media is erased by this operation, data cannot be retrieved
using Media Explorer.
Note that the Erase Spare Media operation will not erase the following data:
Data written by other applications, if the option to overwrite media when content
verification fails is disabled in the Media tab of the Library Properties dialog box.
The Erase Media operation is a low priority job, and is displayed in the Job Controller window.
It can be killed, if necessary.
Retained
Aged
Managed
Failed
Killed
Retained
Retained jobs have not exceeded retention criteria and are automatically available for a browse
or Restore by Job operation.
Aged
Aged jobs are backup jobs on removable media that have:
Exceeded all retention criteria or
Been manually pruned
In either of the above cases, a Data Aging operation must be run to mark the job records as
pruned in the CommServe database. Since the data is on tape and not actually deleted, aged jobs
are available for browse or Restore by Job operations if explicitly included in these operations.
This inclusion can be set in the CommCell environment by default or job specific.
Managed
Managed jobs are backup jobs in a managed magnetic library that have exceeded all retention
criteria and had a Data Aging operation run to mark the job records as managed. Managed jobs
are treated the same as retained jobs for browse and restore operations. An administrative
process periodically checks the amount of free space available in a library and compares that
against the high watermark for managed data. If the high water mark is exceeded, the process
erases managed jobs until the low water mark for managed data is reached. Jobs are erased in
chronological order starting with the oldest. Use of managed jobs on magnetic libraries
maximizes the use of the library capacity for data availability.
Failed
Failed jobs are jobs that, for some reason, did not successfully write all required data to protected
storage. Indexes for failed jobs are not retained in cache or written to the protected media.
However, successfully written chunks from failed jobs can be explicitly included in Restore by
Job operations.
Killed
Killed jobs are jobs that are prematurely terminated by the software or a user before successfully
writing all required data to protected storage. Indexes for killed jobs are not retained in cache or
written to the protected media. However, successfully written chunks from killed jobs can be
explicitly included in Restore by Job operations.
Spare Media
Appendable Media
Overwrite Media
Exported Media
Bad Media
Mixed Retention
Spare Media
Spare (or Scratch) media usage is determined by the job type and the usage options selected in
the storage policy copys associated scratch media pool. Spare media will be used when:
The active job stream fills its current active tape and requires spare media
A new active job stream starts with the option to Use New Media
The job is a Synthetic Full job. (appendable media may be a source of data, hence not
a choice for writing)
Note: The order in which spare media is used is selectable within the Spare Media pool. The
options are to Use new media first (default) or Use recycled media first. In using recycled media,
the media with the least amount of usage and/or errors will be used first.
Appendable Media
Appendable Media will be used if:
The new media requirement is not in support of a synthetic full backup. This will use
a tape from the scratch pool vice appendable media since the appendable media may
contain data that is required to create the synthetic full.
Overwrite Media
Media will be overwritten if it is:
Exported Media
Exported Media will be recalled if :
Bad Media
Bad media can be used for data recovery operations only. Bad media will remain in the assigned
media pool until such time as all usable data has been aged off. The bad media will then be
moved to the Retired Media pool awaiting ultimate deletion and removable from the library.
Mixed Retention
By design, storage policy copies use separate media largely to prevent mixing different retention
requirements on the same piece of media. Should this happen, the media is retained for the
longest retained job on the media and may not recycle back for use as expected. This condition
can be caused if you use different backup and archive retention setting or use extended retention
without separating the jobs to different media. The CommVault software warns you if different
backup and archive retention requirements are set. It does not warn you if extended retention is
used.
Media Labeling
Auto Stamping
Barcode Patterns
Updating Barcodes
Media Labeling
Media labels are used to externally identify and track removable media. Exported media is
recalled using the external label. Any removable media loaded into a drive will validate the On
Media Label to prevent a mis-labeled tape from being used.
Auto Stamping
Media in Standalone and PnP libraries should be labeled for manual handling. If the option to
Auto Stamp Media in Drive is selected and an unidentified (or new) media is loaded into the
drive, the system automatically stamps a new label using either the specified Barcode Labeling
Scheme or a default label, which is the media creation date and time. If the option is cleared, the
unidentified (or new) media must be manually stamped with an appropriate label.
Media discovered in blind libraries will automatically be labeled. A default labeling scheme is
used unless the user specifies an alternate barcode labeling scheme. Care must be exercised
while generating the labeling scheme to ensure that each media label is unique. If the specified
labeling scheme is not unique, and if duplicate labels are found, the system automatically
appends a media ID to make it unique.
Barcode Patterns
When you use one or more standard barcode pattern(s) in the libraries in the CommCell
architecture, you can define different Barcode Patterns to automate the distribution of media
when you have specific purposes for the media in the library. For example:
When you have a library with different drive types, we recommend you have specific
barcodes for media associated with each drive type. This will help you to easily
manage and administer the media in the library.
When you have media that are not used by the Media Agents in this CommCell
environment (referred to as Foreign media in the software) stored in the library and
use Vault Tracker agent to manage the movement of such media.
If you already manage media using barcode patterns, you can use this method to
automate the process.
Spare, Cleaning, and Foreign Media Pools support the use of Barcode patterns for automatic
distribution of new media.
Updating Barcodes
Barcodes labels can wear out, fall out and be assigned different barcodes. Media from libraries
with different labeling requirements may be required for use in alternate libraries. (For example,
a auto stamped media from a standalone library can be assigned a barcode and used by a barcode
capable library)
Additionally, some libraries recognize a 6 character barcode while others recognize 8 characters.
A firmware upgrade may affect all your barcode labels in your existing library. The CommVault
software provides the means to update media barcodes in all of the previously cited conditions.
Individual media labels can be edited and scalable tools allow you to add/remove prefix/suffix
for all media with the library.
Threshold Settings
Threshold Settings
Media that exceeds usage or error thresholds jeopardize data protection and should be removed
from use. When the CommVault software evaluates scratch pool media for use, media that has
exceeded thresholds will be automatically moved into the retired media pool the software tracks
statistics for each tape and optical platter which can be seen in the odometer tab of the medias
property page.
The Control Panels Hardware Maintenance control lists the vendor recommended thresholds for
cleaning and expiring media. These thresholds can be changed, but any change should be
verified with the media vendor to ensure support and reliability of the media in recording or
reading data correctly.
Exporting Media
Importing Media
The CommVault software offers several features to automate the process of managing media
within the library. You can automate the media discovery process in the library using one of the
following methods:
When you configure a library, select the default media type and then click Yes in the Discover
Media Options dialog box to automatically discover the media in the library.
Importing is the process by which you move media that are outside a library into storage slots
within the library.
Exporting is the process by which you physically remove one or more media from a library.
You can import/export media through the librarys mail slot (if available).
You can open the library door and insert/remove media by hand.
If the library involved is a blind library, then an inventory will be required when media is
imported in order to discover and track the media within the library.
When media is imported/discovered, the system assigns it in one of the following ways:
If it is a previously known exported media it will be assigned to the spare pool or to the
storage policy copy to which it was assigned before it was exported. Information about
exported media is retained in the system; they do not have to be rediscovered if they are
re-imported.
If the media is a new media the system assigns it to a spare media pool based on its
barcode if barcode ranges are assigned to spare media pools. If not, then the new media
will go to the default spare media pool. It can then be manually moved to another spare
media pool as needed.
If a data protection operation requests media that is currently exported the following
operation will be performed:
o If the Start New Media option for exported media is checked, automatically mark
the exported media as full, and use a new media for the data protection operation.
o If the Start New Media option for exported media is NOT checked, the job will
prompt at the Media Agent and the CommCell Console for the media to be
imported. The job will remain in the waiting state with the Reason for job delay
stating that the media is outside the library.
If a restore or auxiliary copy job requires data from an exported media, the system will
prompt you to import the media in order to complete the operation.
Exporting Media
Exporting Media
Exporting is the process by which you physically remove one or more media from a library. If a
media is reserved by a job for a read or write operation, the media cannot be exported.
Media that is left in the mail slot after an export operation will be treated as an exported media.
Pop-up messages associated with media outside the library and the media information in the
CommCell Console will indicate that the media is in the Import-Export slot associated with the
mail slot. You can use Vault Tracker agent and Media Repository in Vault Tracker agent to
manage media residing outside the library.
There are two ways to remove the media from the library:
You can remove the media through the librarys mail slot (if available and supported
by the library).
You can open the library door and remove media from the storage slots by hand.
Removing media through a mail slot offers the following advantage: The inventory update that is
triggered by a mail slot export is much less time-consuming than the full inventory operation that
is triggered when you close the library door. However, under certain circumstances you may
want to open the library door even though a mail slot is available. For example, if you want to
remove many media from a library at once, it may be faster to open the door than to use the mail
slot.
Export by List
Media can be selected from a list of media and exported. This is useful when you wish to export
several media at the same time and know either the medias barcode, or slot number in which the
media resides. This task can be performed from the following levels in the CommCell Console:
Library
Media in Library
Media List dialog box which appears when you select the View Media option. This
dialog box can be accessed by right-clicking a Storage Policy copy.
Media List dialog box which appears when you select the Change Data Path option.
This dialog box can be accessed by right-clicking a Storage Policy copy.
Media List dialog box which appears when you select the Media Not Copied
option. This dialog box can be accessed by right-clicking a Storage Policy and
Storage Policy copy.
Export
Container Transport Location
Picked
PickedUp
Up
Due
DueBack
Back
Returned
Returned
to
toSource
Source
Transport
Disaster Recovery
Library
Vault Tracker agent provides the facility to manage media that are removed from a library and
stored in offsite locations. In practical terms, Vault Tracker agent provides the following
capabilities in your day-to-day operations:
Identifies media that must be sent off-site for storage or brought back from off-site
locations.
Automatically moves the media in sequence in the library and provides a pick-up list
for the operators.
Facility to identify and track media during transit.
Facility to track record and track containers when the media is stored and moved
using containers.
Facility to record and track the movement of non-CommVault or Foreign Media.
Importing Media
Importing Media
Importing is the process by which you move media that are outside a library into storage slots
within the library.
Once again, the media will be automatically discovered, if the Enable Auto-Discover option for
the library is enabled.
If this option is disabled, the media will be displayed (with a '?' icon) in the Media in Library
pool in the CommCell Browser. You must subsequently discover the media, in order to use the
media in the library.
Before using a new media, the Media Agent must collect certain information about it through a
process known as discovery. When a media has been discovered its information is entered into
the CommServe database. The media information is permanently retained; media does not have
to be rediscovered if it is exported from the library and re-imported.
If new media is imported through a librarys mail slot, the import operation triggers a discover
operation. The system will then acquire the slot map from the library and use this as our media
inventory. This is dependent on whether you have enabled or disabled the Enable Auto-Discover
option for the library.
If the automatic discovery option is not enabled, the system will prompt you to provide the
necessary details for the media. In this case when you import media you must initiate a Discover
Media from the Librarys task menu.
In a blind library the discover media operation is available only at the media level.
When you import media in a blind library, ensure that you perform a quick inventory with a full
scan, in order to discover the imported media.
It is recommended that you initiate a full scan operation (if it is not automatically triggered)
whenever you directly import media into a library, especially on libraries without a mail slot.
The discover media operation mounts the media into a drive and if a valid OML is not found, it
writes the On-Media-Label (OML) on the media. Once media has been un-mounted, the system
keeps track of its location.
A discover inventory job is displayed in the Job Controller window and can be killed if
necessary. Also note that both the full and quick inventory processes will automatically discover
new media whenever a new media is found.
For a Standalone Library there is no discovery process for media. A tape is loaded into the
drive and an OML is written only when a job is initiated. The media label can be automatically
generated or manually input. Media labels can also be pre-configured by the Stamp Media
Operation.
If the media being imported was previously used with another application the CommVault
software will not automatically detect that in a barcode-ed library. The media will be placed in
the scratch pool and be available for jobs. Once that media is selected for a job, the system loads
the tape and checks for a header. If we do not recognize the format or header information as one
of our own the media is placed in the Foreign Media pool.
In order to use this media the tape has to be either formatted or the option to Overwrite Media
When Content Verification Fails must be selected from the library properties.
For blind or standalone libraries the verify operation does not discover new media.
A verify operation is displayed as a job in the Job Controller window and can be killed if
necessary.
When media is loaded into a library other than the one it was written in, the system will generate
an event message notifying the user that the media is in a different library. This is just an
informational message and not a problem. Though this media cannot be written to, the data
contained within can be restored without any additional configuration.
Module Summary
Key points to
remember
Summary
Data is stored in Chunks, 2GB for all data on Magnetic media, 4GB for indexed data
to tape and 16GB for Database data to tape.
Magnetic Media is faster for writing and recovering data. Tape media has its
advantages in being a low cost storage media that is easily transportable for offsite
protection and disaster recovery.
Pros and Cons of mixing retention on a single media. Using basic retention vs.
extended retention.
Vault Tracker agent gives the ability to manage media outside of the library.
Storage Policies
www.commvault.com/training
Overview
and Aging
Roles of Verification, Encryption
Overview
Objectives
List and describe the function and role of Storage Policies in managing protected
data.
List and describe the common settings and options available for a Storage Policy and
Storage Policy copy.
Describe how data is retained, managed, and aged within a Storage Policy.
List and describe the options for performing Verification, Encryption and Content
Indexing on managed data.
Understanding Storage
Policies
Design and Philosophy of
Storage Policies
Types of Storage Policy copies
Policies are templates by which administrators can apply a consistent set of rules on a scalable
level. Storage Policies are essential for easy and efficient management of large volumes of
protected storage.
Drive Pools
Media Agent(s) Copies
Libraries
Days
Cycles
Streams
Data Paths Retention
A Storage policy is a single addressable entity that encapsulates the physical data paths and
storage media along with the logical management and retention of one or more copies of data
requiring protected storage. Production data can be easily directed to appropriate protected
storage management by simply associating it with a storage policy.
There are two types of storage policies: Disaster Recovery and Standard. A Disaster Recovery
storage policy is used to separate and manage DR backup data independently from other data
protection jobs. A Standard storage policy can manage data from any backup or archive job or
both.
The only permitted relationship between storage polices is through a linkage called Incremental
Storage Policy. When a standard storage policy has an incremental storage policy enabled, full
data protection jobs are stored and managed by the standard (or parent) storage policy, while all
non-full data protection jobs are stored and managed by the associated incremental storage
policy. This configuration can effectively use each media types concurrency, capacity, and
associated costs to store data. For example; you can direct large, less frequent full backups to
tape media and smaller more frequent non-full backups to magnetic media. Any standard
storage policy can have, or be an incremental storage policy. A single storage policy can be used
as an incremental storage policy for multiple standard storage policies.
Conceivably, a single storage policy can be used to manage all data for a company. Different
data retention and media storage requirements can be accounted for via configurable
characteristics of secondary copies. However, in a practical sense, using a single storage policy
for complex data management requirements is not recommended.
The primary reason for storage policies and the reason for using different storage policies is ease
of data management. Data boundaries can be identified along the lines of data type, data source,
and management requirement. Additionally, business, legal, and security requirements need to
be considered when designing a durable storage policy schema. Taking the time to map out a
storage plan will yield both more efficient and easier management for your data.
How many storage policies should you have? The best and only answer is Only as many as
necessary to meet your business requirements for data management. Too many storage policies
can become a management burden, while too few - a configuration burden.
Secondary
Synchronous
Synchronous Secondary
Selective Contains Copy
All Data
Selective
Contains
Selected Data
Only
A storage policy can have one or more physical copies of the data assigned to it. Each storage
policy copy is a container that provides a distinct set of management tools for protecting its
associated data. Each storage policy has at least one copy called the Primary copy which is
created by default. The Primary copy receives its data directly from a subclients data protection
job. Additional copies of the data can be created called Secondary copies. Secondary copies can
also receive their data from the subclient via an inline copy option; or via an Auxiliary copy
operation from a specified copy source.
There are two types of secondary copies - synchronous and selective. A synchronous copy
contains all jobs from the source copy for the associated subclients. Data requiring more
granular or point-in-time restore capability should use synchronous copies. A selective copy
contains only selected backup jobs from the source copy for the associated subclients. This
selection of full backups can be, job, or time-pattern based. Manual selection of backups is
possible for full and non-full job types. Selective copies are primarily used for offsite, long term
storage for data used in Disaster Recovery.
Note that within a storage policy, subclient data can be copied from the Primary copy and
between secondary copies. While you can re-associate subclients to different storage policies,
you cannot copy/move their data between different storage policies.
Storage Policy copies can reside on the same or different libraries and on the same or different
media. A common example would be to have the primary copy on magnetic disk media and
secondary copy on removable media such as tape.
Updating Secondary
Copies
Subclient Subclient
Inline
Auxiliary copy
Primary
Inline C
opy
Auxiliary
Copy
Secondary
Inline Copy
An Inline copy requires that the data for the primary and secondary copy pass through the same
Media Agent. The Inline Copy option is enabled on the secondary copys property page.
Anytime a data protection job is performed, the data will be sent to the primary copy and also to
this copy set at the same time. If the data protection job to a primary copy fails for any reason,
the inline copy will also fail. If only the inline copy fails, the primary copy job will continue and
an Auxiliary Copy job can be run to complete the update of missing data to the secondary copy.
Auxiliary Copy
An Auxiliary copy is an immediate or scheduled operation that copies updated job data from a
source copy to one or more target copies. Auxiliary Copy can be performed between any two
Media Agents, regardless of their host operating system.
A source copy can either be the Primary copy (the default) of a storage policy, or a secondary
copy of your choosing with some limitations as described in this document and CommVaults
Books Online. The destination can be any active secondary copy or all active secondary
copies. If multiple active secondary copies are being updated by the same auxiliary copy
operation, they are done so in the chronological order in which they were added to the storage
policy.
An auxiliary copy operation copies only Valid Data it does not mirror media (i.e. tape for tape).
Valid data is defined as data from a data protection job that has not failed or been killed. If a data
protection job should not (or can not) be copied, it must be manually marked as Do Not Copy in
the source copy Job View. Since a source copy can contain data from failed or killed data
protection jobs, an auxiliary copy operation may result in consolidation of the copied data on
fewer medium. In fact, auxiliary copy can be used as a consolidation method to move only valid
data to another copy in order to delete the less efficient source copy.
If the source copy has multiple streams, they are, by default, auxiliary copied serially - one
stream at a time. If there are multiple streams on the source, you can enable an auxiliary copy job
option to copy a number of those streams in parallel. The parallel copy option enables the use of
maximum possible or a limited number of data streams as defined by available resources.
While performing auxiliary copy operations, priority is given to using LAN-free read data paths.
This means that if multiple source and destination Media Agents are involved, the Media Agent
with the most LAN-free read paths will be used.
Managing Streams
When should you Multiplex?
Combine Streams
During a data protection operation, the source server sends data to the Media Agent for storage
on the media. During a data recovery operation, or restore, the Media Agent accesses data on the
storage media and sends it to the destination server.
Managing Streams
Job and Device Streams can parallel and
combine data. Managing Streams is
THE most critical operational task next to
managing media.
Managing Streams
Solution Architects and Administrators must plan for and manage the efficient allocation of
resources to move concurrent streams of data. Failure to do so can result in poor throughput
performance, waste of valuable resources, and an inability to meet business requirements.
There are two types of data streams. Each has different management controls and requirements.
Job Stream
A job stream can be described as a single instance of coordinated movement of data between the
Client and Media Agent. Each stream will have a reader on the data source and a writer on
the target source. A job configured for; and using multiple job streams can improve the rate at
which data is written to or retrieved from the storage media. Of course, available resources (e.g.
bandwidth, memory, CPU cycles, etc.) are the ultimate determining factor.
The maximum number job streams in a single job is determined by the max number of
readers enabled on the subclient or the max number of device streams (including multiplexing)
available in the associated Storage Policy whichever is smaller. This setting will also affect the
concurrent number of streams used in a Restore by Job operation.
The maximum number of concurrent running job streams within the CommCell is limited
by the High WaterMark value shown in the Job Controller window. This max value can be
changed in the Control Panels Job Management applet.
The maximum number of concurrent running jobs streams to a Media Agent is limited by
the value set by the Maximum Number of Parallel Transfer Operations. The Media Agent
option of Optimize for concurrent LAN backups allows a Media Agent to increase this value
by reducing the amount of memory and CPU cycles required for each job stream. This option is
enabled by default.
Device Stream
A device stream can be described as a single instance of coordinated movement of data between
a Media Agent and a library media device. A device can be a single stream (tape drive) or
multiple streams (disk drive). Multiplexing enables a single stream device to support multiple
concurrent job streams.
The maximum number of device streams in a library is determined by the number and type of
physical media devices
The maximum number of device streams available to a Media Agent is determined by the
aggregate total of maximum number of streams available in each library that are accessible and
allocated to the Media Agent. Device stream Allocation Policies can be set at the Drive pool and
Master Drive Pool level for removable Media Libraries and at the Library and Mount Path levels
for Magnetic disk libraries.
Multiplexing
Multiplexing allows multiple job streams to be written to one device stream. This is
accomplished by the Media Agent combining the job stream data and transferring it to the device
via one device stream.
A Multiplexing factor is enabled/set on the Media tab of the Primary storage policy copy. The
Multiplexing factor setting has no effect on any auxiliary copy operation other than Inline copy.
Note that the maximum multiplexing factor that can be set from the CommCell Console is 10
and the system displays a warning message when the multiplexing factor is set to 5 or above.
Best Practices
It is recommended that you keep the following in mind when performing data multiplexing:
Use different storage policies for file system and database type data before performing data
multiplexing. Therefore, there wont be differences in the chunk sizes of the different types
of data.
If possible use the Restore by Jobs option to restore multiplexed data, especially when
restoring large amount of data. This will provide the optimum performance during the restore
operation as there are fewer tape rewinds to secure the data.
It is recommended that you perform data multiplexing for jobs that have similar speeds (i.e.
two database jobs), instead of mixing faster jobs (i.e. file systems) with slower jobs (i.e.
databases). Mixing faster and slower jobs results in data stored on media that is not uniform.
Hence, data recovery operations of slower clients will have added performance penalty.
The multiplexing factor is determined based on the ratio of how fast the tape drive is
compared to the disk. For example, consider the following ratios:
Tape write speed = 75 GB per hour
Disk read speed (backup) = 25 GB per hour
Tape read speed = 100 GB per hour
Disk write speed (restore) = 50 GB per hour
Tape write speed/disk read speed (backup) = 75/25 = 3 Multiplexing Factor
Tape read speed/disk write speed (restore) = 100/50 = 2 Multiplexing Factor
It is recommended that the lower of the two ratios (2 Multiplexing Factor) should be used as
the multiplexing factor if you want no-penalty data recovery operations.
Combine Streams
Auxiliary Copy
A A A
B Magnetic B
Library C
C
B
Auxiliary Copy
A A
B Magnetic A, B, C
Library
C
B
Combine to Streams
Whether copying in serial or parallel fashion, an auxiliary copy operation normally copies each
source data stream to a separate destination stream. This means if there were four data streams on
the source copy, then the auxiliary copy operation would use four data streams on the destination
library.
Each stream requires its own destination media. In some cases such as copying from a multiple
stream magnetic library to a limited device tape library, this may not be the most efficient or
desired method of copying data. As such, the destination storage policy copy has an option to
Combined to <n> Streams. Combining streams is done serially and is used to improve media
storage efficiency not throughput.
The Combine to <n> Streams option is considered a media management tool and is configurable
only on secondary copies.
Example: 4 subclients with 2 data readers configured for each subclient will have a total of 8
streams. Setting the Combine to Streams option to 2 will result in a 4 to 1 consolidation and the 8
streams will be combined onto 2 tapes.
Data Retention,
Management, and Aging
Determining Data Retention
Requirements
Managing Backup and Archive
Retention
Understanding Cycles and Days
Data Aging
Retention Variants
Most agents follow basic retention rules. Basic retention rules are defined by the user as
retention days and retention cycles. These parameters determine how much data is retained and
for how long. For a data cycle to be eligible for data aging, both its retention days and retention
cycles must be exceeded.
Determining Data
Retention Requirements
Legal Requirements Customer Company Requirements
Confidence
Cost
Backup
Cycles
Days
Archive
Days
Backup
Each subclient is associated to a storage policy. How long the backup data will be retained on the
backup media is determined by the retention rules set in the Storage Policy Copy Properties
dialog box. This will affect media usage, and is an important consideration when planning your
backups. A longer retention period, for example, could use more media for securing the data over
time.
If a retention period other than infinite (default) is selected, the data will be pruned according to
backup cycles in relation to the retention rules you set in the Storage Policy Copy Properties
dialog box. Pruned data can be overwritten on the backup media.
The backup data from a subclient will be retained according to the rules set for the storage policy
associated with it. The ability to define data in subclients, and then associate them to specific
storage policies allows you to prioritize exactly what data is retained and for how long.
Archive
Data Aging rules are unique for the Data Archiver and Compliance Archiver Agents. All data and
jobs in a migration archiving or compliance archiving operation must meet the basic or extended
retention rules in days (as specified in a storage policy copy) in order to be aged. The Migration
Archiver and Compliance Archiver Agents do not support retention cycles.
Understanding Cycles
and Days
Example
S M T W T F S
Cycle 2
INC
INC
INC
INC
INC
INC
Full backups
scheduled every
FULL
Cycle 3
INC
INC
INC
INC
INC
INC
Sunday
FULL
12noon
backups scheduled
all other days
A Retention Cycle is defined as a group of dependent data protection operations starting with a
full backup and including all subsequent data protection operations up to, but not including, the
next full backup. Dependent operations are incremental, differential, or transactional log backups
that support or depend upon a full backup. Retention days specify the minimum number of
days to keep all data protections jobs in the cycle not the individual jobs.
The default settings for time and cycle parameters are set to infinite, but can be changed to better
suit the retention requirement of the data being secured.
Retention time, also referred to as Days, is defined as the amount of time that a cycle needs to
be available for a single subclient. Retention time is calculated in terms of 24 hour days from the
completion time of the last data protection job in a cycle until the start time of the data aging job.
Data Aging
Frees storage media for re-use
Evaluates stored data against retention
rules and copy requirements
For data that exceeds retention
Erases data on magnetic storage (default)
Use managed disk space to preserve data in
magnetic libraries
Flags data on removable storage as Aged
Data until overwritten or erased
Index for erased or overwritten jobs are
deleted
Data Aging
Unless you have an unlimited storage budget, youll want to delete all protected data that is no
longer required and free up media for re-use. This is accomplished through the Data Aging
operation. Retention and conditional criteria is evaluated and those jobs or cycles that no longer
meet this criterion can be pruned. Pruning will either flag or remove data pointers from the
CommServe database and, as appropriate, erase (magnetic) or reclassify (removable) media to
spare status. Note that reclassifying tape media to spare status can only be done when all the
data on a specific media is pruned off.
While retention criterion is the main factor evaluated by a Data Aging operation, there are many
conditional variants that may exclude a job from being pruned. These include such things as
transaction log jobs where the associated database is still retained and data that have not been
auxiliary copied to a secondary copy as required. Consult Books Online for conditions that
affect each data type. These conditions are not intended to impede re-use of media, but rather
ensure complete restoration can be accomplished as expected.
Data aging is performed on all active storage policy copies within the CommCell group. Storage
policy copies can be excluded from data aging by clearing the Enable Data Aging option on the
Retention tab of the copy properties page. Different data aging schedules for each Storage
Policy copy can be configured in the Aging Options tab of the CommCells Data Aging task
configuration dialog window.
Data on a magnetic library is classified as Managed when it exceeds retention requirements, but
storage usage does not exceed the managed disk space threshold limit. Managed data is treated
the same as retained data for browse and restore. Disk space is periodically checked for used
capacity. When the managed disk space threshold is exceeded, the managed data is erased
oldest first until the lower threshold limit is reached.
Tape media is serial vs. random access and cannot be overwritten until all jobs on the media have
been flagged as Pruned in the CommServe database. Until the tape media has either been
specifically erased, or is overwritten through re-use, the Aged jobs on the tape can be browsed
and restored. Unlike Managed data, the ability to browse or restore Aged data must be
specifically enabled or selected. Aged data is accessible in a Restore by Jobs option and can be
browsed for individual objects if the Browse/Recovery control option to Show aged data during
browse and recovery is selected.
To extend the availability of aged jobs on tape, you can mark a specific tape as Prevent Re-use
which will move the tape to the Overwrite Protect Media pool
Retention Variants
Data not copied to secondary storage
Transactional log data
Failed jobs
Manually retained/disabled/pruned Jobs
De-configured or re-associated
subclients
Incremental Storage Policy
Retention Variants
There are variants to the Retention/Aging rules. These variants exist to ensure every reasonable
expectation for recovering data is met. Some of the more prevalent variants are:
Logs older than the oldest non-prunable data are prunable regardless of the retention rules
of the copy.
For logs that are more recent than the oldest non-prunable data, these logs are not
prunable if:
o They have not exceeded the retention days criteria.
o They reside on a primary copy and have not yet been copied to a secondary copy.
o They reside on the copy that has the longest days retention criteria amongst the
other synchronous copies within the storage policy. The log data can be valid,
partial, or disabled.
If the status is partial or disabled, the valid copy of the log is kept on the copy with the
longest number of days retention criteria among the remaining copies.
For each backup copy of the database at least one copy of associated logs will be retained. Those
logs will be retained on the copy with the longest retention value setting.
Logs are retained beyond that copy's retention settings as long as the associated backup copy of
the database exists. Logs on lesser retention value copies are pruned in accordance with the set
retention value.
Restoring a database with transaction logs requires that you either (a) Do not select a copy
precedence for restore, or (b) you select a copy precedence that has the transaction logs. In the
case of (a), the restore process will automatically determine which copy holds the required logs
and restore those logs.
Jobs may be disabled for copy by the administrator if an archive copy job fails because the job
has corrupt data.
Jobs can be pruned manually before their normal data aging event. Pruning a Full backup job
will include all dependent incremental or differential jobs. Pruning jobs manually is usually
done to free up media.
If the incremental storage policy is also being used as a regular storage policy (and has full
backups), the full backups will be also pruned according to any basic and extended retention
rules that are set.
It is recommended that the retention rule for the full storage policy be greater than the
incremental storage policy. Data on incremental policy will be pruned earlier if it has shorter
retention comparing with full policy. If the incremental storage policy has longer retention than a
full storage policy, this may result in dangling incremental jobs.
In addition to storing protected data, there are additional actions a Storage policy can be
configured to perform to further assist in the protection and recovery of data.
Encryption
Inline Offline
Subclient Subclient
A,B A,B
A B A B
C
C Encrypt C Magnetic
C Magnetic
Primary Copy
Primary Copy
Secondary(A,B)
Secondary(A,B)
Secondary (C)
Secondary(C) Encrypt
Overview
The software allows encrypting data both for transmission over non-secure networks and for
storage on media.
The data can be encrypted online during a data protection job if transmission paths or the
primary copy security is required. Keep in mind the encryption may impact the performance of
the job. Encryption will always be performed after software compression.
Encryption can also be done offline during an auxiliary copy operation to any secondary copy.
Encrypting a secondary copy provides protection if storing the copy in an unsecured location as
well as protecting the data during transportation.
The flexibility of key management schemes makes data encryption useful in a wide variety of
configurations. Keys are generated for each chunk and can be retained internally, externally, or
on the protected media. During restores, keys can automatically be applied or you can require
manual entry of a pass phrase. You can select from several algorithms and key lengths.
The default encryption method uses the CommServes Media password to perform the
encryption. A copy of the Media password is kept in the CommServe database and on the
protected media. During restore the key is applied automatically to decrypt the data.
CommVaults Media Explorer can also be used to access the data directly from the protected
media.
In Simpana 8.0, the Media password setting has been extended down to the Storage Policy level
allowing administrators to set a different Media password for each storage policy.
If you are concerned that media may be misplaced, data can be encrypted before writing it to the
media and the keys stored only in the CommServe database. In this way, recovery of the data
without the CommServe system is impossible - not even with CommVaults Media Explorer.
This No Access mode is also completely transparent. Once enabled, it will work requiring no
additional activity on your part.
If you need only network security, the encryption keys are randomly chosen for every session.
Data is encrypted on the Client and is decrypted on the Media Agent and the keys are discarded
at the end. The entire process is completely transparent. All you have to do is to enable
encryption, and select the cipher and key.
Encryption keys can also be protected with your own pass-phrase (RSA algorithm with 1024-bit
keys) before being stored in the database. If the database is accessed by unauthorized users, and
the media is stolen, the data will still not be recoverable without the pass-phrase. This highest
level of security comes at the price of having to enter the pass-phrase for every recovery
operation and not being able to run synthetic full backups. You can also export a file that
contains the scrambled pass-phrase of the client computer to a dedicated directory on another
computer, the system can recover the clients data to that (and only that) computer without
prompting you for the pass-phrase.
Explicitly enabling synthetic full backups in the GUI will create a copy of unlocked encryption
keys in the database, which will be accessible only to synthetic full data protection operations. In
this case the regular data recovery operations will still prompt you for a pass-phrase, but
synthetic full data protection operations will not.
Data Verification
Verifies that all selected data is valid for
restoring and for being auxiliary copied
Data Verification can be scheduled or
executed on demand
Verification options can be set on Storage
Policy copy
User can also select individual backups for
verification and/or select individual media
for verification
Data Verification
During a data verification operation, protected data is checked to see that it is available and
readable. CommCell metadata is used to determine where the data is stored, what chunks are
used and whether the media and chunks can be located and successfully read. Short of restoring
the data, this is the best way to verify accessibility of protected data. You can verify all or
selected jobs on a specific copy or media. Verification is a schedulable job and can be run
serially or in parallel for multiple stream copies. Data Verification is accomplished by selecting
a storage policy. All copies in the storage policy can be verified or only a selected copy.
Data verification can be run to check if valid data can be successfully auxiliary copied. On the
other hand, if an auxiliary copy operation is successfully run then the data can be considered
verified.
Module Summary
Key points to
remember
Summary
Purpose for Storage Policy Copies - Primary (backup), Sync (Specify by client, takes
all data), Selective (specify by client fulls only if automatically selected, good for
remote sites with little resources)
Retention can be defined as Cycles and Days, Days only, Basic (cycle and days,
Infinite, Spool) Extended (weekly, monthly, quarterly, yearly).
Data Aging - When does it happen, Why does it happen, What does it do?
There are two types of Streams - Job based (from client), Device which is defined at
the Storage Policy.
Clients
www.commvault.com/training
Clients Module
Overview
Agents
Backup and Archive Sets
Configuring Subclients
Overview
Objectives
List and describe the common types of Agents available and their function in
protecting/restoring data.
Describe the function/role of a Backup/Archive Set in defining and managing Agent
data.
Describe the role of Data Classification in determining archive and backup set
content.
Describe a subclient and the purpose of the different property settings used for
facilitating data protection and movement.
Describe the function and configuration steps for using a subclient policy.
Host system
Physical machines
Clustered Systems and
Applications
Virtual machines
Properties
A client is the host system addressable by an IP Address that provides the resources necessary for
an Agent to access the data. Clients can be physical machines, VMWare or Microsoft Virtual
Server style virtual machines, or cluster virtual machines in a supported clustered environment.
Data accessed through a client can reside on local/SAN disks or available through CIFS/NFS
paths.
During data protection and restore operations, the Client must have access via TCP/IP with the
CommServe host and supporting Media Agent(s). Connectivity with the CommServe host is
required to maintain control over the operation, update indexing, and for tracking the data on
these systems. Although CommVault software does have some robust features that can
overcome minor communication gaps, extended loss of communications will cause backup or
restore jobs to fail.
Always consult the latest available documentation to see what file system or application is
supported.
Properties
In the CommCell Console, Client specific and common shared agent properties are set and
selected at the Client level. These properties include Client Priority; Job Results Folder
retention, location, and low space alert threshold; Encryption option; and Content Indexing
option.
Understanding the
role of Agents
Agent Roles
File System
Application
Archive
Client Agents are software modules that perform data protection and data recovery operations for
specific operating systems or applications. Different file and application data types can not or
should not be protected or restored in the same manner. Each data type has unique requirements
to ensure consistency and accuracy when collecting or restoring the data. Multiple agents may
be required to protect all types of data residing on a computer.
A File System Agent provides an indexed-based backup and granular restore capabilities of files
and system metadata as may be required to perform full recovery of operating system and user
files.
An Archive Agent provides the movement of infrequently accessed data from high cost
production storage to lower cost storage with transparent recall capability. A set of rules is
defined by the administrator to archive selected objects (files/messages/documents) to alternate
storage and optionally leave a stub object in their place. The archiving frees up production
space and/or slows down the growth of disk space usage. The stub object contains the necessary
information to locate the archived object in protected storage. Native or client mechanisms are
provided to facilitate the automatic and transparent recall of object when the stub is accessed by
a user or application.
Non-indexed based agents are ones where the degree of granularity is insignificant and the need
to quickly protect or restore the entire data set for consistency purposes is paramount (e.g.
Database Instances, Image level agents).
Indexed-based agents use the Index Cache managed by the Media Agent for granular level
browse and restore. Non-indexed agents use the CommServe database to track restore objects.
Oracle is an exception in that it uses its own method of tracking (RMAN) protected objects for
restore.
A backup set is a logical grouping of subclients representing a complete set of data related to a
specific agent. For example, a backup set for a file system agent will contain all files on all
drives within the Client machine.
Understanding Backup
and Archive Sets
Backup/Archive
set
Logical grouping of all
subclients
Provides for Distinct
Management
Defines Data access Subclient Content
and/or Collection
Method
A Backup or Archive set is a logical management grouping of all subclient content for the parent
agent. Since subclient content can vary as well as their protection schedule and storage, a
consistent means must be provided to address (e.g. User login, application name, scan method)
and Browse/restore all agent data as a single entity. Some backup sets provide options for
application wide rules used by all subclients when identifying, protecting, or restoring data.
User-defined backup sets provide multiple management options for the same set of data. When a
new user defined backup set is created, it becomes a complete collection of data on the client for
the specific Agent.
The advantage to using multiple backup sets is when customized backups need to be performed
that do not conform to the current subclient design within the default backup set. Additional
backup sets can be created and scheduling can be set up to only backup these subclients when
needed. Note that duplicate data within separate subclients in separate backup sets can not be
backed up simultaneously. Conflicting operations can cause jobs to go into a pending state until
running jobs are completed.
Caution should be used when creating multiple backup sets. Improper configuration and
scheduling can result in failed jobs or data being backed up multiple times.
All Archiver Agents support the ability to create additional archive sets, i.e., user-defined
archive sets. This feature is useful for archiving client data using different archive options or
schedules, and for workload balancing.
If there is an overlap in the data archived between various Archive Sets on the same client, data
must be retained on all the storage policies used by various subclients in all Archive Sets on the
client in order to recover data.
Minimizes scan
time
The Data Classification Enabler (DCE) enables you to monitor data dynamics in real time
reducing the need and impact of a scan phase to identify data for a backup or archive job. With a
larger number of objects to scan, DCE can reduce data content identification time to seconds
instead of minutes or hours.
With Windows Archiver clients, DCE can read user and user group information from Active
Directory and use this information to further define archive set content. All the rules for Data
Classification are configurable from the DataClassSet subclient properties Rules tab of the File
Archiver for Windows Agent. Data Classification rules include the following:
Folders/Files Owned By - Allows you to select and exclude files belonging to specific
users and user groups
File Paths - Allows you to select and exclude files based on file location and specific
file characteristics (e.g., file extensions)
SQL Query Strings - Allow you to use SQL query-like commands to define more
complex rules based on your requirements
For backup, DCE is supported with Windows and UNIX File System iDataAgents and also with
Microsoft Exchange Mailbox backup, archive, and content indexing iDataAgents.
DCE is also used by the Storage Resource Manager for data collection.
Configuring Subclients
Configuring Subclients
If all data had exactly the same protection management requirements there would be no need for
subclients. But all data is not created equal. Some data needs to be retained longer while other
data needs to be protected more frequently. Some data may need to go directly to tape media for
immediate offsite storage and others - not. The functionality of subclients allows separation of
content and its management as business and the content may require.
Content is mutually
exclusive within a
backup set
Can use wildcards
in defining content
Limited to last level
of data path
Archive Content
Data protection content is defined at the subclient level for File System Agents. The default
subclient, created during installation, has the unique characteristic of including all protected data
not explicitly covered by other subclients within the backup set. Additional subclients can be
created by the administrator to separate and manage a subset of the backup set data.
The content of each subclient is mutually exclusive within the backup set. This means the
content, as defined by one subclient, will not be protected by another subclient. The only
exception to this is the NAS NDMP Agent subclient. Administrators must manually configure
NDMP subclients to not have overlapping content.
In the case of application subclients, the inclusive feature of the default subclient means it can
automatically "discover" all data requiring its protection. In some Agents, this automatic
discovery feature can be disabled (for example: SQL Server), filtered (for example: Lotus
Notes), or otherwise configured to assign content to other subclients (for example: Exchange
2000/2003 mailboxes and GroupWise databases).
For a Data Archiver subclient you can establish the criteria to determine which content will be
scanned (i.e., mailboxes, libraries, volumes, folders, and directories) and which data objects will
be archived (i.e., messages, items, and files). Depending on the agent type, archive rules can be
configured for on variations of disk space, quota, time, and/or size. With Data Archiver for
Windows with Data Classification Enabler installed, you can further expand the content rules to
include user and group ownership.
The use of regular expressions in defining content is limited to the last level in the data path. For
example: F:\Users\[A-L]* is an acceptable use of regular expressions in defining content,
whereas *.pst is not an acceptable use.
For File System subclients, the use of regular expressions requires the explicit enabling of the
subclient option to Treat characters as regular expressions. This is required for some operating
systems (for example: MAC OS), because certain characters such as * or ? can be part of a
legitimate file path or name. Keep in mind that this option does not appear in the default
subclient since use of regular expressions in the default subclient is not supported.
Using Filters
M De e
es le
sa t e ach s
ge d C ile
s F
Global Filters /T
M
P k
Unix, Windows, ba
*.
Netware
Local Filtering
Exclusion
Exception Inbox
Subclient policy filters User Files
System Files
Using Filters
Filtering unnecessary data from data protection operations can reduce backup time, storage
space, and recovery time. Most, but not all Agents include some filtering capability at the
subclient level.
File systems' subclient filters can be defined for a path, directory, or file level. The exception is
NAS NDMP Agents which only support file names or directory names. Some application Agents
such as Exchange Mailbox, Lotus Notes Database, and Lotus Notes Document also allow filters
to be defined.
Exception filters can only be defined at the subclient level for supported agents. An
Exception filter allows you to define directory or file exceptions to a filter defined in
the exclusion section. For example, you can "exclude" the C:\Temp directory with the
"exception" of C:\Temp\log directory. In some cases, wildcards are supported in the
Exception filter.
Subclient Policy filters are made up of Exclusion and Exception filters and applied
by a subclient Policy. You can elect to not include subclient Policy filters in a
subclient and thereby define your own Exclusion and Exception filter for each
subclient.
CommCell Level Policy (Global) Filters are made up of Exclusion filters that can
be defined for each various operating systems and applications. Note that both Linux
and MAC OS fall under the UNIX global filter. Filters defined at global level can be
included or excluded in a subclient filter definition.
scripts/batch commands
Return code evaluated
Exception: Netware
No timeout limit
Data Protection/Archive jobs consist of several sequential phases. Each phase must complete
before its successor can begin.
The Subclient Properties (Pre/Post Process) tab allows you to use these phases as triggers to start
pre/post processes. For example, copying files from a share to your local disk and then deleting
the files after the data protection/archive operation has run, or you might want to run a database
integrity check just prior to backing up your databases or workspaces. Pre/Post processes can
also include tools, services, and messages.
For each batch file/shell script that is associated with a subclient's Pre/Post processing phase, the
system examines the return code. A return code of zero indicates the phase was run successfully,
and the job continues. A non-zero return code is interpreted as a failure condition, the phase is
retried, and the job is put into a pending status.
Under normal circumstances a specified post process command is only executed upon the
successful completion of the respective job phase. However, you may still want to run a post
process even if the job phase did not complete successfully, especially if your post process is
used to bring a database online or release a snapshot. The system provides you with an option to
run post processes when the respective job phase is killed, fails, completes with one or more
errors, or is interrupted due to a media reservation issue--in addition to successful job phase runs.
The Run Post <phase> Process for All Attempts option can be selected on the subclient
Properties (Pre/Post Process) tab dialog to enable this functionality.
Keep in mind that there are a couple of scenarios where a post process will not be executed for
all attempts to run the job phase, even if this option is selected: when a user suspends/stops the
job during the respective phase, or if the job is interrupted during that phase due to Operation
Window rules. If you encounter either of these conditions and your post process is designed to
bring a database online, release a snapshot or execute some other critical process, you may want
to manually perform these tasks depending on the situation.
NetWare will not wait for pre/post processing commands to complete, unless a command delay is
configured for that process. If a command delay is not configured, NetWare will immediately
move to the next phase of the backup job after launching the command. Also, command failure
will not prevent the next phase of the backup job.
Deduplication
Enable or disable Data Compression either on the client or the Media Agent.
Configure the transfer of data in the network using the options for Network
Bandwidth Throttling and Network Agents.
Although the Data Compression setting is editable on the target subclient, once it is
changed there, that target subclient will override or ignore any further Data
Compression settings specified or changed in the associated subclient template.
Deduplication
Deduplication identifies the duplicate items in a data protection operation and maintains
references for the duplicate items. When a data protection operation is performed for the first
time all the data is stored; if the same data is subsequently identified in another data protection
operation, reference counters are incremented and the data itself is physically not stored in the
storage media.
Note that there are appliances that support deduplication of data (e.g., Centera). Such hardware
can be configured as a magnetic library with the option to write the data in a deduplicated format
enabled. Deduplication is supported by both backup and data archival products, specifically for
the file system and email attachments, in order to provide optimization in storage when copies of
the same data is backed up and stored.
Subclient Policies provide the facility to define various subclient properties such as subclient
contents, filters, storage policy, etc. as a common template and attach the subclient policy to
appropriate clients based on their file system. Mutual exclusion across different subclients
defined in the subclient policy is enforced.
A user specifies the file system type (Windows File System, Windows File System with System
State, File System, Netware File System) for which a subclient policy is being defined. One of
the subclients defined in the subclient policy is designated to be the default subclient (Used for
Indexing reasons). Cloning of a subclient policy will be supported.
Subclient Policies are created at the subclient Policy logical node in the CommCell Console.
When a user chooses to associate a subclient policy, the default subclient is changed to point to
the subclient that is marked as default within the subclient policy. Other subclients are created on
that backup set and pointed to the corresponding subclient Id from the subclient policy.
The name of the existing default subclient is also changed to the name of the default subclient
within the subclient policy.
Module Summary
Key points to
remember
Summary
Backupsets and Archivesets are used to logically group subclient content making data
recovery easy.
You can define data for various user types independently of where the data is located,
by using the Data Classification Enabler.
Some common property settings for subclients include defining content, establishing
content filters, using pre/post processing commands, and selecting storage options.
Subclient policies allow you to define multiple properties in a common template and
attach this policy to appropriate clients.
Job Management
www.commvault.com/training
Overview
Overview
Objectives
List and describe the common task management tools used for scheduling, controlling
and executing jobs.
Describe the various job options available when configuring and executing Backup
and Archive jobs.
Configuring Data
Protection Tasks
Data Protection is a comprehensive term used to account for both Backup and Archive jobs. All
jobs are managed by the JobManager service run on the CommServe host. Jobs are configured
and run in distinct phases where the status of each phase determines whether the next phase will
be run.
Backup
Full
Full backups are required and are the preferred means of backup in terms of data
protection and restore speed. A full backup captures all data as defined by the subclient
content. Full backups are the preferred method of data protection given that sufficient
resources are available.
Incremental
Incremental backups are the most efficient form of backup capturing only the data that
has changed since the previous backup. While incremental backups save space and time,
a full subclient restoration would require access to the most recent full backup plus all
incremental backups up to the requested restore time. This may involve more media and
time to perform a successful restore.
Differential
Differential backups combine fast restore with a more efficient use of storage than normal
full backups. A differential backup captures all data that has changed since the previous
full backup. Each differential backup is then cumulative of all data changes. A full
subclient restoration would require the most recent full backup plus the most recent
differential backup. This could result in faster restore jobs.
Synthetic Full
Synthetic Full is listed as a data protection (backup) option for some subclients, but it is
not a genuine data protection job since it obtains no data from the source subclient.
When a normal backup operation is executed, an image file is generated. This image file
contains a list of all objects within the scope of the subclients defined contents
excluding filtered files. These objects are collected from data already in protected storage
and written back to protected storage as an equivalent full data protection job. Actual
scheduled protection is not achieved until an incremental backup from the client is
completed. The synthetic full task includes an option to run an incremental backup just
before or after the synthetic full is run.
A synthetic full may be used when a normal full backup cannot be reasonably performed
due to client or network resource limitation.
Archive
Understanding
Synthetic Full
DA
Incremental
from client
before or
after
Synthetic
Full
As the name suggests, a synthetic full backup is a synthesized backup. It is created from the most
recent full backup (i.e., standard or synthetic) and subsequent incremental and/or differential
backups. The resulting synthetic full backup is identical to what would have been created had the
last backup for the subclient been a full backup. Unlike full, incremental, and differential
backups, a synthetic full backup does not actually transfer data from a client computer to the
backup media. In this sense it is not really a backup at all, rather a backup consolidator.
Because synthetic full backups consolidate backup data into one archive file, they provide a
means of enhancing restore performance, particularly when a given full backup cycle contains
many incremental backups.
An example of the usefulness of synthetic full backups is the case where a Client is remotely
located from a Media Agent, with a WAN connection that is limited in bandwidth. In such a case,
you may not want to perform full backups on a regular basis across the WAN, but you do want a
archive file of your backup data to enhance restore performance.
Synthetic full backups are treated by the system as standard full backups. As a result, they
delineate full backup cycles in the same manner as standard backups. This is an important point
to remember, since retention periods are defined in part by the number of full backup cycles.
Using synthetic full backups in an unguided manner may cause the unintentional expiration of
data.
Synthetic full backups can either be started manually or scheduled to occur at regular intervals.
Executing Data
Protection Tasks
Job Initiation Options
Advanced Backup/Archive
Options
Creating a Schedule Policy
Individual Data Protection jobs run at the subclient level. If you chose a level higher than a
subclient (i.e., backup set, etc.), you are prompted to confirm that you want to back up all the
subclients below that level/node.
Schedule Policies
Command Line
On Demand
On Demand Data Protection Operations allow content to be specified as an external input at the
time of initiating a data protection operation. Whereas traditional backups/archive operations are
performed on subclients which have fixed content configured prior to performing the operation,
On Demand Data Protection Operations allow you the flexibility of specifying content each time
you perform a backup or archive operation.
Schedule
Scheduled data protection operations provide a convenient means of securing data without user
intervention. You can establish data protection schedules for each subclient using the CommCell
Console. When scheduling data protection operations, you need to establish a schedule for each
subclient. For example, a backup schedule always contains a full backup and may contain one or
more other backup type operations. When combined for a given subclient, these backups
comprise a full backup cycle.
Use the CommCell Browser to schedule or initiate backups at the subclient, instance and/or
backup set level depending upon the agent. Selecting the backup set or instance saves you from
having to select the individual subclients. If you select a backup set or instance, the same
schedule or data protection operation requested is applied to all subclients.
Schedule Policies
If you have a large number of clients/backup sets/subclients or storage policies in your
CommCell group that require the same backup or auxiliary copy schedule, you can manage their
schedules through a Schedule policy. Schedule policies allow you to associate a schedule or
groups of schedules to any number of clients/backupsets/subclients or storage policies within
your CommCell group. There are two types of schedule policies, Data Protection and Auxiliary
Copy. During the CommServe system install the System Created schedule policy covers all
supported agent types. Additional schedules can be created to support individual agent or all
agent types.
A schedule policy allows you to define a maximum of six schedules, and associate them by agent
type to any number of client computer groups, client computers, backupsets, or subclients. You
also can define the type of schedule such as Data Protection operation or Auxiliary Copy
operation; set up an alert; as well as configure Advanced Options as described in the Advanced
Backup/Archive options section. In the case of an Auxiliary Copy operation, the Schedule Policy
association is to the Storage Policy.
Decoupling is used to remove a subclient from its association from the Schedule Policy, but does
not delete the scheduled actions. Decoupled scheduled actions are treated as individual actions
and can be edited or deleted as such.
Command Line
The Command Line is useful for incorporating Data Protection and Recovery operations into
your own scripts and scheduling programs. Using the Command Line interface you can create
an action using a single command with arguments or create scripts that include an answer file.
An easier method may be to use the CommCell Browser to generate a script and answer file
using the options you select.
The commands are integrated with the base component installed with all agents and are therefore
available on all computers which have any CommServe system, Media Agent, or Agent software
installed. In order for the commands to function, the CommVault Commands Manager service
should be up and running on the CommServe system.
Automatic
Automatic auxiliary copy operations are performed every 30 minutes to copy data to the
automatic copy. These operations will only occur when new data that needs to be copied is
found on the primary copy (source). You can change the interval at which the automatic
auxiliary copy operation is performed on the job initiation tab of the auxiliary copy dialog box or
by editing the Automatic Copy Schedule.
Advanced Backup/Archive
Options
Data Options
Startup Options
Job Retry
Media Options
Data Options
By default, the advanced backup option to Follow Mount Points option is checked. This option
allows the Windows operating systems to be backed up including any drives mounted within that
subclient. If a drive is mounted as a folder it will be backed up when this option is enabled. This
can sometimes result in data on the drive being backed up twice if the resource also belongs to
another subclient. UNIX local mount paths are automatically included in the Subclient. NFS and
CIFS (UNC format) mount paths can be included in a backup/archive if explicitly added to the
subclient content.
Startup Options
Priorities can be defined at various levels within the CommCell environment. The priority levels
can be set based on the Client, Agent type or job, as well as on the fly through the Job Controller
window. When creating a backup operation, use the Priority settings to control the job priority
level for the specific operation that is being configured. How CommVaults software priority
and pre-emption features function is discussed later in this module
.
The Startup in Suspended State option will place the job in the Job Controller in a suspended
state. This is useful when starting the job is dependent upon other jobs or an external event
whose timing cannot be pre-determined or detected electronically. When more than one job is in
suspended state, the backup administrator can decide which operations should begin first.
Media Options
Depending on the type of operation being performed (Full/incremental/differential), different
options will be available on the Media tab. When performing full backup operations, the option
to Create New Index will be available. By default this option is selected. If cleared the previous
full index will continue to be used. Over time, the Index files can grow significantly larger
requiring more space to backup and slowing down browse operations.
If manually creating a new index, consider scheduling a monthly full job that will create a new
index, and for each Full task during the month. This will allow you to browse up to the last
monthly full, but will keep the Index Cache from growing unmanageably large.
When configuring a backup operation the Start New Media and Mark Media Full options can be
used to determine which jobs will be placed on a specific media. With careful planning and
scheduling a set of data protection operations can be scheduled to run and all of these jobs can be
placed on a single media. Use the Start New Media for the first job and Mark Media Full for the
last job creating keystone operations. These settings are also useful when performing Auxiliary
Copy operations.
In 6.1 an option was added to Allow other Schedule to use Media set. This option is enabled by
default, and can be disabled only if both Start New Media and Mark Media Full options are
selected. Thus, if Start New Media is also selected, all jobs within the same schedule policy
event will be allowed to use the same active media started by the first job.
If the option to Mark Media Full is enabled, the system will wait 10 minutes after job completion
to actually mark the media full. Any job (schedule policy or not) requesting to write to the media
within the 10 minute time frame will be allowed to do so. Jobs within the same schedule policy
event will not have that limit enforced since the Allow other Schedule to use Media Set option is
enabled.
Clearing the Allow other Schedule to use Media set option and enabling Start New Media and
Mark Media Full will isolate each job on its own media as you should expect.
Template for
bulk scheduling
jobs
Scalability tool
Granular control
down to
Subclient level
Scheduling individual data protection job patterns for two, three, or maybe a dozen clients is a
reasonable task load for an administrator. Doing the same for 300 clients can be challenging
without some assistance. The Schedule Policy is a template-based management tool that can be
applied across any number of client systems within a CommCell environment. Schedule policies
allow you to associate a Data Protection or Auxiliary Copy job schedule pattern to any number of
clients, backupsets, or subclients. You can define which type of data protection operation you
want to schedule as well as configure Advanced Backup/Migrate/Archive Options and assign
object association in the CommCell Browser.
The Agent Specific schedule policy allows you to define a maximum of six
schedules, and associate them to any number of client computers, backupsets, and
subclients for a particular agent. An Agent specific schedule policy allows use of all
applicable data protection job options available for that agent.
An All Agent Types schedule policy is a generic data protection schedule policy that
has most of the same options as a regular Data Protection policy. It is supported by
all agents that support scheduling. Only generic data protection job options
applicable to all agent types are available.
submission only)
QCommands provide access to most CommCell environment functions through the command
line, and can be integrated into your own scripts and scheduling programs. Note that scripts can
also be generated through the CommCell Console for some features using the Save as Script
option. All commands have consistent output in all situations to facilitate easier script writing.
Although most CommCell environment operations are available through the command line, the
recommended method for those not familiar with scripting or command line syntax is to use the
CommCell Console as it provides comprehensive support for all options available in the
software.
Configuration
No special configuration is required to use the command line interface. The commands are
integrated with the base package, and are therefore available on all computers which have any
CommServe system, Media Agent, or Agent software installed.
In order for the commands to function, the CommVault Commands Manager service should be
up and running on the CommServe system. The CommVault Commands Manager is a service
that is installed with the CommServe system, and is responsible for handling command line
requests and forwarding them to the Event Management Service of the target CommCell
component.
Log in Sessions
Using the Qlogin command, you can start a User login session, removing the need to log in for
every command. Once the Qlogin command is successful, the login session runs as a service for
all command sessions and remains valid until you explicitly log out using the Qlogout command.
Of course, the service can be killed by an administrator. There is no timeout.
Script Considerations
Consider the following before you save a job as a script:
When you use the CommCell Console to save a script, only the options that are
available for the corresponding QCommand are saved. For example, the Job Retry
option is available in the CommCell Console for several operations such as backup
and restore. However, this option is not available in the corresponding QCommand.
Therefore, if you save a Script with Job Retry options selected, they will be ignored
when the script is generated.
When you use the CommCell Console to save a script, certain characters (for
example, the left bracket [ and number sign #) in front of an object name (for
example, #csmacs as the subclient name) may cause the script to not work correctly
when it is run from the command line. If your script does not work from the
command line, check it for these characters. You may need to rename the object by
removing the characters, and then rerun your script.
When you use the CommCell Console to save a backup script for a Microsoft Share
Point 2003 Document Agent and run it from the command line, the script will only
back up the latest version of a document. This will happen even when you choose the
"Backup All Versions" option when generating the script.
Activity Control
Job Management Options
Activity Control
CommCell environment
Client Level
Agent
Subclient
Activity Controls
Use the Activity Control tool to enable or disable all activity, data protection, data recovery,
auxiliary copy, or data aging operations within the CommCell environment (regardless of the
client computer from which they originate). If you want to control these operations that originate
from a specific client computer or agent, use the Activity Control tab of the property sheet of the
specific client computer or agent.
The Activity Control feature allows you to enable or disable operations including scheduled
operations at the following levels in the CommCell hierarchy:
Client Computer Group - Allows you to enable/disable all data protection, data
recovery and/or online content indexing operations on all client computers that are
members of a client computer group.
Client Level - Allows you to enable/disable all data protection, data recovery and/or
online content indexing operations on a specific client computer.
Agent - Allows you to enable/disable the data protection and/or data recovery
operations of a specific agent on a specific client computer. Online Content Indexing
jobs can be disabled for Online Content Indexing Agents.
When disabling activity operations, the CommCell level has the highest precedence while a
subclient has the lowest precedence. For example, if you disable data protection operations at the
CommCell level, then all data protection operations throughout the CommCell environment are
disabled regardless of the corresponding settings of the individual client computer groups, client
computers, agents, and subclients. If, however, a data protection operation is enabled at the
CommCell level, you can still disable data protection operations at the client computer groups,
client computer, agent, subclient levels. By default, all operations are enabled at all levels of the
CommCell hierarchy.
Operation Window
Set Holidays
Custom Calendar
Operation Window
By default, all operations can run for 24 hours. To prevent certain operation types such as Data
Protection or Auxiliary Copy from running during certain time periods within the day, you can
define operation rules using Operation Windows so that these operations are suspended during
those times. The main purpose of this feature is to help you prevent an unexpected, time
consuming operation from disrupting normal operations. Operation Window Rules can be looked
at as Blackout Windows
Operation Window rules are defined at both the CommServe system and agent levels. Rules
established at the CommServe system level apply globally across the entire CommCell
environment. Operation rules established at the agent level apply only to the specified agent.
When an operation rule is defined at both the CommServe system and agent levels, the job will
run outside of the total time frame of both levels.
When using Operation Window, consider the following, there are two types of Agents Indexed
base and Non-Indexed base. Simply stated, Index Based agents use the index cache and the Non
Index Based Agents will not.
An easy example of each might be index based types might include Agents managing granular
objects within a job like a file system backup or mail system messages. Non Indexed Agents
usually are database applications. A listing of these type can be found within Books Online.
Jobs that are started at any time within the Operation Window rules or Blackout period will
still be added to the Job Manager window in a queued state. Once the do not run interval has
passed, these queued jobs will resume automatically.
Jobs that are started before an operation rule can run to completion if the Allow running jobs to
complete past the operation window option is enabled from the General tab of the Job
Management dialog box.
Jobs that are not interruptible (such as Non-Indexed based jobs) will not be terminated if they fall
within the time an operation rule is defined.
Set Holidays
If you need to define days and times that you do not want your schedules to run, you can
establish holidays for those schedules. You can establish a holiday as an annual holiday or for a
specific date within a given year. Annual holidays never expire; other holidays are automatically
deleted once the holiday occurs. Only one holiday per date is allowed. If you set a holiday to
occur on an annual basis, this supersedes any holiday that is already set for the same day.
Custom Calendar
The Custom Calendar feature gives users the ability to create custom calendars so that their
operations can be run within the boundaries of their own custom calculated time. A company
may base their custom year on a calendar that starts in February, ends in January, and whose
months have a unique number of days. This feature allows users to use the same custom calendar
for their scheduled operation, and as the basis for copy data to storage policy copies, and for the
aging of data from these copies.
Job Management
Control
Queue Options
Pre-emption Control
Priorities
Restarts
Job Management options are available in the CommCell Console Control Panel. It sets the
default actions to take when a job starts and how to automatically manage the job when certain
events happen. The areas of job management are:
Queue Options - If a job conflicts with another job or control setting, what action
should the job take? In some cases you may want the job to queue and wait. In other
cases you may not want the job to start.
Pre-emption Control - If limited resources are being competed for, what action
should the Job Controller take when a higher priority task is started? Should lower
priority jobs be pre-empted or not? Select or clear the options as necessary. Note that
in a Gridstor technology enabled storage policy, pre-emption is suspended.
Priorities - Set whether a client or an application has priority. You can also set the
respective application priority.
Job Priorities are based on a three-digit integer. The first digit always represents
operation priority. If client precedence is chosen, the second digit represents client
priority and the third digit represents agent priority. If agent precedence is
chosen, the second digit represents agent priority and the third digit represents
client priority. The lowest job priority number has the higher priority. Hence, a
job with a priority of 066 will run before a job with a priority of 366 if a
resource conflict is an issue.
Restarts
When a job phase fails it will normally go into a pending state awaiting an automatic restart. The
frequency interval and number of restarts can be configured. The default is every 20 minutes and
144 times (48 hours). You can also set the option on some job types as to whether they are
restart-able or preempt-able.
Manage
Data Protection operations
Data Recovery operations
Administration operations
The Job Controller allows you to manage the following types of jobs:
Data protection operations
Data recovery operations
Administration operations
You can select a job in the Job Controller and perform a control action on that job individually. If
you have multiple jobs in the Job Controller that need actions performed on them, you can
simultaneously control groups of these jobs from the Multi-Job Control option displayed when
you right-click on a job.
The various states of the job reflected in the job controller window are self explanatory (e.g.
running, waiting, suspended, completed).
Listed below are the less frequently encountered but equally important:
Dangling Cleanup -The job has been terminated by the job manager, and the job
manager is waiting for the completion of associated processors before killing the job.
Pending - The Job Manager has suspended the job and will restart it without user
intervention. A pending job may be waiting.
Queued - The Job Manager has queued the job and will restart the job only if the
condition which caused the job to queue has cleared.
Waiting - The job is active, waiting for resources (e.g. media or drive) to become
available or for internal processes to start. The job many also be started in the waiting
state if a queue option was selected from the General tab of the Job Management
dialog box.
Module Summary
Key points to
remember
Summary
There are many different options for Data Protection depending on the needs of the
organization.
Global options for job queuing, job pre-emption and job priorities can be configured.
The Job Controller window of the CommCell Console offers the ability to
dynamically suspend, resume, or kill jobs using the Multi-Job feature.
The SQL meta data for the CommServe system is protected in at least two locations
with one operation.
Data Aging will purge data that has exceeded retention. Data is aged differently
depending on what type of media it was written to.
Restore
www.commvault.com/training
Restore Module
Overview
Overview
Objectives
List and describe the common methods for locating data in protected storage for
restore/recall.
List and describe the common methods and options for restoring and recalling data.
Describe the requirements and steps for performing a full File System/Application
Agent restore.
Describe how to use and manage stub files for recall of data.
Image/No-Image Browse
List Media & List Media (Precise)
Exact Index
Find/Search
Image/No-Image Browse
D:\DIR
1/1/06
1/2/06
1/3/06
1/4/06
No Image
FileName 1 2 3 4
A.txt
B.txt A.txt
C.txt B.txt
D.txt
C.txt
E.txt
F.txt D.txt
G.txt E.txt
H.txt
F.txt
G.txt
H.txt
Image/No-Image Browse
Image Browse
Each indexed backup generates an image index of all files/objects scanned at the time of the
backup. The image will always contain all files/objects scanned at the time of backup regardless
of the backup type (e.g. full, differential, incremental). This image index is used when you:
A backup cycle that spans a period of time will have image indexes for each backup job
contained with the index. For example; a subclient doing weekly full backups with daily
incremental backups will have seven (7) image indexes for the list of files/objects it backed up
during the cycle one for each backup job/day. Selecting a point-in-time browse of the backup
data will use the appropriate image index to show you what files/objects existed on the
system/application at that time.
No-Image Browse
What if youre looking for a deleted file thats not in the latest backup image? How do you find
it amongst the backup data? The basic answer is that you can search for the file by doing point-
in-time browses of the backup data. If you are not certain of when it was last backed up, this can
be a hit-or-miss exercise. Additionally, if youre guessing when it was backed up, it might not be
the latest version of the file. To ensure you got the latest version would require methodically
stepping backwards through each backup job. As you might suspect, this can take time. This is
where the Image Browsing option becomes valuable.
Disabling the Image Browsing option will create a browse display of all files within the index
from that point back to the beginning of the index. Deleted files not in the current backup image
but were captured in previous backups, would be displayed in the browse. Essentially, you are
browsing all files that were backed up since the index/cycle started to the selected point-in-time.
If your deleted file was backed up in this cycle, it should appear in the backup browse display. If
not, you can use the Find feature to scan through multiple indexes/range of time to locate the
deleted file or all versions of a file.
A caution with disabling the Image Browsing option is that if you are restoring directories or
drives, more files will be restored and the resultant volume may be more than the capacity of the
device.
List Media
Media Containing Index
Media Containing Jobs within specified
time range
List Media (Precise)
Lists media for specified objects
selected for restore
Lists media for specified subclient
List Media
List media option is useful to predict media required for the following operations:
To ensure that media required by an operation is available in the library, especially if you
are restoring/recovering data across a firewall.
In cases where data spans across several media, to identify the exact media necessary to
restore/recover a file/folder/sub-section of the data.
To identify and restore/recover from a copy that accesses a faster magnetic disk media
rather than slower tape/optical media.
To identify media associated with an alternate copy, when the media containing data associated
with a specific copy is not readily available due to the following reasons:
List Media Associated with a Specific Backup Set, Instance or Subclient is referred to as List
Media in the CommCell Console and provides the following options:
Search media associated with the latest data protection cycle, starting from the latest full
backup. (This is the default option.)
Search media associated with data protection operations performed between a specified
time range.
Search for media associated with a specific storage policy copy, synchronous or selective
copies, with the specified copy precedence number.
Keep in mind that when you search media from a secondary copy, the listed media may not
reflect the entire instance or backup set data, unless all the storage policies associated with all the
subclients have been configured for secondary copies.
For example:
When a data protection operation spans across multiple media and you would like to
know the exact media in which the files you wish to restore reside.
You have a specific set of files (either a random set or a specific set, such as *.doc or
*.txt) that you wish to restore and would like to know the all the media in which the files
reside.
You wish to restore a specific version of the file and would like to know the specific
media in which the version resides.
Exact Index
Past Index Current Index
Exact Index
Object M T W Th F S S Object M T W
A.txt X X X X A.txt X X
B.txt X X X X X X B.txt X X X
C.txt X X X C.txt X
D.txt X X X X D.txt X X X
E.txt X X X X E.txt X
Exact Index
When you browse indexed backup data, the default index set restored/used will be the last index
set generated in the backup cycle. This is true even if you are doing a point-in-time browse. This
means, that no matter when within a backup cycle you want to browse, the last index set
generated in that backup cycle will be used.
There could be circumstances where the media containing the last index set in the cycle is not
available (e.g. disaster recovery, remote restore) or the index set is lost/corrupt. In this case you
can select the Use Exact Index option available in the Advanced Browse options window.
The Use Exact Index option is useful in situations when you want to restore data from specific
media. For example; when media is off-site and you would like to maximize the chances that the
necessary index information comes from the same media.
Find/Search
Find
Available at backup/archive set level
Spans indexes
Search
Available at all levels
Uses content indexing
End user or Compliance
CommCell console or Web-based
Find
The Find feature allows you to search the data protection archives for any file or directory name
or name pattern. Because Find supports the capability of searching multiple indexes, unlike
browse, you can search beyond the last full backup (or new index) as long as the data resides on
an index that exists within the retention period. The Find feature is accessible as a right-click
option on the browse window, and for supported agents a non-browse find is available from the
All Tasks menu. Depending on the agent, the Find option is accessible from either the backup set
or archive set level.
The Advanced Search tab of the Find feature is accessible only to clients that support Content
Indexing and have created a content index for their archived or backed up data. Advanced
Search allows you to search by content (keyword or phrase). For Exchange agents, you can also
search by To, From, CC, BCC and attachment name. Advanced searches with multiple criteria
can be performed. In addition, Exchange agents that support Content Indexing can also narrow
search results by specifying a received time range for messages on the Find (Receive Time
Range) tab.
When using Find, be sure to specify a time range to prevent an unbounded search through all
past data.
Search
Search is possible when a Content Indexing Engine is installed and content indexing jobs have
been successfully completed for either Offline (protected storage) or Online (production server)
data.
However, Data Archiver data residing on CIFS shares do support End-User Searches.
Note that the search function from the CommCell Console does not provide end user search
capability. You must use the Web-based Search Console or Data Archivers Outlook Add-in for
end user search criteria.
Compliance Search
The Compliance Search capability allows compliance officers full access to all computers and
supported applications for searching, regardless of ownership/access attributes for the piece of
data.
Understanding
Restore and Recovery
Restore/Recall Options
Full System/Application Restore
Restore is the act of moving data from protected storage back into a production environment
where it can be accessed through normal production tools. Recovery is a subsequent phase
applied to application data where the restored data is recognized and functionally used by the
application. For example; a set of database files can be restored to production disk space but
they are not usable until the application recognizes, validates, and makes them accessible
through the application. This is sometimes called a Soft Recovery.
Restore/Recall Options
Restore by Job
Recall Archived Objects
Copy Precedence
Restore from anywhere
Out-of-Place Restores
Mapped Restores
Restore to Non-Client hosts
Restore/Recall Options
The simplest form of restore is to copy an object back to its original location. However, not all
restores are simple. The process can be intricate with requirements to filter, redirect to alternate
destination or path, use alternate source, and otherwise manipulate the data being restored.
Flexibility in restore, recall, or recovery of data is an essential feature in any data protection
product.
Restore by Job
Accessible from BackupSet level
Multiple jobs can be selected
Newer files overwrite older files
For CommServe DR use DR
Restore option
Restore by Job
The Restore by Job feature provides the facility to select a specific backup job or set of jobs to
be restored. This method of restoring data is faster under some circumstances as it reads
continuously from the tape and retrieves the data independently rather than using the indexing
subsystem which does individual seek offsets on the media.
A key advantage of restore by jobs is restoration of any successfully written data in failed or
killed jobs.
If multiple job streams were used in parallel for the data protection job, then Restore by Job can
use those same multiple streams in parallel for the restore. The degree of parallelism would be a
factor of the degree of multiplexing for the jobs involved. The higher the multiplexing, the more
parallel the restore.
Optimally, youd like each job stream assigned to a device stream on a 1-to-1 basis. Multiple
stream parallel restores in these cases would be as expected. On the other hand, if the job streams
are multiplexed to the same tape, then a single device/job stream to the client will be used. The
parallel restore occurs at the client, where multiple threads will be generated to de-mux the
single stream and restore the data accordingly.
A standard browse/index-based restore for more granular restore may be more appropriate at
times and can be faster by not restoring unnecessary objects. However, an index-based restore is
single stream only, regardless of how many streams may have been used in parallel by the data
protection job. Restore by Job cannot be used to restore the Windows System State data.
Temporarily or permanently
bring back objects from
protected storage
Persistent Pipeline established
The recall or non-browse recovery feature provides the facility to transparently and either
temporarily or permanently brings an object back from an export location for a specific
operation. Transparency is provided by a stub object substituting for the actual object which
initiates the recall automatically when accessed through normal means. The temporary capability
is useful if you need an object for a specific purpose or length of time and do not wish to reset
the objects archive criteria. A permanent recall is useful if the object being recalled will be used
multiple times and you wish to reset the objects archive criteria.
When performing a non-browse recovery of archived data from magnetic media or tape, the
system maintains a common open pipeline known as a persistent pipeline between the target
client and the protected media source for multiple stub recall requests. The pipeline stays
persistent for 20 minutes (by default) to reduce the time to open a pipeline for each individual
stub recall request. The initial object being restored may take a few seconds more time as the
pipeline is established. Subsequent restores are very quick.
When performing a non-browse recovery of archived data from a stub, keep in mind that the data
can only be recovered in-place to the same path/destination on the same client from which the
data was archived.
If the message or item you want to recover was archived using the Do not create stub option or
the stub was manually deleted after archiving, the object must be restored using the browse
recovery method.
Stubbed, non-browsed restores can only be recalled in-place to the items previous location.
However, for the Exchange Mailbox Archiver Agent, the restore destination can be the same
mailbox, a different mailbox, or a PST file on another Exchange Server within the same
organization and site with a compatible Exchange Mailbox Archiver Agent installed and
operational.
Important Notes:
Prior to performing a PST migration archiving operation or a PST recovery operation with the
Exchange 2003/2007 Mailbox Archiver Agent, ensure that the MAPI32.dll file has been copied
into the <Software Installation Path>\Base installation directory. After completing a PST
archiving or recovery operation, ensure that the MAPI32.dll file is removed from the <Software
Installation Path>\Base installation directory on the Client, otherwise non-PST
restore/recovery/retrieve jobs for Exchange may go into a pending state.
When data is archived from a server to another location, a pointer - or what we call a stub - is left
in place of the file. This stub enables transparent recalls of the data. Although the data has been
moved, it is still available to the user or application as if it was still on the server. After a stub has
been recalled, the icon will change back to the normal icon for the respective file type. The only
visible difference between a stub and the file is the icon representing the file. The icon is
changed to denote the archived status of the data. The icon doesnt change much, the original
application icon will still appear but with a small clock in the left bottom corner. This represents
a potential delay in accessing the file while its being recalled from remote storage.
Copy Precedence
iDA
1 The
Each
If aaoption
Each
If
The copy
Storage
precedence
option
copy to
Storagetospecify
Policy
PolicyCopy
precedence
specify is
ais
Copy
a
Client has
specified,
specific
hasaarestore
specified,
specific copy
only
restore
copy precedence
onlyprecedence
that
thatcopy
precedence
precedence
copy
number.
will
for be
restore
number.
will
for be used
If
Ifno
restore
used may
for
no specific
may
for restore
be
beused
specific usedto
restore to
MA Magnetic Library precedence
whether
bypass missing
precedence
whether
bypass or not
is
missing
or notused,
or
more
is used,
or suspect
more Galaxy
suspect
Galaxy
will
relative
backup
willuse
usedata
relative
backup each
data
dataor
data
each copy
exists
or to
toforce
copy
existsinforce
inon
on use
use
precedence
other
of ananalternate
copies
precedence
other
of order
within
alternate
copies data
order
withinto
data the
to path
find
the
path
find
2 the
Storage
for
therestore.
data
Storage
for Policy
to
torestore.
restore.
data Policy
restore.
Tape Library
Remote Site
iDA
3
Client
Tape Library MA
Copy Precedence
Storage Policy Copy precedence is a priority sequence used by a restore job to determine which
storage policy copy to use when multiple copies are involved. Copy precedence is initially
assigned in the order in which a storage policy copy is created (e.g. 1, 2, 3 ). As such, the
primary copy is by default (and normally) the first precedence (#1) copy. However, the primary
status of a copy and its copy precedence are not related. While data from the client is always
written to the primary copy, there is no requirement that the primary copy also be the first copy
used for restore. The precedence sequence of a copy can be changed in the Copy Precedence tab
of the Storage Policy Properties dialog window.
For a restore request, where the copy precedence is not specified (default), the software assumes
any copy will do and will check each copy precedence in sequence until it finds one available to
satisfy the restore request.
When you manually specify copy precedence in the restore request you turn off the automatic
copy selection and force the software to use the copy you selected regardless of whether it has
the most recent data or no relevant data at all.
Additionally, when manually specifying copy precedence, you should select the copy precedence
before browsing the backup data set. Browsing one copy and restoring from another may yield
unexpected results. For example; a restored directory size and content might not be the same as
you viewed in the browse.
The primary copy is no longer available for a data recovery operation due to a
hardware failure.
You know that the media containing the data from data protection operations for a
particular copy have been removed from the storage library. In this case, you can
choose to browse/restore/recover from a copy whose media are inside the library.
You want to browse/restore/recover from a copy that accesses faster magnetic disk
media rather than slower tape media.
You know that the media drives used by a particular copy are busy with another
operation and want to browse/restore/recover from a different copy to avoid resource
conflicts.
You want to perform backups to a primary copy in one location and do restores from
a secondary copy in another location. (This can be pre-set for scheduled restores or
command line restore (Oracle RMAN) by altering the precedence of the secondary
copy on the Storage Policy Properties dialog window.
Tape Library
MA
?!
Import
Tape Library
AA
Data
Therestore
Data
The media
from
fromrequest
restore
media can
arequest
aclient
can then
client
then that
isthat
be
isbe
written
exported
requires
writtento
exported
requires data
from
to managed
data
from from
the
managedthethe
from the
library
media
library
media will
library.
and
will check
library.
and imported
check Bylibrary
imported
By into
libraryinto
managed
another
inventory
managed
another
inventory managed
towe
managed
to see
mean
wesee first
meanlibrary
an
first
library
an
inventory
within
who has
the
inventory
within
who has the
the issame
the maintained
media,
media,then
issame
maintained then
by
CommCell.
restore
bythe
thelibrary.
CommCell.
restore the
thedata.
library. data.
Data can be restored/recovered from any compatible library and drive type in the CommCell
group. By default the software automatically identifies and restores/recovers data from any
configured library in the CommCell environment, even if the media is not available in the
original library in which the data protection operation was performed. Data Recovery operations
use a specific data path Media Agent, Library and Drive Pool - to perform the restore
operations. When the default options are selected, the system automatically chooses the most
appropriate data path.
Media associated with stand-alone libraries can be imported into the regular libraries (and vice
versa) to perform Data Recovery operations. When moving media from a standalone to a
barcode-enabled library, make sure it has a compatible barcode label and change the Medias
Identifier in the CommCell Console to match the barcode number before importing it.
This feature is mostly used in a disaster recovery scenario where a new library is configured and
media from previous library(ies) are imported to perform restores. There is no requirement to
change a storage policys data paths to include the new library unless you intend to continue data
protection jobs with the new library.
Out-of-Place Restores
Same Client/Different path or name
Cross-platform
Compatible File System
Windows <-> UNIX/Linux - Yes
Netware <-> any other FS - No
Cross-application
Exchange Mailboxes on any version
All Databases Exact version only*
System State
Use Browse Options
Restore to different client is supported
Out-of-Place Restores
By default, data is restored to the same computer from which it was backed up.
You can restore to a different parent directory path by clearing the Restore to same paths
option. All files restored will have the new path pre-pended to their current path. Hence a file
with a fully qualified path name of D:\user\data\document\file.txt restored to a path of
F:\Restored would result in a restored file of F:\Restored\user\data\document\file.txt. Multiple
files with different path names would have the same new path pre-pended.
The Preserve source path option can be used to keep or remove levels of the restored path and
file name. This option would best be used in re-organizing directory structures in conjunction
with the restore.
Files can also be renamed by appending a suffix to the filename itself not the extension.
Maintaining the extension keeps the application association consistent.
To change the destination computer, you can select one from a list of established clients within
the CommCell environment, with Operating Systems/Applications that support the out-of-place,
cross-platform, or cross-application restore operation from this client. If the client name does not
appear, the cross-platform OS or application does not support the out-of-place restore.
Mapped Restores
Mapped Restores
When performing normal restore of the selected data youre given the option to designate one
specific destination. If the restored data needs to go to multiple locations (for example youre
using a restore to restructure your primary storage) you would need to perform multiple restore
jobs. Using a map file provides the ability to restore individual files and folders within a single
job to different locations.
The map files contains a list of files to be restored with their corresponding restore paths. In
addition the mapping options also provide the ability to specify whether the unmapped files that
were selected for restore should be restored to the selected restore destination.
The map file restore options are also supported by Restore by Job feature and restores from the
command line. Map file restores are supported by the Macintosh, Unix and Windows Agents.
The map file should be a CSV (Comma Separated Value) file. On Windows, the map file can be
of any recognizable type, such as .txt, .doc, .xls, .rtf, etc. On Unix, use an editor such as VI to
create the map file. The format of the mapping should be as follows: "<source
path>","<destination path>
The specified paths for both the source and destination should be absolute paths within
the same client computer.
If the specified destination folder is not available, the necessary destination folder will be
created during the restore.
The source and destination type should be the same. For example, a folder must map to
another folder and a file to another file. (The results may be unpredictable if this is not
followed.)
Computer names or UNC paths should not be used in the source and destination names.
To filter a file, add the source folder and file and then add an empty double quotes as the
destination. For example: "C:\dir5",""
Use a mapping file only when there are a large number of files to be restored with
multiple destination paths. Consider the following examples: You wish to restore 10 files.
Within these 10 files, 8 files must be restored to the same location, while 2 files must be
restored to separate destinations - in this case select these 10 file in the Browse window,
specify the destination path for the 8 files to the common location and only include the 2
files in the map file.
As much as possible filter the files from the Browse window, rather than filtering the files
using a map file.
Restores to Non-client
hosts
UNC Path
DA Non-client DA
Impersonate
User
Restore
Backup
MA
By default, a restore job can only read/write data from/to the designated client. This is due to the
characteristics of the user which the software services run. Data residing on non-client systems
can be backed up by a client via a UNC path if an authorized user and password is provided.
Conversely, data can be restored via a client system to any non-client systems via UNC path if an
authorized user and password is provided.
When specifying UNC path content in a subclient, a dialog window will appear enabling you to
enter a user name and password. The same user name and password must be valid for all UNC
paths specified in the subclient content.
For restore, the Advanced Restore options to specify a different restore destination and to
impersonate a user must be used. The specified user must have write permission to the share
path and directory on the remote client.
Full System/Application
Restore
Recovers data files, directories
and system state
Performed via Browse at
Data Files
The difference between a normal restore and a full system restore is the severity of the problem.
Normally, if data is lost or removed, it is recovered from the archives using the normal restore
procedures. However, when a normal restore operation cannot correct a software and/or
hardware corruption problem, some level of full system restore is required.
If the root directory of a UNIX system or the System State of a Windows system is lost, a Full
system restore is required.
When you perform a full system restore, the client computer must have a functional and
compatible version of the Operating system. With UNIX, the default install partition should not
be on the same disk partition that will contain the restored root file system.
Full Agent restore always restores file system data in place. If the original path is not present, the
file restore will fail. All Partitions/disks included in the restore must be configured, accessible,
and of sufficient size to receive the restored data. If you are recreating and repartitioning the
system disks and do not know how large each volume was, you can browse the backup data and
view the size of each volume on the backup data browse screen.
If you are restoring to different hardware, other issues need to be addressed in order to
successfully and fully recover a system.
Application Restores
An application is made up of its managed data set and supporting executable and configuration
files. The application must be functional in order to restore its managed data. In some cases this
may require using a File System restore to get necessary supporting files in place. Additionally,
the application must be in a recovery state in order for a restore and full recovery to work
successfully.
For example; With Exchange the System Attendant Service must be running in order to restore
databases. The System Attendant Service will run without the presence of the databases.
However, if the databases do exist they must be enabled for overwrite by a restore. This can be
done using Exchange admin tools, or it can be done automatically by selecting the appropriate
option during an Exchange database restore.
On the other hand, SQL Server service will not run without the master, model, msdb and temp
databases. In order to do a full application recovery of SQL Server you must build suitable
replacement databases in order to start the SQL Service - and then restore your original databases
from protected storage. Microsoft provides utilities to rebuild the necessary databases.
The key thing to remember is that restoring an application to full functionality requires
cooperation between many parts. Understanding and practicing application recovery is essential
to successful recovery.
Module Summary
Key points to
remember
Summary
Finding the data to restore is simple, understanding the index cache is helpful.
Finding the media is simple and that information can be obtained at various levels
within the GUI.
CommVault software also offers search capabilities for data, but should be used
with date restrictions.
Full system restores are available for extreme situations where the entire system
needs to be recovered.
Monitoring
www.commvault.com/training
Monitoring Module
Overview
Monitoring Options
Alerts
Reports
Overview
Objectives
List and describe the common options available to monitor CommCell operations.
List and describe the value/purpose of the more commonly used reports.
List and describe the value/purpose of the more commonly used alerts.
List and describe options for report generation and alert notification.
Monitoring Options
CommCell Explorer
Job Controller
Resource View
Event Viewer
Monitoring Options
CommCell Explorer
Client Views
Job Views
Media Views
Configuration Views
Database Views
CommCell Explorer
The CommCell Explorer views provide a way to query information on the CommCell
components directly from the SQL database.
These views are provided in addition to the CommCell Console Report Selection feature.
You can use the views provided with CommCell Explorer or customize them to reflect the data
in any manner appropriate to your organization. You can query the database with Microsoft SQL
Mgmt Console or any SQL database query tool. Query results can be displayed through Explorer
or you can use products such as Microsoft Excel or Crystal Reports to format your query output.
If you modify a view or create a new view, you must reapply them after each new release.
See Books Online for a list of views included with CommCell Explorer
Job Controller
Job status updates every n minutes
Displays job information and any
errors
Filter job
Optional display by job
Current media
Associated events
Log files entries
Job Controller
The Job Controller allows you to monitor the following types of jobs:
Data protection operations
Data recovery operations
Administration operations
You can view detailed information about these jobs as well as job events and the media used for
each job. Information about a job is continually updated and available in the Job Controller
window. When a job is finished, the job stays in the Job Controller for five minutes. This time
can be changed by opening the control panel and selecting the Display control Job Filter tab.
Once a job is finished, more information about that job is obtainable using the Job History.
Job Updates
By default, the system updates the information in the Job Controller every time a chunk is
written, every 5 minutes and every 2048MB (2GB) of data transferred. These values can be
changed by opening the Control Panel and selecting the Job Management control Advanced tab.
Job Information
You can view the following information about a job in the Job Controller:
Job Status
Job Details
Media
Events
Log Files
Completed With One or More Errors -The job has completed with errors. This is
relevant only to Exchange 2000 Database, Oracle, and CommServe Express
Recovery, and File System backup jobs.
Dangling Cleanup - The job has been terminated by the job manager, and the job
manager is waiting for the completion of associated processors before killing the job.
Queued - The job conflicted with other currently running jobs (such as multiple data
protection operations for the same subclient), and the Queue jobs if other conflicting
jobs are active option was enabled from the General tab of the Job Management
dialog box. The Job Manager will automatically resume the job only if the condition
that caused the job to queue has cleared.
Resource View
Drives in Library
Media currently used
Resource View
The resource view displays all the drives in the library and the media that is currently used in the
drive at any point in time. The media information includes the barcode\identifier of the media
and Job ID associated with the job currently using the media. If the media is not used by a job
the resource view displays the Job ID as Cache Mounted.
The Resource View includes information on the operation phase to show the specific actions that
are occurring on the media in the drive such as writing/verifying/reading OML, loading,
unloading, etc.
From the drive level, you can release the reservation on resources, if the reservation is not
released when a job has failed or killed in the Job Manager.
In addition, for regular libraries the resource view also displays the slot information in the library
and information on media that is currently in the process of being exported.
For Magnetic libraries, the status and free space information for the mount path is displayed.
The Resource can be viewed from the drive list in the right-pane of the CommCell Browser
when you click Master Drive Pool or Drive Pool in the tree.
Event Viewer
Used to monitor CommCell
environment activities
Filter display by:
Severity Levels
# of Events available
Search events by:
Time Range
Severity
Job ID
Character pattern
Event Viewer
The Event Viewer allows you to monitor activities that are occurring within the CommCell
environment. This information is useful for troubleshooting and informational purposes. For
example, you may want to know if your system is experiencing hardware problems, or what jobs
have started or completed.
Some events will also generate an alert if an alert for that event is configured within the
CommCell environment.
By default, the maximum number of events displayed in the Event Viewer is 200. You can
modify this number by opening the control panel and selecting the Display control Event Filter.
You can modify the types of events to be displayed based on the severity level.
The default maximum number of events retained in the Event log is 10,000. This number can be
changed by opening the Control Panel and selecting System. The Event log is maintained in the
metadata database.
Some search queries can be saved and selected at any time from the Select from this search
query field. A search query based on a particular Job ID cannot be saved.
Alerts
Understanding Alerts
Alert Output Options
Recommended Alerts
Understanding Alerts
Understanding Alerts
CommVaults Alert feature provides real time notification on events occurring within a
CommCell environment. These conditions can range from minor occurrences that do not require
intervention to severe occurrences that need immediate intervention. The system detects
conditions within two minutes of the occurrence.
Alert coverage is hierarchical and can be defined for each alert created. Alerts can be configured
for any object that can be seen by the user creating the alert.
Best Practices
When possible, configure alerts to cover Client Computer Groups vice individual clients and
when using the E-mail output option, use group e-mail addressing vice individual users. Groups
are more consistent and wont require you to modify alerts when individual Clients or users are
removed or added.
Alerts can be sent to their intended recipients by email/pager; to the CommServes application
log accessible from the Windows System Event Viewer; by SNMP Trap(s) to a SNMP capable
monitoring application; or initiate a command line program, executable script, or batch file.
Alert wording can be customized by the user. Relevant, localized information can be added to
reports. Tokens are provided when variable data from the alert is required. Tokens can also be
passed as arguments for the command line script. Command line arguments support the use of
all tokens except attachments (e.g. <PROTECTED_OBJECTS>) or lists (e.g. <MEDIA_LIST>).
E-mail alerts can be sent to a CommCell User Group, External User Group, or to individual
users. In order for a user to receive an alert, the user must be enabled and have a valid SMTP
address. If the mail server is down, the system will attempt to send alert e-mail notification for
the next four hours. If after four hours, the mail server is still down, these alerts will be removed
from the system, and will not be sent to the recipient. Additionally, an individual alert
notification can be generated rather than multiple alerts being generated within a single alert
notification.
In the event that an alert condition persists, an escalation notice can be generated via alternate
means.
Recommended Alerts
Insufficient Storage
Phase or Network Failure
Recommended Alerts
Insufficient Storage
Media Management->Library Management
The most common reason for data protection job failure is no spare media. You run out of
magnetic disk space or have no spare media inside the library. This is particularly prevalent
during weekends when large data protection jobs are usually scheduled and no one is about to
monitor the status of spare media.
The Insufficient Storage alert is dependent upon the Low Watermark threshold value assigned to
the magnetic library, Scratch, and Cleaning Media Pools. Ensure you set these values high
enough to allow the alert to give you ample warning to replenish the media before jobs start to
fail.
<Device> Offline
Media Management->Device Status
Library, mount paths, or tape devices that are taken offline intentionally, or by themselves due to
some problem can have significant impact on your ability to complete jobs in a timely manner.
CommVaults CommCell Readiness Report checks for device status and should be run before
each data protection window. However, for more real time notification we recommend you set
an alert for offline status of key devices.
Reports
Understanding Reports
Recommended Reports
Understanding
Reports Simpana
Monitor
CommServe
Output Formats
HTML
CommServe
TEXT
Job XML
Folder/File
Template
Available Reports
A variety of reports can be created from the Metadata in the CommServe database. Pre packaged
reports or tailored reports to a particular aspect of data management can be created. Through
filter criteria, you can customize each report to include only the data that is required. Report
templates can be customized and are available to you to save, run, schedule, edit and view.
CommVaults Simpana Monitor is a hierarchical reporting and management application for one
or more CommCell groups. Simpana Monitor reports can include data from multiple
CommCells and have user selectable graphing capability.
CommCell Explorer
CommCell Explorer is a set of database views that can be used to create customized reports.
Recommended Reports
Compliance Report
Media Information Report
Recommended Reports
Module Summary
Key points to
remember
Summary
Many different reports are available with an abundance of options to configure. They
can be scheduled for email, or saved to a file among other options.
The CommCell Explorer views provide a way to query information on the CommCell
components directly from the SQL database.
The Job Controller window offers the availability to monitor and manage running
jobs.
The Event Viewer allows you to monitor all activities that are occurring within the
CommCell console.