A CONCEPTUAL DISASTER & RECOVERY FRAMEWORK FOR XAVIER UNIVERSITY ATENEO DE CAGAYAN
DENIS RAMON MERCADO & IKE GAAMIL MINIMUM FINAL REQUIREMENT FOR MIT 204, MIT PROGRAM XAVIER UNIVERSITY (ATENEO DE CAGAYAN)
MARCH 17, 2012 2
TABLE OF CONTENTS
SECTION INTRODUCTION2 PROJECT MANAGEMENT APPROACH..2 PROJECT SCOPE3 MILESTONE LIST..3 SCHEDULE BASELINE AND WORK BREAKDOWN STRUCTURE...8 COMMUNICATIONS MANAGEMENT PLAN...10 SCHEDULE MANAGEMENT PLAN..15 DATA RECOVERY GUIDELINES..16 SPECIFIC CRISIS SITUATION FIRE DISASTER PREPARATION..26 NATURAL DISASTER..28
3
INTRODUCTION A disaster event can cause significant loss of the Universitys records and information technology systems and has the potential to cause major disturbance to the University's ability to operate effectively. This can result in financial loss, public embarrassment and a loss of credibility and goodwill. Xavier University Ateneo de Cagayan depends significantly on Information Technology Services as the campus service provider for computer-supported information processing, campus-wide networks, telecommunications, and technology support for University students, faculty, and staff. As a result of this ever-increasing reliance on technology, IT services require a comprehensive Disaster Recovery Plan to assure these services can be re-established quickly and completely in event of a disaster. The plan provides guidelines for ensuring that needed personnel and resources are available for both disaster preparation and response and that the proper steps will be carried out to permit the timely restoration of services. The purpose of the Disaster preparation and recovery plan is to provide for the safety and well-being of people on the premises at the time of a disaster and it will address the continuity of the critical business operations. Through this planning, it will minimize the duration of a serious disruption to business operations and resources. In return, it will minimize immediate damage and losses and identify critical lines of business and supporting functions that will ensure organizational stability and orderly recovery. II. PROJECT MANAGEMENT APPROACH Recently, there are major calamities that happened in the Philippines particularly in Cagayan de Oro city. Due to these unavoidable circumstances there will be possible major risk for the University in all aspects and it will affect its continuity in business. Therefore, there is a need for an updated disaster recovery plan. 4
Identification of the responsible department for the assign task will be vital for this plan. All Departments, offices, staff, faculty and all the sector of the university must have the knowledge when or where to go if the calamity will happen. This will organize the man power support for the university and will have efficient response operation. III. PROJECT SCOPE The scope of this disaster recovery plan for Xavier University addresses scenarios where the information systems and related technological infrastructure are physically damaged and/or require relocation due to fire and earthquakes.
This plan will only address the recovery of systems under the direct control of the Computing Services Department that are considered critical for business continuity. It has the overall Disaster Organization Chart but we are limited only to the Computer Science Recovery team. Also, given the uncertain impact of a given incident or disaster, it is not the intent of this document to provide specific recovery instructions for every system. Rather, this document will outline and a general planning of a recovery process which will lead to development of specific responses to any given incident or disaster.
IV. MILESTONE LIST Milestone Description Date Project study details A Discussion of the necessity of the study and its desired content Week 2 Project planning and formulation of a proposed disaster recovery system Detailed planning and formulation of the disaster recovery system Month 1 Project plan review Review of the important things to do in case of emergencies
Month 2 5
Project plan approval General information of the project management the team and this will be the basis of approval. Month 3 Project summary This includes points of contact and prime contractor information Month 4 Assessment of every department Assessment of information system 2 weeks Design of appropriate process flow Designed the process flow and the data center 2 weeks Implementation and installation of disaster recovery system Implementation or installation and testing of the disaster recovery system. Month 6
6
a. Project approval PROJECT MANAGEMENT PLAN Project name: Plan to Formulate IT and Disaster Recovery Date: February 18, 2012 Plan Release #: 1.0 Project Manager: Ike Gaamil
Ike Gaamil Dennis Mercado Project Manager Lead Engineer
Fr. Roberto C. Yap SJ Dr. Lina G. Kwong University President Academic Vice-President
7
b. Project Summary
PROJECT SUMMARY
Comments: _____________________
Please answer the following questions by marking Yes or No and provide a brief response as appropriate: Is this an updated Project plan? If so, reason for update: ____________________________________________ Budget for project by fiscal year and is project funded? If so, for what amount(s) and periods(s): Budget Amount: 200,000 php Year: FY 2012 Funded? __X__ Yes ______ No Budget Amount: 500,000 php Year: FY 2013 Funded? __X__ Yes ______ No Budget Amount: Year: Funded? _____ Yes ______ No Total Amount: 700,000 php
Disaster Preparation and Recovery Plan Project Name Xavier University Organization Feb. 20, 2012 Start Date: Ike Gaamil and Dennis Mercado Submitted By: Avinu corporation Prime Contractor February 25, 2012 Date Awarded Development Life Cycle (Design, Development, Integration, Testing or Implementation Current Stage of project X Yes _____No Project is on Schedule: Project is within the budget X Yes _____No 8
Points of contact: Position Name/Organization Phone E-Mail Project Manager Ike Gaamil 0905-5685-638 ikegaamil@xu.com Lead engineer Dennis Mercado 0926-9452-161 dmercs@xu.com Senior Technical Sponsor Mario Feliciano 0925-2411-234 mariofel@xu.com Procurement Contact Joseph Sabal 0252-4252-231 joseph@xu.com
Prime Contractor Information: Company: Avinu Corporation Position Name/Organization Phone E-Mail Project Manager John Smith 0927-2412-738 johnsmith@avinu.com Senior Management Sponsor Daniel Diaz 0928-9242-141 dandiaz@avinu.com Senior Technical Sponsor Rusty Lim 0927-2611-341 Rustylim@avinu.com
9
Schedule Baseline and Work Breakdown Structure: Schedule Baseline Level WBS Code Element Name Description of Work Deliverables Budget Resources 1 101 Project Initiation Obtaining support and organize committees
2 102 Assessment/ Risk identification Identification of risk hazard
3 103 Mitigation Protection and management of system
2,000,00 0.00 php
4 104 Business continuation Planning of the Continuation of business function
5 105 Crisis management Plan of Management during Crisis
13,000,0 00.00 php
6. 106 Emergency Response Emergency management and response during the crisis
7 107 Crisis Management Implementation of the management during the crisis
8 108 Business Continuation Continuation of business after the crisis
LEGEND: FS = The specific task must finish prior to starting the identified task SS = Two identified task start at the same time, but are not linked to finish at the same time. FF = Two identified task finish at the same time, but are not linked to start at the same time. Blank = Task has no dependency Lag = Additional days can be added for reserve to ensure project stays on schedule 10
Work Breakdown Structure Pre-Incident Project Initiation Assessment/Risk Identification Risk Quantification Obtain management Support Organize Planning Commitees Natural Hazards Fire, Explosion Develop loss scenarios Possible estimation of impact on buildings, operations, personal and environment Identify and prioritize critical functions Identify Resource Requirements Mitigation Protection systems Hazard Elimination Develop Recovery Strategies Assess alternative operating strategies Plan Developement Business Continuation Crisis Management Emergency Response Documentation Implementation and training Testing Maintenance and Updating Changes in personnel, Facilities, operations, and hazards Regulatory Requirements Results of Plan Testing Incident Resonse Emergency Response Crisis Management Business Continuation Evacuation Fire fighting Management Decision Making Internal Communications Executive alternative operating Strategies Restore Critical Function Effect long-term Recovery Hazardous Materials Rescue Medical Security Property Conservation External public relations 11
V. Communication Management Plan Communication Plan Xavier University Communication Plan is designed to provide an orderly flow of accurate, effective and timely information to the Xavier staff and campus during the onset of a crisis situation, or a situation of potential crisis affecting the University campus telephone, data network and, computer and information systems. It is the responsibility of each department to communicate with their customers and other Xavier staff. Coordinating with Campus Services Help Desk and other key entry points will provide the communication link in communicating service interruptions. Communication Guidelines The focus of this section is to decide in advance how every department will communicate with internal and external audiences in the event of an unplanned service interruption. This plan recognizes the importance of addressing and supporting communication needs and issues that emerge at the service level. Individual departments will need to extend this plan for the specific requirements of their area. Enterprise IT Emergency Communications During a campus IT emergency, defined as a serious situation not (or perhaps not yet) having been declared a disaster, the IT Security Officer has primary responsibility for immediate response. All emergency IT messages will be sent to the college and departmental emergency contact list by the University IT Security Officer. a. Develop a Plan of Action. Determine how Xavier University Information Technology Services will respond to any service interruption by defining the specific actions to be taken, outlining the way that appropriate information should flow to different audiences, and identifying appropriate spokespersons for various constituents. Particular attention should be paid to determine a priority order under which audiences will receive information, as well as a regular schedule of news updates. Each department will work with University Relations to gather accurate and substantial information regarding the situation and details regarding the University response. University 12
Relations, working with the department, will provide notification to customers, employees, and the general public on progress toward recovery. Audiences/contacts that should be considered during a crisis: Chief Information Officer/Xavier University Information technology services Staff Campus Services Information technology Services Help Desk Customers College/Unit IT Emergency Contacts University Administration University Relations and the Public Public Safety Facilities Management Business Office University General Counsel b. Plan Enactment. Notices should be issued in a timely manner, before the story and speculation starts leaking out on its own. It is the organizations policy to be open and honest in communication no matter where the blame lies. Provide factual information to University Relations and authorities as quickly as facts have been verified, and use every means of communications available to offset rumors and misstatements. c. Follow Up. After the plan is activated, the Disaster Recovery Manager will determine subsequent actions and decide if other employees need to be involved. The following information must be gathered and its accuracy verified to provide an incident report to the Director. What has happened? Who is involved? When did it happen? 13
Where has it happened? Was anyone injured? Could interruption have been prevented? Financial Loss? What impact may this crisis have on the organization: Does this situation run the risk of escalating in intensity? To what extent will the situation be noticed by the media and/or monitored by governmental agencies? Will the situation interfere with normal site or business operations? Could this situation damage the organizations reputation? To what extent could this situation directly impact the organizations financial standing? Xavier University Communication Guidelines University Relations serve as the authorized spokespersons for the Institution. All public information must be coordinated and disseminated by their staff. University policy requires that only certain administrators may speak on behalf of the University. In the event that regular telecommunications on campus are not available, University Relations will center media relations at a designated location. Information will be available there for the news media and, as possible, for faculty, staff, and students. Cellular and other emergency telephone numbers are available to Public Safety and other designated units. Official information will be made available as quickly as possible to the Campus Information Center. After hours a University Relations representative is on call evenings, weekends, and holidays to assist University units in communication with the campus and the general public dealing with media emergencies and other unusual circumstances. The representative on call will provide media assistance and alert appropriate University administrators as necessary. 14
Information technology Service and Escalation Protocol Disaster Recovery Director - Fr. Roberto C. Yap SJ Disaster Recovery Managers Lina G. Kwong Academic Vice-President ESTRELLA C. CABUDOY Director, University Library REYNALDO ANTONIO R. MANTE Director, Human Resources Office VERNA A. LAGO University Registrar ENGR. LENNIE K. ONG University Treasurer ENGR. GERARDO S. DOROJA Dean, College of Computer Studies HARRIET FERNANDEZ Head, CISO Crisis Decision Academic Office Recovery Team Team Leader Back up Coordinator Library Recovery Team Team Leader Back up Coordinator Human Resouce Recovery Team Team Leader Back up Coordinator Registrar Recovery Team Team Leader Back up Coordinator Finance Recovery Team Team Leader Back up Coordinator CS Recovery Team Team Leader Back up Coordinator CISO Recovery Team Team Leader Back up Coordinator Entry Points 214 -HELP(4357) ITS Help Desk 232-2421 Administrative Help Desk 215-2515 Telephone Repair and Voice Mail Help Desk 253-3500 Campus Operator 236 - Campus Information Desk Customer Faculty/Staff Students/Parents Xavier University Hospitals or Clinic Reaction Response
15
Computer Science Recovery Team and Staffing
CS Recovery Team Team Leader: Engr. Gerardo S. Doroja Desktop Recovery Team Mr. Francis Lee Mondia Data Center Recovery Leader Mr. Joseph Anthony C. Sabal Telecommunication Recovery team Ms. Harriet Fernandez Network and Web Recovery Leader Mr. Fren Marlon Peralta Computer Operator and User Support Agent: Cristina Amor Cajilla Rozaldy Gutierrez Maria Ramila Jimenez Rhea Suzette Mocorro Elvira Yaneza Meldie Apag Sheryl May Jagonia Shiela Dimasuhid Florence Reyes Software Recovery Leader Paulo Javier Gener 16
Schedule Management Plan
17
DATA CENTER Recovery Team:
The Data Center Recovery Team is composed of personnel within Information Technology Services that support Xavier University central computing environment and the primary data center where all central IT services, the Networks Operations Center and other central computing resources are located. The primary function of this small working group is the restoration of the existing data center or the activation of the secondary data center depending on the severity of the disaster. This teams role is to restore the data center to a condition where individual recovery teams can accomplish their responsibilities with regard to server installation and application restoration.
The team should be mobilized only in the event that a disaster occurs which impacts the ability of the existing central computing facility to support the servers and applications running there. The University President has the responsibility to keep the IT Director up to date regarding the nature of the disaster and the steps being taken to address the situation. The coordination of this recovery effort will normally be accomplished prior to most other recovery efforts on campus as having a central computing facility is a prerequisite for the recovery of most applications and IT services to the campus.
DESKTOP Recovery Team:
The Desktop Recovery Team is composed of personnel within the Information Technology Department that support Xavier Universitys desktop hardware, client applications, classrooms, labs and academic development systems. The primary function of this small working group is the restoration of Xavier Universitys desktop systems, classrooms and labs to usable condition. During the initial recovery effort, the team is not responsible for restoration of any data the user may have on their desktop computer. Central Washington University recommends all users store data files on the file servers, which are backed up nightly, to support data recovery. 18
The team should be mobilized in the event that any component of the network or telecommunication infrastructure experiences a significant interruption in service that has resulted from unexpected/unforeseen circumstances and requires recovery efforts in excess of what is experienced on a normal day-to-day basis. The University President has the responsibility to keep the IT Incident Director up to date regarding the nature of the disaster and the steps being taken to address the situation. The coordination of this recovery effort will be accomplished with other recovery efforts on campus by the IT Incident Director.
NETWORKS AND WEB Recovery Team:
The Networks and Web Recovery Team is composed of personnel within Information Technology Services that support Xavier Universitys network infrastructure including all cable plants, switches, routers, network applications, file servers, electronic email servers and web services. The primary function of this small working group is the restoration of Xavier Universitys LAN and servers to the most recent pre-disaster configuration in cases where data and network loss is significant. In less severe circumstances, the team is responsible for restoring the system to an operational status as necessitated by any network hardware failures or other circumstances that could result in diminished performance.
The team should be mobilized in the event that any component of the network infrastructure experiences a significant interruption in service that has resulted from unexpected/unforeseen circumstances and requires recovery efforts in excess of what is experienced on a normal day-to-day basis. The University President has the responsibility to keep the IT Incident Director up to date regarding the nature of the disaster and the steps being taken to address the situation. The coordination of this recovery effort will be accomplished with other recovery efforts on campus by the IT Incident Director.
19
SOFTWARE Recovery Team: The Software Recovery Team is composed of the IT Specialists within Information Technology Services that support the ERP and Software system as well as the User Application Specialists and a Network Specialist. The primary function of this small working group is the restoration of all modules of the Application Software to the most recent pre-disaster configuration in cases where data loss is significant. In less severe circumstances the team is responsible for restoring the system to an operational status as necessitated by any hardware failures, network outages or other circumstances that could result in diminished system performance.
The team should be mobilized in the event that the Application Software and ERP systems experience a significant interruption in service that has resulted from unexpected/unforeseen circumstances and requires recovery efforts in excess of what is experienced on a normal day-to-day basis. The University President has the responsibility to keep the IT Incident Director up to date regarding the nature of the disaster and the steps being taken to address the situation. The coordination of the PeopleSoft recovery effort will be accomplished with other recovery efforts on campus by the IT Incident Director.
TELECOMMUNICATIONS Recovery Team: The Telecommunications Recovery Team is composed of personnel within the Information Technology Department that support Xavier Universitys voice networks. The primary function of this small working group is the restoration of Xavier Universitys voice networks to the most recent pre-disaster configuration in cases where voice network loss is significant. In less severe circumstances, the team is responsible for restoring the voice network to an operational status as necessitated by any failures or other circumstances that could result in diminished performance.
The team should be mobilized in the event that any component of the network infrastructure experiences a significant interruption in service that has resulted from unexpected/unforeseen circumstances and requires recovery efforts in excess of what 20
is experienced on a normal day-to-day basis. The Head of Telecommunications has the responsibility to keep the IT Incident Director up to date regarding the nature of the disaster and the steps being taken to address the situation. The coordination of this recovery effort will be accomplished with other recovery efforts on campus by the IT Incident Director.
DISASTER PREPAREDNESS A critical requirement for disaster recovery is ensuring that all necessary information is available to assure that hardware, software, and data can be returned to a state as close to pre-disaster as possible. Specifically, this section addresses the backup and storage policies as well as documentation related to hardware configurations, applications, operating systems, support packages, and operating procedures.
Data Recovery Information: Backup/Recovery disks and tapes are required to return systems to a state where they contain the information and data that was resident on the system shortly prior to the disaster. At Xavier University full backups of all servers are performed weekly. Those servers not in the full backup list have an incremental done. Backup/Recovery tapes are stored in the locations and for the retention periods outlined summarized in the table below: Daily Period Storage Location Authorized Personnel Weekly Backup Xavier University Data Center Network and Operation Personel
Central Data Center and Server Recovery Information: In the event of any disaster which disrupts the operations in the Data Center, reestablishing the Data Center will be the highest priority and a prerequisite for any IT recovery. As such, Information Technology Services is required to have detailed information and records on the configuration of the Data Center and all servers and equipment located in the Data Center. Detailed information is documented in the 21
database and this database is updated and copied monthly to CD and stored in a vault with the backup tapes. The operations staff is responsible for keeping the hardware inventory up to date.
Network & Telecommunication Recovery Information: In the event of any disaster which disrupts the network and/or telecommunications, reestablishing the connectivity and telephony will be a high priority and a prerequisite for any IT recovery. Recovery of these services will be accomplished in parallel or immediately following recovery of the Data Center. As such, Information Technology Services is required to have detailed information and records on the configuration of the networking equipment. Detailed information of switches and routers is documented in the database and this database is updated and copied monthly to CD and stored in the vault with the backup tapes. The networking staff is responsible for keeping the networking inventory up to date.
Application Recovery Information: Information necessary for the recovery and proper configuration of all application software located on the central servers is critical to assure that applications are recovered in the identical configuration as they existed prior to the disaster. Detailed information on critical central applications will be documented in the database and this database is updated and copied monthly to CD and stored in the vault with the backup tapes. Server administrators are responsible for keeping the application inventory up to date.
Desktop Equipment Recovery Information: Information necessary for the recovery and proper configuration of all desktop computers and printers supported by Computer Support Services is critical to assure that client systems can be restored to a configuration equivalent to pre-disaster status. Detailed information on client systems (both PC and MAC) is also documented. This web site is backed up nightly.
22
DISASTER RECOVERY PROCESSES AND PROCEDURES Incident Command Team: The role of the IT Disaster Recovery Team (under the direction of the Incident Director) is to coordinate activities from initial notification to recovery completion. Primary initial activities of the team are:
Incident Occurrence: Upon the occurrence of an incident affecting the IT services at Xavier University, the Head & Assistant Head of Information Technology will be notified by campus security and/or other individuals. Personnel reporting the incident will provide a high-level assessment as to the size and extent of the damage. Based on this information, the assistant Head of IT will assume his/her responsibilities as the Incident Director, and will contact the other members of the Incident Command Team, and provide them with the following basic information:
Brief overview of the incident, buildings affected, etc. Which Incident Command Headquarters (ICH) will be used Scheduled time to meet at the ICH for initial briefing Any additional information beneficial at this point. No other staff members are to be contacted at this point, unless directed by the Incident Director.
Incident Assessment: The IT Disaster Recovery Team will receive an initial briefing from the Incident Director (ID) and any other personnel invited to the meeting. The Disaster Recovery Team will assess the situation, perform a walk-through of affected areas as allowed, and make a joint determination as to the extent of the damage and required recovery effort. Based on this assessment, the team will make a determination as to whether the situation can be classified as routine and handled expeditiously via normal processes, or if a formal IT disaster needs to be declared.
Once an IT disaster has been declared, and the preceding steps to notify the XU Management Team and the Recovery Teams have been accomplished, ongoing responsibilities of the Incident Command Team and Director include:
23
Securing all IT facilities involved in the incident to prevent personnel injury and minimize additional hardware/software damage. Supervise, coordinate, communicate, and prioritize all recovery activities with all other internal / external agencies. Oversee the consolidated IT Disaster Recovery plan and monitor execution. Hold regular Disaster Recovery Team meetings/briefings with team leads and designees. Appointing and replacing members of the individual recovery teams who are absent, disabled, ill or otherwise unable to participate in the process. Provide regular updates to the Xavier University (XU) Management Team on the status of the recovery effort. Only the XU Management Team and/or their designees will provide updates to other campus and external agencies (media, etc.) Approve and acquire recovery resources identified by individual recovery teams. Interface with other activities and authorities directly involved in the Disaster Recovery (Police, Fire, Department of Public Works, XU Teams, etc.) Identify and acquire additional resources necessary to support the overall Disaster Recovery effort. These can include 1) acquiring backup generators and utilities, 2) arranging for food/refreshments for recovery teams, etc. Make final determination and assessment as to recovery status, and determine when IT services can resume at a sufficient level.
Disaster Recovery Teams: Disaster Recovery Teams are organized to respond to disasters of various type, size, and location. Any or all of these teams may be mobilized depending on the parameters of the disaster. It is the responsibility of the Incident Command Team to determine which Disaster Recover Teams to mobilize, following the declaration of a disaster and notification of the Xavier University Management Team. Each team will utilize their respective procedures, disaster recovery information, technical expertise, and recovery tools to expeditiously and accurately return their systems to operational status. While recovery by multiple teams may be able to occur in 24
parallel, the Data Center and Network/Telecommunications infrastructure will normally be assigned the highest priority, as full operational recovery of most other systems can not occur until these areas are operational.
Database Recovery Team: 1. Take appropriate steps to safeguard personnel and minimize damage to any related equipment and/or software. 2. Assess damage and make recommendations for recovery to Database services. 3. Identify other individuals required to assist in recovery of these applications, and report this information to the ID for action. 4. Restore degraded system function at backup site and inform user community of the restrictions on usage and/or availability. 5. Coordinate software replacement with vendor as required. 6. Coordinate Database services recovery with other recovery efforts. 7. Execute plan to restore Database services to full function. 8. Provide scheduled recovery status updates to the Incident Director to ensure full understanding of the situation and the recovery effort. 9. Verify and certify restoration of the Database services to pre-disaster functionality.
Data Center Recovery Team: 1. Take appropriate steps to safeguard personnel and minimize damage to any related equipment and/or software. 2. Assess damage and make recommendations for recovery of Central Data Facility. Determine if use of alternate/cold site is required. 3. If the alternate data center site is required, execute all necessary steps to notify appropriate personnel and secure backup facility. 4. Identify other individuals required to assist in recovery of data center, and report this information to the ID for action. 5. Develop overall recovery plan and schedule, focusing on highest priority servers for specific applications first. 6. Coordinate hardware and software replacements with vendors. 25
7. Recall backup/recovery tapes from on campus or off-campus storage, as required to return damaged systems to full performance. 8. Oversee recovery of data center based on established priorities. 9. Coordinate data center recovery with other recovery efforts on campus. 10. Provide scheduled recovery status updates to the Incident Director to ensure full understanding of the situation and the recovery effort. 11. Verify and certify restoration of the data center to pre-disaster functionality.
Desktop Recovery Team: 1. Take appropriate steps to safeguard personnel and minimize damage to any related equipment and/or software. 2. Assess damage at all areas affected, and make recommendations for recovery. 3. Identify other individuals required to assist in recovery of desktop services, and report this information to the ID for action. 4. Develop overall recovery plan and schedule, focusing on highest priority areas of the campus infrastructure/desktop services first 5. Coordinate hardware and software replacement with vendors. ( 6. Oversee recovery of desktop computing services (workstations, printers, etc.) based on established priorities. 7. Coordinate recovery with other recovery efforts on campus. 8. Provide scheduled recovery status updates to the Incident Director to ensure full understanding of the situation and the recovery effort. 9. Verify and certify restoration of the desktops to pre-disaster functionality.
WEB Recovery/Network and Telecommunications Recovery Team: 1. Take appropriate steps to safeguard personnel and minimize damage to any related equipment and/or software. 2. Assess damage and make recommendations for recovery. 3. Identify other individuals required to assist in recovery of services, and report this information to the ID for action. 26
4. Develop overall recovery plan and schedule, focusing on highest priority areas of the campus infrastructure first. 5. Coordinate hardware and software replacement with vendors. 6. Oversee recovery of messaging, telecommunications and network services based on established priorities. 7. Coordinate messaging, network and telecommunications recovery with other recovery efforts on campus. 8. Provide scheduled recovery status updates to the Incident Director to ensure full understanding of the situation and the recovery effort. 9. Verify and certify restoration of the Messaging, Network and Telecommunications infrastructure to pre-disaster functionality.
ERP/Application Software Recovery Team: 1. Take appropriate steps to safeguard personnel and minimize damage to any related equipment and/or software. 2. Assess damage and make recommendations for recovery to ERP/Application Software services. 3. Identify other individuals required to assist in recovery of these applications, and report this information to the ID for action. 4. Restore degraded system function at backup site and informs user community of the restrictions on usage and/or availability. 5. Coordinate software replacement with vendor as required. 6. Coordinate PeopleSoft services recovery with other recovery efforts. 7. Execute plan to restore ERP/Application Software services to full function. 8. Provide scheduled recovery status updates to the Incident Director to ensure full understanding of the situation and the recovery effort. 9. Verify and certify restoration of the PeopleSoft services to pre-disaster functionality.
Telecommunications Recovery Team: 1. Take appropriate steps to safeguard personnel and minimize damage to any related equipment and/or software. 27
2. Assess damage and make recommendations for recovery. 3. Identify other individuals required to assist in recovery of these services, and report this information to the ID for action. 4. Develop overall recovery plan and schedule, focusing on highest priority areas of the campus infrastructure first. 5. Coordinate hardware/software replacement with vendor as required. 6. Oversee recovery of voice network services based on established priorities. 7. Coordinate the voice network recovery with other recovery efforts. 8. Provide scheduled recovery status updates to the Incident Director to ensure full understanding of the situation and the recovery effort. 9. Verify and certify restoration of the voice network to pre-disaster functionality.
Specific Crisis Situation Fire Disaster Preparation a. Structural fire separation IT structure shall be separated from adjacent areas by fire-resistant walls (of 90 minutes minimum resistance) and non-flammable materials. Separating walls within IT facilities shall offer at least 30 minutes of fire resistance and be made of non- flammable materials. They shall reach form the raw floor to the raw ceiling. IT equipment should possibly be distributed to several rooms.
IT areas shall not be located in the same fire zone as other high hazards. High hazard shall be separated by fire break walls or complex separation walls with increased stability requirements in the case of fire and made of non-flammable materials. b. Interior fittings The interior fitting should be made of non-flammable materials; if this is not possible, at least materials of low flammability should be used as well as materials that do not drip while burning. As few halogen-containing plastics should be used as possible, both in IT facilities and in adjacent areas. c. Fire detection and fire alarm systems (FDAS) 28
IT facilities including adjacent rooms shall be monitored by automatic fire detection and fire alarm system. Apart from the IT facility area itself these could include:
- Plant rooms of air conditioning or venting systems; - Ventilation ducts from air condition systems, including fresh air sampling pipes; - Rooms of power supply and emergency power supply; - Data archive rooms; - Paper storage rooms; - Rooms beside or above/below the IT facility d. Fire extinguishing systems For the entire protection of IT equipment, automatic fixed fire extinguishing systems can be recommended. Both gas extinguishing systems and water extinguishing systems are appropriate for IT equipment, as well as adjacent areas, such as storage areas, archives and offices.
For application in IT areas, extinguishants with as little residue as possible are preferable. It must be a non-corrosive and non-electrically conductive extinguisher. The following extinguishing systems meet these requirements. - Carbon dioxide, (C02) fire extinguishing system - Inert gas extinguishing systems; - System with chemical extinguishants e. Fire Extinguishers In the actual IT rooms and in the adjacent rooms, a sufficient number of appropriate fire extinguishers shall be available f. Organizational fire protection A fire protection concept showing the necessary organizational measurements shall be established and specified for the IT facility. The following items shall be considered: - Reduce fire load to the minimum; - Set up regulations and installation instructions; - Strictly do not allow fire-hazardous works; if necessary, a permit is required and fire protection regulation shall be followed; - Instruct and supervise contractors; - Keep and control cleanliness and tidiness permanently; 29
- Eliminate ignition sources; - Smoking ban; if necessary, provide separate fire protected smoking zones; - Ban private electrical appliances All necessary fire protection measures shall be agreed by the insurer, with the internal representative for safety and health and fire protection and with the responsible public fire brigade. Fire brigade plan, showing the facilities for the fire brigade and equipment in and around the building shall be handed to the competent fire brigade.
g. Further loss prevention measures During a fire, damage to the respective IT equipment can generally be minimized by well-timed manual disconnection of the voltage supply. Manual emergency abort devices shall be protected against accidental operation and abuse. If IT equipment rooms are air-conditioned, they shall have their own air- conditioning systems.
The equipment itself must be marked or identified in a categorical type through color or any form of identification for the purpose prioritizing the systems in case of some fire or natural hazard events.
Other Severe Weather Upon notification of a warning: Remain calm and avoid panic. Go to an area of safety. (Rooms and corridors, in the innermost part of a building. Stay clear of windows, corridors with windows or large free-standing expanses. Do not use elevators. Close all doors, including main corridors, making sure they latch. Crouch near the floor or under heavy, well-supported objects and cover your head. Be alert for fire. In the event of a fire, the fire plan should be utilized. Listen to your radio for news and updates if possible.
30
Flood and/or Water Damage Assess your own safety and act accordingly. Do not walk or work in standing water which may have contact with wiring and may be electrified. Call Facilities Management and your Building Coordinator and develop a plan to coordinate with Facilities Management. (After hours: if you are unable to reach Facilities Management or the building coordinator, notify Department of Public Safety. Work with Facilities Management to: turn off water supply if water is flowing from pipes, and provide equipment and personnel to clean up water. If the water emergency involves a threat or damage to Information Technology Services or facilities, make the calls indicated below until you reach someone. The first person you reach in the notification list below will begin implementation of the salvage procedures.