Maintenance and Production Support

The maintenance and production support (MOPS) team consists of three distinct teams: CR Handle the life cycle of change requests on existing modules Production Support Rework code in order to close post production bugs Internal Support Manage/Monitor/troubleshoot/Report on the day to day transactions of the OnlineSBI portal including the seven associate banks System Admin: Manage/Monitor/Troubleshoot the complete infrastructure of the OnlineSBI portal. Handle interactions with third party vendors who supply/configure and maintain specific components in the infrastructure. This document aims to highlight the increasingly complex role performed by the MOPS team.

Insights on Maintenance Activities

The portal has grown from a small application providing Internet banking facility six years ago to a huge network of interlinked applications, with a mammoth infrastructure spanning over 4 geographic locations. The portal caters to the banking needs of diverse audience starting from retail customers, to large corporates, merchants, suppliers, Government agencies and financial institutions. SBI has an all encompassing bird eye view of the complete infrastructure- but due to the nature of their corporate hierarchy, sometimes seem unaware of the nity gritties of maintaining such a huge portal. The availability of cutting edge technology and tools to manage such complex infrastructure has encouraged the client to advice and query on these technologies. Satyam has adopted steep learning curves in order to support this infrastructure. In retrospective of the vast technology changes that the portal has undergone in the past 5-6 years, the team has been through a continuous learning process to upgrade their technical skills and provide the highest level of support and consultancy services. Satyam takes initiatives to provide high end security features such as encryption, SFTP facility to corporates to extend the goodwill of SBI to high profile corporates. Such initiatives are not taken for any monetary gain or recognition from client. However this definitely ensures that the team is trained and exposed to new and exciting technology. Satyam needs to customize the corporate banking solution to the specific needs of these agencies based on these interactions. SBI may sometimes play a passive role in such interactions but is strictly kept in the loop as far as mails to such agencies are concerned Requirements for customizations and change requests are not elaborated by SBI. The team drafts an understanding document and sends it across for approval. Based on this understanding the CR is developed and hosted for

acceptance. Ad hoc requirement changes are also pitched in during the iteration, and may often need to be incorporated in the same CR. Interactions with Merchants, Government agencies, corporates are required to understand their IT systems. Satyam often needs to initiate such interaction on their own, in order to understand the full scope of requirements. Complex and unique customizations are often done for each merchant, supplier, vendor or corporate, based on the business activities of the entity. This often is treated as routine maintenance jobs and often goes unaccounted for. Interactions with third party sites cannot be tested due to constraints in staging environment. It has been advised to client to provision for a pre production setup to handle such issues. Satyam is often made to interact with external vendors and entities such as IRCTC, CRIS, GE etc. to fulfill their transactions or generate reports on their transactions. This often leads to phone calls directly from these vendors both during office hours and at odd hours/holidays etc. Satyam has been very responsive to such calls.

Sysadmin Activities at Offshore:

Team consists of OS admin, App Admin, SCM, PL/SQL, E-security specialist and maintains Development, Staging, Production and DR environment

OS support:
Maintenance includes all OS activities installation & configuration, Patch updation, trouble shooting. Installation of Solaris OS in all the new servers in Development and DR environment. Updation of Solaris patches in all the servers in the Development and DR environment. As far as Staging and Production is concerned, the onsite OS admin takes care. Trouble shooting of OS is also done in the development and DR environment. All activities with respect to the application functioning pertaining to the OS is done by the offshore-sysadmin in all the four environments.

Application support :

Installation & configuration of IBM Websphere Application server, IBM HTTPServer , maintenance, trouble shooting and patch updation in all the above four environments. Offshore sysadmin plays a major role in this activity. All the application related issues in all environments are handled by the team. The offshore team facilitates the development/testing/defect teams to have smooth rollover of the new codes into production. All infrastructures related issues with respect to the application are handled by the offshore team. Coordination with the development/CR/Defect team is done by the offshoresysadmin team to ensure smooth functioning of the application in all environments. Interactions with the respective team leads are done for any application related troubleshooting in all the environments. All the new modules, CRs done by the development team is moved to UAT and production by the offshore-sysadmin team. Load balancing device - Netscaler which is used in the setup is managed by the Application admin of the offshore-sysadmin team. New configuration of sites and SSL certificates management is done by the team. Maintenance of all sites in the web servers are maintained by the offshoresysadmin team.

DBA: DBA is actively involved in the performance tuning of the application. Database designing is done by the DBA and all the complex application queries are tuned and updated to the CR team for their Production rollover. Online checking of performance of queries is also done by the DBA. All triggers, procedures are written by the DBA.

Security:

Esecurity checks the site and monitor the logs for any vulnerabilities. Maintains a database of the phishing IPs and risk prone sites. Providing the necessary security methods for all new applications. Constantly interacts with the client regarding security issues and preventions. Updating the latest technologies on E-security and updating the client

SCM:
The SCM maintains the version control of the entire source code. All production source code are updated by the SCM. Periodical backup of source code is taken and sent to the client. Interaction with Various Vendors: The offshore-sysadmin team interacts with various vendors for troubleshooting. OS admin interacts with Sun & Wipro for OS related support Coordination is done with Sify and Wipro for all DR related datacenter support. All trouble shooting is done with the help of Wipro for all OS issues. OS admin physically goes to the datacenter and takes up with the help of the vendor. App admin interacts with the IBM support team for various application related issues. Tickets are rasied with IBM and recommendations from IBM is carried out in UAT and then in Production environments. Networking vendors Sify and Datacraft are called for any network or DNS related issues which affects the activity of the sites. DNS management done by Sify is contacted for all DR drills.

Day to Day Activities of Offshore-sysadmin team: Deployment of applications in UAT and Production environment Configuration management of the source codes given by the development/cr/defect teams Monitoring of all OS, Applications, DB queries for all environments Automation of admin activities is done by the team Troubleshooting of all day to day issues pertaining to application, OS and web servers Checking of all logs and log rotation is done by the team.