WELLS FARGO, Inc.

MEMO
From: Chaitanya Eranki, Systems Architect, Consumer Lending Technology, Wells Fargo
To: Prashanth Shinde, Director, Consumer Lending Technology, Wells Fargo
Date: December 2, 2014
Subject: Software Test Automation Proposal
The ever-increasing complexity of financial software, combined with greater competition and
high costs of application downtime have triggered the need for testing to new heights. It
accounts for more than 50% of total application development costs (Stanford journal 2).
Software testing costs of Consumer Lending Technology division (CLT) at Wells Fargo, Inc., have
risen 120% over the last four years resulting in a 10% decrease in profits. The division is also
facing a challenge of completing projects in a lesser amount of time using minimal resources
because of increased competition and an overload of projects. In this context, software testing
within CLT needs to be made more efficient and cost-effective by improving processes and
frameworks currently in place.
Test automation is an investment that has produced substantial results for many software
development organizations. It improves efficiency and effectiveness of a testing process by
reducing the need of human involvement in repetitive or redundant tasks. Large scale software
firms such as Infosys and Microsoft have reported an overall reduction of 40% in the
development efforts after deploying automation solutions (Infosys 1). This memorandum
illustrates benefits of software test automation and discusses the implementation of an
automation methodology within CLT after performing suitability analysis.

Advantages of Test Automation in CLT

Shorter test cycles
A 6070% reduction in the test cycle time can be expected (Hewlett Packard 7) because of
Faster action triggering than in manual testing
Round the clock test runs

Higher Return on Investment (ROI)

Test Automation is cost-effective in comparison to manual testing. Lower costs translate into
higher profits. A Return on Investment (ROI) analysis is performed on one of the CLT
applications to illustrate this fact. Table 1 shows the ROI calculations performed for HPSI
application, a potential automation candidate. It can be observed that the returns are 202% for
every dollar invested in manual testing. Whereas, automation yields 320% for every dollar
invested. This clearly shows that test automation will lead to more profits in the long-run.

TABLE 1: ROI COMPARISON (IBM 7)

Lower defect resolution cost

Manual testing enables detection of only a few critical defects in the early stages of
development lifecycle. It costs more to fix defects that are found later in the lifecycle. Table 2
shows that it costs almost five times more to fix a coding defect once the system is released
than it does to fix it in unit testing. (U.S. National Institute of Standards and Technology, 2002).
This cost can be reduced by early detection and resolution of defects. An automated routine
can test significantly more cases in a day and find defects that manual testers might miss.

TABLE 2: COSTS TO FIX DEFECTS IN EACH STAGE OF DEVELOPMENT LIFECYCLE (NIST, 2002)

Wider scope of testing

With additional tools and effort, it is often possible to configure special runs of the automated
test cases in order to perform nonfunctional testing such as performance and load testing.

Greater employee morale

Automating manually intensive tasks will enable the test engineer to apply himself to more
challenging activities that will require him to think and attack the application in a strategically
considered way. This will result in greater employee enthusiasm and energy at work.

Test automation suitability for CLT applications

Testing activities in CLT can be
broadly classified into ten different
categories. A suitability analysis was
performed to understand if
automation can be applied to those
categories. Table 3 displays the
result of the analysis. It was
observed that eight out of the ten
categories can be automated. This
analysis demonstrates the potential
for automation in CLT.

TABLE 3: Suitability of analysis within CLT (ISPE 3)

Implementation of Automated Test Lifecycle Methodology (ATLM)

A systematic approach (Figure 2) needs to be followed in order to maximize test coverage and
returns. The decision to automate test suite represents the first phase of ATLM. The second
phase is related to test tool acquisition. Eighty three percent of CLT applications are web-based.
Tools currently available in the market for automation of web applications are:
HP QTP Concurrent licence cost of $30,000/year (includes maintenance)
Selenium (Open Source) No licence cost
HP QTP is the market leader (Figure 1) for web application
automation and comes with 24X7 support. Selenium is a
robust open source tool that comes without support. Even
though QTP comes at a cost, it clearly outweighs Selenium in
terms of features offered, and support provided. Smooth
operation is important for our division that is new to test
automation and HP QTP perfectly fits the bill.
Phases 3, 4, 5 and 6 are related to the project activities using
automation. Firstly, our test engineers have to be trained on
the new tool. HP QTP provides a comprehensive one-month
training at no extra cost. For CLT, the ATLM lifecycle is
estimated to take approximately four months to complete.
Management approvals and system setup can be finished in
FIGURE 1: Market Share (HP 6)
a months time. Planning and development activity will last
for a couple of months. Fourth month will be dedicated for test execution and closure activities.

Recommendation
Test automation provides much more than
defect count and metrics. It brings a unique
value to the project: insight and enlightenment.
Used in combination with an iterative approach
such as SDLC, it enables team members to focus
on effective testing throughout the project
rather than just at the end. It provides valuable
learning that can be applied to future projects.
From a management standpoint, automation
saves effort and cost resulting in higher profits.

1. Decision to
Automate Testing

6. Test Program
Review &
Assessment

Implementing this proposal will help CLT reduce

5. Execution and
its testing costs while building stronger
Management of
Tests
employee relations. It would be ideal to launch
a trial run in the first quarter of 2015,
4. Test Planning,
automating a couple of applications. If this
Design &
proposal is adopted, I have some additional
Development
thoughts about applying statistical algorithms to
automation frameworks. I look forward to
FIGURE 2: ATLM (ISPE 7)
discussing the details of this proposal (including
an implementation schedule) with you and answering any questions you may have.

2. Test Tool
Acquisition

3. Automated
Testing
Introduction
Process

References
Saswat Anand et al., 2013, An Orchestrated Survey on Automated Software Test Case
Generation, Antonia Bertolino, J. Jenny Li and Hong Zhu (Editor/Orchestrators),
Stanford Journal of Systems and Software x(y), xxCyy
Grossman, Paul. "Automated Testing ROI: Fact or Fiction?" Www.hp.com/go/quality NW
4AA2.4175E (2009): n. pag. Hewlett Packard. HP, Feb. 2009. Web.
Infosys. "Realizing Efficiency and Effectiveness in Software Testing Using a Comprehensive
Metrics Model." Test Efficiency & Effectiveness - Metrics (2012): n. pag. Infosys Ltd.,
2012. Web.
Munch, Stefan, Peter Brandstetter, Konstantin Clevermann, Ernst Reiner Schafer, and Oliver
Kieckhoefel. "The Return on Investment (ROI) of Test Automation." International
Society for Pharmaceutical Engineering ISPE 32.4 (2012): (n.d.): 1-8. Print.
Adams, Edward. "The Business Argument for Investing in Test Automation." The Rational Edge
(n.d.): 1-10. IBM, 2002. Web.