www.availabilitydigest.

com

Faster Payments – Bringing Payment Processing Into the 21st Century

June 2010
Damian Ward, VocaLink

Abstract
Faster Payments is the first new payments service to be introduced in the UK for
more than twenty years. For the very first time, it has enabled phone, Internet and
standing-order payments to move in near real time - almost at the touch of a button.

On 27 May, 2008, VocaLink implemented the hub of the Faster Payments Service
(FPS) in the UK. The Faster Payments Service is revolutionary and is the first application of the VocaLink
Real-Time Payments Platform.

The Faster Payments Service operates on a 24x7 basis, providing near real-time interbank transfers for
Internet and telephone-banking payments. The delivery of this unique service has changed the dynamics of
consumer banking behaviour and the payments industry.

This paper presents a brief overview of the development, implementation and operation of the UK Faster
Payments Service and why it is able to operate at 100% service availability thorough the use of active/active
technology.

© 2010 Sombers Associates, Inc., and W. H. Highleyman / Damian Ward (VocaLink)

www.availabilitydigest.com
For discussion, contact editor@availabilitydigest.com
1
Award Winning
VocaLink triumphed at the Tenth Anniversary Financial Sector Technology (FST) Awards, winning the ‘best
of the best’ trophy for the Faster Payments Service.

In 2010, the FST Awards celebrated their tenth anniversary; and to honour the
occasion they broke open the vaults and looked back over previous winning entries
to select what they considered to be the ‘best of the best’ out of all the projects
recognized over the last decade.

The judges were looking for noteworthy, revolutionary projects that have changed
the industry. At the awards ceremony held at the London Lancaster Hotel, over 500
attendees from across the industry saw VocaLink’s Chief Operating Officer, Ian
Gausden, collect the winning trophy on behalf of VocaLink, chosen as the overall
winner for the Faster Payments Service.

About VocaLink
VocaLink is a specialist provider of transaction services to banks, their corporate customers, and government
departments. On a peak day, the VocaLink automated payment platform processes over 90 million
transactions. Annually, over 9 billion transactions are processed:

 Its switching platform connects the world’s busiest ATM network of over 63,000 ATMs.
 Its Real-Time Payments platform provides the central infrastructure for the UK Faster Payments
Service.

VocaLink are also working with BGC (Bankgirocentralen) to process most of Sweden’s domestic payments.

With Voca’s heritage of bulk payment processing and LINK’s understanding of real-time transaction
switching, both companies were already recognised as leaders in their respective fields. As pioneers of
electronic transactions, the companies saw each other as natural partners to deliver this vital service to the
payments industry. A strong working partnership was forged. On 2 July, 2007, Voca and LINK merged; and
VocaLink was born.

VocaLink has a proven history of successfully delivering new and innovative services in the highly
competitive payments arena.

Faster Payments Background

The UK Faster Payments initiative
has its origins in the Cruickshank
Report, which was published in 2000.
The report highlighted the need for a
low-cost way to transfer money
quickly.

In May, 2005, the Payment Systems

Task Force, chaired by the UK Office
of Fair Trading, announced that an
agreement had been reached with
the UK banking industry to reduce
clearing times on phone, Internet and
standing-order payments. The UK
banking industry committed to
develop a system whereby payments
would clear in half a day.

Voca and Link made the conscious decision to look at the concept of faster payments imaginatively. Rather
than settle for incremental improvements in clearing times, they looked to design, build and implement a
Real-Time Payments Platform that would offer strategic benefits to the payments industry.

© 2010 Sombers Associates, Inc., and W. H. Highleyman / Damian Ward (VocaLink)

www.availabilitydigest.com
For discussion, contact editor@availabilitydigest.com
2
In October, 2005, a joint proposal from Voca and Link was selected to deliver the payment-processing
infrastructure for the Faster Payments Service.

When the Faster Payments Service went live in May, 2008, it far exceeded the original requirements. It was
more ambitious, more innovative and more exciting for users than the original proposition. Instead of same
day or next day clearing, payments appear in the recipients’ bank accounts within seconds of being initiated.

Implementation Time Scales

The success of the Faster Payments programme depended on having the right technology infrastructure in
place. In building the infrastructure, VocaLink faced many challenges, including a fixed (and public) go-live
date and a fixed budget.

VocaLink, in partnership with APACS (Association for Payment Clearing Services, the U.K. payments
association) and 13 participating banks, were confident in their ability to deliver a high-quality service to the
UK market. The Real-Time Payments Platform design was performed in-house at VocaLink working with key
suppliers.

The real-time transaction switch uses a customised version of the FIS (formally eFunds) Connex Advantage
1
application running on HP NonStop server hardware. This is coupled to a bespoke back-office and
settlement engine built using the Java/Oracle/Sun technology stack. The end result is a robust and fast
system, sized to be able to cope with the projected high volumes of Faster Payments traffic for years to
come.

The decision by VocaLink to provide real-time technology offers the greatest business benefits to banks,
consumers, and corporate customers. The VocaLink strategy reflects a belief that there will be continued
convergence between customer channels, such as Internet, mobile, point-of-sale and ATM. The banking
industry will need to support these channels, and to do so individually is time-consuming and expensive.

Faster Payments Launch

The Faster Payment Service was launched on 27 May, 2008. Payment volumes during the first day
surpassed industry expectation by 300%. Within the first three weeks, 1.7 million payments with a value of
£1 billion had been processed. By the end of 2008, around two-thirds of UK phone and Internet payments
were processed through Faster Payments.

The Faster Payments Service has delivered a world-class solution. It is operating under ever-increasing
volumes and is performing beyond expectations. In major system implementations, there are usually teething
problems; but this service launched with no significant issues and has been recognised by the banking
industry and HM Treasury as an outstanding achievement.

Volume Requirements
Sized to process 300 million payments annually:
 Equates to 542 tps at each site at launch
 Growing to 717 tps in 2011

1
For our U.S. English readers, “bespoke” means “custom.”

© 2010 Sombers Associates, Inc., and W. H. Highleyman / Damian Ward (VocaLink)

www.availabilitydigest.com
For discussion, contact editor@availabilitydigest.com
3
Availability Requirements

 24/7/365 Service Availability

 SLA: 100% during the 00:00 – 06:00 Standing Order batch
run
 SLA: 99.75% overall
 Service credits and penalties apply for exceeding or failing to
meet SLA.

Faster Payments Technology

In the design of the Faster Payments System, VocaLink took a holistic
view of availability. The service availability is supported through:

 Application deployment model (dual site operation, each

capable of 100% of workload).
 HP NonStop server hardware and NonStop operating system.
 Connex Advantage Application design (NonStop
fundamentals).
 Faster Payments Message Protocol (designed to always fail
safe, retries permitted).
 VocaLink high-availability network infrastructure (dual
connections to all data centres).
 VocaLink data centre locations (geographic separation).
 VocaLink power and environmental systems (redundant,
backed-up power and cooling).

The Faster Payments Application

The FPS Core switch is built from the Connex Advantage full-function
EFT switch baseline:
 Same baseline as used for the LINK ATM switch. VocaLink
have almost 20-years experience with this platform or its
predecessors.
 Leverages HP NonStop fundamentals, including NonStop process-pair technology.
 High performance core processing engine.
 Pathway subsystems for services around the core.
 Load-shared active/active single database configuration.
 Near-linear scalability in line with NonStop hardware expansion.
 Throttles workload to protect self and scheme participants.

Dual HP NonStop Servers Per Site

At the time of deployment, the highest
performance HP NonStop platform
available was the NS16200. In order to
meet the volume requirements
stipulated by the scheme, 16 CPUs per
system were required. However, this
would have resulted in no rapid
upgrade path should volume be higher
than predicted. The decision was made
that each site would have two
NS16200 HP NonStop servers with
eight CPUs apiece.

The application was split across these

two servers, with the core application
on what is known as the ‘switch’ server
and pathway services placed on the
‘services’ server. The switch and

© 2010 Sombers Associates, Inc., and W. H. Highleyman / Damian Ward (VocaLink)

www.availabilitydigest.com
For discussion, contact editor@availabilitydigest.com
4
services servers are clustered using Expand over ServerNet, and the result is near-equal workload on each
server.

Performance testing of the distributed application showed negligible performance degradation after making
this change. Measurements indicated that interprocess communication times across the Expand network
were only slightly higher than normal CPU to CPU interprocess communication timings.

VocaLink now have the ability to add additional CPU resources to each HP NonStop server without
significant effort should this be required. Each site with its two clustered servers forms a Faster Payments
processing node.

Service Availability
The FPS scheme stipulated a very
high SLA for the FPS service, and
from day one a dual-site strategy was
pursued. However, the existing dual-
site processing model used by the
Connex Advantage application did not
provide the features required by
VocaLink for the FPS service.

In the standard Connex Advantage

model, each external member
connects to either node A or node B
but significantly not to both.
Transactions are routed between
processing nodes to find their
destinations as required. Members will
still see a small outage each time they
are switched from one FPS node to the other following a failure or during maintenance.

Additionally, in a distributed processing model with an even number of members on each node, significant
levels of transactions need to be processed on both nodes. This will occur where the sending and receiving
members are connected to different FPS nodes. This mode of operation is known as CP (or Continuous
Processing).

VocaLink Active/Active Model

In the VocaLink active/active model
of operation, scheme members
connect to both FPS processing
nodes simultaneously. Members
alternately send transactions to each
FPS processing node.

When one node fails or is taken

offline for maintenance, any sessions
to it are closed down; and member
processing continues uninterrupted
using the remaining node.

When the offline FPS processing

node comes back into service, its
sessions are reestablished with the
member systems; and processing
resumes.

© 2010 Sombers Associates, Inc., and W. H. Highleyman / Damian Ward (VocaLink)

www.availabilitydigest.com
For discussion, contact editor@availabilitydigest.com
5
In the event of transient errors, transactions are still able to cross from one processing node to the other in
order to find an active path to their destination. Live running shows that this scenario is only followed for a
very small number of transactions. The vast majority are processed entirely on one or the other processing
node.

This processing model allows a single node to be taken out of service for maintenance without loss of
service to the members.

Since launch the FPS service has maintained a service availability of 100% despite scheduled single-site
running to cater to application updates or as a precautionary measure while power or other environmental
work is undertaken. In the last year, the service has switched four times to a single-site configuration to cater
to this upgrade work.

Protocol Fail Safe

As part of the overall FPS system design, the application protocol was designed to always fail safe (i.e. the
state of a transaction is always known).

The FPS Switch performs duplicate

monitoring to prevent the same
transaction being submitted more
than once in error. As part of the
failsafe protocol scheme, members
are permitted to resend transactions
once they time out. When the
outcome of the original or resent
transaction is known by the FPS
Switch, it will send the completed
transaction outcome to the submitter.
When the transaction outcome is not
known by the FPS Switch, the
transaction is resubmitted to the receiver.

Other failsafe features of the protocol include reversal on timeout and the processing of payments in a
synchronous and asynchronous manner.

SIP transactions are processed in a synchronous mode, where the transaction must be accepted by the
receiver before an acknowledgement is sent to the submitter. SO payment transactions are processed in
asynchronous mode, allowing the FPS Central Infrastructure to acknowledge the submitter prior to sending
the payment to the recipient.

FPS supports 98 payment flow and recovery scenarios.

Ensuring Quality of Service

FPS supports three major transaction types:

 SIP (Single Immediate Payment) - transactions where a response is required by the end user within
15 seconds.
 SO (Standing Order) - payments where a batch needs to be processed within a set time frame, but
individual transaction timings are less important.
 Unsolicited Messages (USM) - inform participants of the states of the FPS Central Infrastructure and
other member systems. These messages are used to communicate management information
between participants.

It is essential to maintain adequate response times for SIP transactions at all times, as these have end-users
waiting for the transactions’ outcome responses. To ensure this is possible, each transaction class uses
dedicated communications sessions for both the sending and receipt of payment transactions.

© 2010 Sombers Associates, Inc., and W. H. Highleyman / Damian Ward (VocaLink)

www.availabilitydigest.com
For discussion, contact editor@availabilitydigest.com
6
Cross-Site Replication
In order to support the VocaLink
active/active processing model,
extensive use is made of replication
products, specifically the GoldenGate
replication engine.

Files are either synchronised across

sites to maintain consistent system
context or are copied to a shadow
location on the remote node should
recovery be required. Latency-critical
files such as the encryption keys are
replicated by the Connex Advantage
application itself.

Delays in replication of data can cause failures in processing transactions. The FPS Central Infrastructure
mitigates the risk in three ways:

 Ensures that data is replicated quickly.

 Uses logic within the application to safeguard against delays.
 An application design that ensures that data replication fails safe.

How to Control Volumes

The FPS scheme membership comprises members of varying sizes with different transaction volume
requirements. In order to ensure the FPS Central Infrastructure and transaction receivers are not flooded
with Standing Order payments, a simple but effective method of throttling is applied. Each member has a
preset number of sessions configured between it and the FPS service. Each session can have only a single
transaction outstanding, and a session cannot be reused until a response is received or timed out for that
session.

This method of control, although simple, shows how smart system design can be used instead of a more
brute-force approach. The method has been proven to protect the FPS system and its participants and
maintain SIP performance even during the SO batch run.

Faster Payment Settlement System

The FPS Settlement System supports configurable settlement cycles, normally three per banking day at
15:45, 07:00 and 13:00.

The FPS Settlement System also

provides transaction and settlement
position enquires to scheme members.
The following ad-hoc settlement queries
are supported:

 Multilateral Net Settlement

Position
 Bilateral Settlement Positions

The Multilateral Net Settlement Position

shows all of the members’ positions
against each other.

Bilateral Settlement Positions show an individual member’s position against each of the others. A member
can only enquire on their own settlement positions.

© 2010 Sombers Associates, Inc., and W. H. Highleyman / Damian Ward (VocaLink)

www.availabilitydigest.com
For discussion, contact editor@availabilitydigest.com
7
Fraud / Money Laundering Risks
The entire UK Payment Industry was wary of the implications of the FPS scheme. The idea of money flowing
around so quickly and its potential for abuse caused concern with regard to Fraud and Money Laundering
specifically.

In order to address these concerns and mitigate the risks, many settlement, risk, and limit checks have been
built into the FPS system, including single transaction limits, maximum member liability limits, and liability
warning thresholds.

There is a maximum system-wide value set for any single transaction - no item processed by FPS can
exceed this limit.

Summary
The FPS system is a success story of which VocaLink are rightly proud. It has brought about a step change
over traditional payment channels and has positioned the UK payments infrastructure well for the removal of
paper cheques in 2018.

The ‘end-to-end active/active’ model works to maintain service availability. This is proven each time
VocaLink take a processing node in and out of service for maintenance without business-service impact.
Specifically, having scheme members connected simultaneously to both FPS processing nodes is essential
in ensuring system availability.

The ease in which a processing node may be removed and reintegrated with the FPS system has made
VocaLink more confident with regard to change management and change scheduling. Riskier changes may
be undertaken because the FPS service can easily be isolated at one processing site while maintenance is
performed at the other. Application versions and even new NonStop hardware can be added without
impacting the service availability in any way.

FPS scheme volumes are increasing - the announcement by the UK Payments Council that paper cheques
will be phased out in the UK by 2018, displacing four billion payments per annum into other channels, will
increase volumes significantly.

VocaLink are confident the model will scale as they have only 8 CPUs per system at present. Using the
current architecture (NS16200), VocaLink can double the compute power of the existing infrastructure. A
move to HP NonStop Blades (NB50000) will allow each FPS site to be reduced to a single-system footprint.
Introduction of quad-core CPUs will reduce the footprint further and will allow for massive growth.

There are many different ways to replicate data; and each replication technique must be analysed, taking
into account the sensitivity to latency and the probability of collisions when selecting the most appropriate
replication method.

With thought, some aspects of the system can be designed to be failsafe in the event of a data replication
failure or delay. Where this is not possible, real-time data is best replicated by the application.

The Faster Payments Service has revolutionized payment settlement, moving it from a slow batch process to
a nearly instantaneous, interactive service. FPS’ use of active/active technology to eliminate planned
downtime and to survive unplanned failures ensures the continuous availability of settlement services to the
retail community.

© 2010 Sombers Associates, Inc., and W. H. Highleyman / Damian Ward (VocaLink)

www.availabilitydigest.com
For discussion, contact editor@availabilitydigest.com
8