MODIFIED BGP HAS FASTER CONVERGENCE

Swapan K. Ray, SMIEEE, Professor, Computer Science and Engineering. Department, Jadavpur University, Kolkata-700032; Email: skray@ieee.org Prasenjit Ghosh, Programmer Analyst Trainee, Cognizant Technologies Solutions, Kolkata. Santanu Sen, Senior Lecturer, Guru Nanak Institute of Technology, Sodepur, W.B.
Abstract: BGP, the de-facto standard for the inter-Autonomous System (AS) routing protocol in the Internet, suffers from a slow convergence problem following a fault. A Modified BGP (MBGP) has been proposed to make the route selection in BGP more intelligent and proactive towards achieving faster convergence. The MBGP incorporates two modifications, namely, deleting all routes containing a withdrawn route as a segment and advertising an unreachable AS as a Lost Destination. Based on a simplified, policy-independent and timerless implementation, functioning of both BGP and MBGP has been simulated for a number of different BGP graphs. Convergence behaviour, following randomly injected node and link failures, has been studied for both BGP and MBGP. The test results demonstrate spectacular convergence speed-up in MBGP in several cases where either or both of the modifications are meaningful.

I. INTRODUCTION Border Gateway Protocol (BGP) is the de-facto standard for the inter-autonomous system (AS) or interdomain routing protocol in the global Internet. [1],[2]. It allows the BGP-speaking routers in different ASes, peering at private or public exchange points, to exchange network reachability information with their neighbours for the purpose of routing of packets between different ASes within the Internet. Though BGP is inherently a kind of distance vector routing (DVR) protocol [3], [4], yet BGP differs from the DVR in at least two important ways. First, instead of just advertising its distance, in total number of hops, to a certain destination, an AS advertises its complete AS-Path upto the destination. Second, the path or route selection in the BGP is based on not only the shortest AS-path but also the policies that may be chosen by each router, in matters like choosing to accept or ignore any received advertisement, choosing to advertise or not advertise a chosen path to a certain neighbour, etc. AS, etc., so that BGP-speaking routers do not always select the shortest path. Routing instability, caused by a link or router failure, and the resultant delayed convergence is a serious problem with the BGP [5],[6],[7],[8]. Initially, it was thought [8][6] that software bugs and unsafe and mutually unsatisfiable policies in the implementation of AS routers resulted in persistent route oscillations but soon it was established that even modifications to BGP policies which guarantee the nondivergence of the protocol [7] could not significantly improve the routing instability [8]. Labovitz et al [8] reported an important fact that the policies in the majority of routers deployed in the Internet default to a shortest path first - route selection policy. Their experimental measurements showed that, after a fault, inter-domain routers may take even tens of minutes to reach a consistent view of the network topology. These delays stem from temporary routing table fluctuations formed during the operation of the BGP path selection process on the Internet backbone routers. They identified two

factors, namely, the unforeseen interaction of protocol timers with specific router vendor implementation decisions and the ambiguity in the BGP specification, as being mainly responsible for the delayed convergence in BGP. Their research clearly demonstrated that though BGP is free from the count-to-infinity or looping problem, yet it badly suffers from the other inherent and well-known problem of the DVR, namely, slow convergence. In a recent study [9], a modified DVR algorithm (MDVRA) has been proposed which converges fairly quickly and avoids counting-to-infinity. The MDVRA incorporates an intelligent and proactive updation of routing tables and performs, before accepting any good news brought in by a received update message, a pair of plausibility checks or cross checks. The present paper reports the results obtained from a recent work [10] done towards making BGP converge faster which utilizes some of the ideas generated in [9] and [11] towards achieving faster convergence in the basic DVR algorithm. Since, as noted in [8], most BGP router vendor implementations ultimately default to the best path selection based on the shortest AS path, policies have been left out of consideration in the present study and the routing in BGP has been assumed to be based solely on the shortest AS-path hop count. Following this introductory section, a brief review of the BGP is provided in section II. Section III describes the two modifications that have been introduced in the proposed Modified BGP (MBGP). A scheme for implementation and simulation of the BGP is briefly described in section IV. Simulation of the MBGP and some results of performance comparison between BGP and MBGP are presented in section V. Finally, some concluding remarks are made in section VI. II. BRIEF REVIEW OF BGP BGP is a complex protocol and much of its complexity lies in its complex policies, although, strictly speaking, policies are not part of the BGP routing protocol. Since policies have been left out of consideration in the present study, only the salient features of the BGP that concerns routing are briefly reviewed in this section. The Internet has a routing hierarchy and is viewed, at the highest or backbone level, as an interconnection of thousands of ASes, each AS or domain being itself a large internetwork. Most of the ASes are internet service providers (ASP) at different tiers, large university campuses, corporate offices, etc. ASes exchange interdomain routing information through BGP. Each AS is identified by a 16-bit AS# and has its CIDR-ized network prefix. There may be one or more BGPspeaking routers within each AS. However, all of them may not communicate with external peers; some may only

communicate with the BGP-speaking peers within their own AS. BGP views each AS either as a transit network or as a stub network. The former has two or more BGP speaking routers connected to external peers in adjacent ASes and allow transit traffic to pass through them. The later usually has one BGP-speaking router, connected to an external peer, which only generates or receives packets. Sometimes, however, a stub network may be multi-homed [2]. BGP peers exchange four types of messages, namely, OPEN, KEEPALIVE, UPDATE and NOTIFICATION. However, two peers need not be physical neighbours; they only maintain a TCP connection that offers a reliable transport of information. After booting and configuration, a BGP-speaking router sends an OPEN message to its peer to establish a session. The OPEN message contains the BGP version number (presently 4), senders AS number and IP address, and a Hold Time; KEEPALIVE messages are exchanged between peers often enough not to cause the Hold Timer to expire. UPDATE messages, the heart of the BGP, are used to exchange routing information between BGP peers. Finally, a NOTIFICATION message is sent by either peer when it detects an error condition and decide to close the TCP connection. Since only UPDATE messages are of interest in the present work, we describe them in some details. Routing information contained in UPDATE messages has two forms, namely, announcements and withdrawals. A route announcement advertises either a new route, i.e., route to a newly discovered destination, or an alternate or changed route for an existing destination. The latter may be caused due to either a fault (i.e., a link or router failure) on the existing route or a changed policy decision to prefer the alternate route. A route withdrawal is sent when the existing route to a destination AS is no longer available owing to either a fault or a changed policy decision and, additionally the destination has become unreachable via any other path. Thus a route withdrawal is an explicit withdrawal while a route announcement may convey an implicit withdrawal. An UPDATE message contains at most one route announcement but may contain any number (including zero) of explicit withdrawals. A route announcement specifies the complete AS-path upto the destination while a withdrawal announcement only specifies the unreachable destination prefix. The router receiving a route announcement uses the AS-path for both loop detection and policy decision, the loop detection being achieved by rejecting any route that includes the routers own AS number in the path. In order to reduce the routing overhead traffic, BGP specification includes a minimum route advertisement interval timer which ensures that a minimum amount of time (must be considerably less than the mutually agreed hold time) must elapse between two successive advertisements made by a router for the same destination. This timer, however, applies to only route announcements and not withdrawals. In order to dampen routing instability, a BGP route dampening algorithm is used on many routers to hold-down or refuse to believe route updates for a presettable time interval when such route updates arrive much too frequently. However, the use of both

the minimum route advertisement interval timer and the BGP route dampening timer appears somewhat ad-hoc and, as reported in [8], efficacy of both is doubtful. III. MODIFICATIONS PROPOSED IN THE MBGP A good amount of study has gone into the problem of delayed convergence in BGP and several measures towards its faster convergence have been suggested [5], [8]. In the present work, efforts have been directed towards a more proactive and intelligent route selection, following the receipt of an UPDATE message. Two modifications, as described below, have been introduced in the MBGP route processing algorithm to make the protocol converge faster. Modification 1. Deleting all paths containing a failed path: This modification has been adopted from [9] where a router Rj carries out a plausibility check before acting upon each route advertisement received from its neighbours. If router Rk is a common neighbour of both Rj and its neighbour Rl (i.e., Rj, Rk and Rl form a triangle in the BGP graph), and if Rl claims, for reaching a certain remote destination Ri via Rk, a distance less than what Rk itself claims for reaching Ri, then Rj simply rejects Rls advertisement as implausible. In the context of BGP where path vectors are sent instead of distances, this translates into the following scenario. Consider that a router Rj presently reaches a remote destination Ri via one of its neighbours Rk (i.e., the route Rk .Ri is presently the active route of Rj for reaching Ri). Suddenly, Rj receives from Rk an explicit withdrawal of the destination Ri. It is obvious that (i) either a link or a router on the path RkRi has failed and, additionally (ii) no alternative route to Ri via Rks other neighbours (including Rj and Rl) could be found. Now, because the path Rk..Ri no longer exists, MBGP requires that Rj first removes all backup (non-active) routes which contain (i.e., includes as a segment) the failed path vector Rk.Ri, deletes the active route Rk.Ri to Ri and, finally, looks for the shortest alternative route for reaching Ri, from among all the remaining backup routes. If successful, Rj makes this route the active route for reaching Ri and advertises this new active route to its neighbours; otherwise, it advertises an explicit withdrawal of the destination Ri. Modification 2. Declaring an unreachable AS as a Lost Destination: This modification has been adopted from [11]. If Rj is the only neighbour of router Rk, then Rk ( Rk lies within a stub AS) is a singly connected pendant neighbour of Rj so that all routers presently reach Rk via Rj only. If Rj ever finds that Rk has become unreachable, then Rj realizes that not only Rj itself but no other router in the entire Internet can reach Rk which has, effectively become a Lost Destination (LD) to the entire Internet. Obviously, it will be futile for any router (including Rj itself) to look for an alternative path for reaching Rk. Hence Rj floods a special explicit withdrawal message called Lost Destination message on the entire Internet.

MODIFICATION 1 and MODIFICATION 2 have been incorporated in the simplified implementation of BGP [10]. The following section presents this simple implementation and simulation of the BGP and the MBGP. IV. BGP IMPLEMENTATION AND SIMULATION

BGP implementation is, to a large extent, vendordependent. As a consequence, little information is available in the open literature on how BGP is implemented on a router. However, using the broad approach to BGP implementation available in [1], we have built up the scheme shown in Fig. 1 for a simplified implementation of the BGP for the purpose of this research. The behaviour of the BGP and the MBGP have been simulated based on this scheme of implementation. However, the issue of policies regarding selection and advertisement of routes has not been considered and routing has been based solely on the shortest AS-path. A BGP-speaking router, having n Network Interfaces (NI), receives from and sends to its n neighbours four different types of messages, namely, OPEN, KEEPALIVE, UPDATE and NOTIFICATION, over each of these n NIs. A Front End Process (FEP) interacts with all the NI inputs and all the BGP timers. The FEP handles all input messages itself except UPDATE messages and generates a Neighbour Unreachable Trigger (NUT k ) whenever any neighbour Rk becomes

unreachable, i.e., Rk sends no KEEPALIVE message before the local Hold Timer for Rk ticks. Besides the FEP, there are three tables, one information base and four procedures that collectively implement the BGP in the router. The three tables are the Neighbour Table (NT) which keeps the records of all the n neighbours, the BGP Routes Table (BGPRT) that stores all routes, both active as well as backup, and the Active Routes Table (ART) which stores only the active routes that are being used for reaching different destination routers and that have been advertised to the neighbours. The Policy Information Base (PIB) stores the policies that have been configured into the router to guide its route selection and advertisement procedure. Out of the four procedures, the main one is the Update Message Handling Procedure (UMHP) which calls, one by one, the other three procedures, namely, the BGPRT Updation Procedure (BGPRTUP), the Route Selection Procedure (RSP) and the Advertisement Procedure (AP). Whenever an UPDATE message arrives at any NI, the FEP calls the UMHP. After some initial processing of the message, the UMHP first calls the BGPRTUP for updating the BGPRT. If the received message contains a route announcement, the route is either inserted as a new entry or is made to replace an existing entry in the BGPRT. In case of a withdrawal, the existing entry is deleted. Before return, the BGPRTUP creates a BGPRT Modified Element List (BGPRTMEL) which stores the row number alongwith the kind of modification done (Insert/ Replace/ Delete) for each of the modified entries in the BGPRT. The UMHP next calls the RSP, and passes it the pointer to the BGPRTMEL. The RSP performs the necessary selection of the active routes by considering the contents of the BGPRT, the BGPRTMEL, the NT and the NUT, if any. In case an NUT k is received, all the routes previously advertised by Rk are first deleted from both the ART and the BGPRT and then only alternative active routes are then selected, if possible, for the active routes deleted from the ART. Besides modifying the appropriate entries in the ART, RSP also prepares, before return, an ART Modified Elements List (ARTMEL) which is similar to the BGPRTMEL. The UMHP finally calls the AP by passing it the pointer to the ARTMEL. By accessing the ART, the ARTMEL and the PIB, the AP advertises each modified active route, as either announcement or withdrawal, to all the neighbouring routers. The AP alongwith the PIB have, however, not been included in our study. Based on the BGP implementation scheme described above, a study of the convergence behaviour of the BGP has been carried out through simulation., We have assumed, for simplicity, that each AS, stub or transit, has a single BGPspeaking router (node) having one or more external BGP connections. For simplicity, we have also assumed ideal TCP connections with no channel delay or errors, and ideal routers with no processing delay. The simulation program SIMULATOR.CPP first builds, by scanning a graph.grf file containing some given parameters, the desired graph of the network and then builds up, through exchange of UPDATE messages, the steady state routing table of each AS. For this purpose, a global message queue has been employed so that

messages generated by different routers are processed according to their order of generation. Failure of a node or a link, chosen randomly, is now imposed on the graph and UPDATE messages are again exchanged between the ASes via the global queue till the BGP reconverges and routing tables become stable again. Of course, no new fault is imposed until BGP has converged after the previous fault. The convergence behaviour of BGP in response to various faults was displayed by generating a Hyper Text Markup Language (HTML) output file which was viewed on the Internet Explorer. V. MBGP SIMULATION AND PERFORMANCE IMPROVEMENT The method of implementation and simulation for the MBGP is similar to that described for the BGP except for the following changes. 1. Every UPDATE message is now assumed to include a 1-bit Pendent Neighbour (PN) flag and every withdrawal inside it a 1-bit Lost Destination (LD) flag. 2. When a router Rk sends an UPDATE message to its neighbour Rj, the former sets PN=1, if it is a PN of Rj
.

3.

4.

5.

6.

7.

The NT of each router is augmented with an additional column storing the most recent 1-bit PN status of each neighbour. Both the BGPRTMEL and the ARTMEL are augmented with an additional kind of modification of an entry, in addition to the existing 3 modification types, namely, Insert, Replace and Delete. The new modification type is Delete-Per which is a permanent delete in contrast to the existing Delete which has been redesignated as a temporary delete or DeleteTemp. Delete-Per implies that no router should spend any effort to find out an alternative route for this Lost Destination because that will be futile. When the FEP in a router discovers that a certain neighbour Rk has become unreachable, it sends a NUT (k) signal to its RSP. The RSP then checks from the NT if Rk was its PN. If Rk was its PN, the RSP sets a Delete-Per (instead of a Delete-Temp) modification in its ARTMEL against the entry for Rk so that the AP advertises a withdrawal of the route to Rk with LD=1. When a router receives an UPDATE message containing a withdrawal with LD=1, its BGPRTUP as well as its RSP simply set the modification type for the corresponding entries in its BGPRTMEL and ARTMEL, respectively, as Delete-Per. The AP in the router advertises each Insert and Replace entry in the ART as a route announcement and each Delete-Temp and Delete-Per entry as a withdrawal. However, against a Delete-Per, the withdrawal is accompanied by LD=1.

It should be noted that the above set of changes collectively implement only the second modification in the MBGP and allow it to achieve faster convergence by effectively flooding the Lost Destination information quickly and hence reducing futile message exchanges. The first modification is implemented by only introducing suitable changes in the RSP algorithm as was earlier explained. The MBGP was simulated in an identical manner as the BGP after incorporating the implementation changes described above. In order to compare the performance of the MBGP with that of the BGP, 12 simple graphs of different nature, having their number of nodes lying between 4 and 9 with the degree of individual nodes varying from 1 to 4, were generated for study. A total of 26 failures, 14 router and 12 link, with the failed routers and links chosen in a constrained random manner, ensuring at least one router failure and one link failure in each of the 12 graphs, were injected. BGP convergence behaviour was studied in terms of the total number of messages exchanged in the network before reconvergence, following each fault. Out of the 26 cases of failure, MBGP showed a result which is marginally inferior result (highest inferiority 150:165) in only 2 cases, identical in 16 cases and considerably superior (highest superiority 456:24) in 8 cases. Performance of both BGP and MBGP was identical with graphs of regular or near regular structures like a ring, a complete graph, a complete binary tree, etc. A router failure took increasingly more time to converge than a link failure as the graph became more richly interconnected. For example, the number of message exchanges for a router failure was 6, 16, 30 and 150 for a complete binary tree with 7 nodes, a ring with 6 nodes, a complete graph with 4 nodes and a complete graph with 5 nodes, respectively. Corresponding figures for a link failure were 10, 15, 4 and 6, respectively. Out of all these eight cases, the MBGP showed identical results as BGP in 7 cases. However, for the fourth graph and against a router failure, the MBGP took 165 exchanges to converge compared to 150 of BGP. Performance of the MBGP was spectacularly superior in the graphs shown in the figures 2(a) through 2(c). They show the injected failures and the corresponding number of messages exchanged before reconvergence. Modification 2 coupled with the quick dissemination of the Lost Destination information, made possible by the complete connectivity of the router 2 with the remaining routers, appear to lie behind the improvement in Fig. 2(a). Improvement in Fig. 2(b) and in Fig. 2(c) were apparently due to modification 1. The marked extra improvement showed by Fig. 2(c) over Fig. 2(b) was due to complete connectivity of routers 5 and 3 with the other members of their respective clusters. VI. CONCLUDING REMARKS Studies based on two algorithmic modifications have been carried out to improve the speed of post-fault convergence in BGP, the critically important interdomain routing protocol that runs in the Internet backbone. Since BGP is inherently a distance vector routing (DVR) protocol,

some ideas generated in a recent research towards making the DVR converge faster have been applied to make the BGP also converge faster. The basic approach adopted in the proposed Modified BGP (MBGP) has been to make the route selection process in the BGP more intelligent, proactive and efficient. The policy aspects which can influence the selection and advertisement of routes in the BGP have not been considered for simplicity as well as because of the recent research findings that the policies in the majority of routers deployed in the Internet default to a shortest path first - route selection policy. Graph Fault in the Graph Link (2,5) failed Node 5 Failed (a) Link (3,4) failed Node 4 Failed (b) Link (3,4) failed Node 4 Failed 413 26 118 58 No. of messages BGP MBGP 204 204 20 20

available on the RFC on BGP and necessary modifications proposed for the MBGP have been incorporated. A simulation program based on this scheme of implementation has been written in C++. Display of the result of this simulation, namely, the exchange of messages by the BGP-speaking routers, has been provided by generating a HTML output file which can be viewed on any standard web browser. Testing towards the desired comparison between MBGP and BGP has been carried out by generating different graphs and imposing randomly generated router and link failures on each graph, one failure at a time. The detailed results of this study have been presented and they appear fairly encouraging. Spectacular speed-up, measured in terms of the total number of messages exchanged before reconvergence, has been obtained with MBGP in some of the graphs against some of the failures where one or both of the proposed modifications were meaningful. Further research on this problem is in progress. ACKNOWLEDGEMENT The authors gratefully acknowledge the help received from Sri Jeevesh Kumar in the preparation of the manuscript. REFERENCES
Y. Rekhter and T. Li, A Border Gateway Protocol 4 (BGP-4). <draft-ietf-idr-bgp4-25.txt>,2003. 2. I. V. Beijnum: BGP, OReilly SPD, First Indian Reprint, May 2003. 3. D. Bertsekas and R. Gallagar: Data Networks, 2nd Ed., Englewood cliffs, NJ, Prentice Hall, 1992. 4. G. Malkin, RIP Version 2, RFC 2453, 1998. 5. T. Griffin and G. Wilfong, An analysis of BGP convergence properties, Proc. ACM SIGCOMM, Aug. 1999, pp. 277-288. 6. C. Labovitz, G.R. Malan and F. Jahanian, Internet routing instability, IEEE/ACM Trans. Networking, vol. 6, No. 5, Oct. 1998, pp. 515-528. 7. T. Griffin and G. Wilfong, A safe path vector protocol, Proc. IEEE INFOCOM, vol. 2, Mar. 2000, pp. 490-499. 8. C. Labovitz, A. Ahuja, A. Bose and F. Jahanian, Delayed Internet Routing Convergence, IEEE/ACM Trans. Networking, vol. 9, No. 3, June 2001, pp. 293-306. 9. S. K. Ray, S. K. Paira and S. Sen, Modified distance vector routing avoids count-to-infinity problem, Proc. Intl. Conf. CODIS 2004, held in Calcutta during Jan 8-10, 2004, pp. 31-34. 10. P. Ghosh: Modification of BGP towards faster convergence, Master of Comp. Sc. & Engg. (MCSE) Thesis, Jadavpur University, 2005. 11. S. K. Ray, S. Sen and P. Ghosh, Modified distance vector routing for fast convergence and no counting-to-infinity, Manuscript under preparation. 1.

128

60

456

24

(c) Fig. 2 Graphs, imposed faults and messages exchanged before reconvergence shown for BGP and MBGP. Two modifications in the BGP route selection processing algorithm have been proposed. In the first one, a router utilizes the withdrawal advertisements while in the second one, it utilizes its prior knowledge that a neighbour has no other neighbour than itself, i.e., it is singly connected or pendant. According to the first modification, if a neighbour of a router withdraws its route to a certain destination, then the router deletes all existing routes which contain the withdrawn route as a segment. According to the second modification, if a pendant neighbour ever becomes unreachable, the router realizes that this neighbouring router has got disconnected from the network so that it should be treated as a Lost Destination and all routers must not send or forward it any more packets. In order to study and compare the convergence behaviour of the BGP and the MBGP, a scheme of implementation for the BGP has been developed based around the information