High Speed Internet Access Using Satellite-Based DVB Networks

Nihal K. G. Samaraweera and Godred Fairhurst Electronics Research Group, Department of Engineering University of Aberdeen, Aberdeen, AB24 3UE, UK. Abstract
Digital Video Broadcasting (DVB) provides the opportunity for high-speed Internet delivery direct to users homes and offices. Most applications require full duplex communication, and the return path from the user to the Internet will be provided using lower speed terrestrial connections. Two challenges are presented when using DVB for high-speed Internet access: the poor performance of TCP over network paths with a high bandwidth delay product (LFN) and the impact of asymmetric network paths. The second effect is most significant and results in a problem, which we have termed ACK congestion. The performance of TCP using a DVB satellite link is analysed through simulation and a set of techniques is proposed which provide significant improvement.

Figure 1 shows individual users and a remote Local Area Network (LAN) accessing a data server (located in the Internet or at a DVB hub site) using the DVB link. A client sends requests for data transfer (and later acknowledgments as the session progresses) through the terrestrial network, while the server transfers the data to the client through the higher speed satellite link. A client networking device (the client computer itself or the router) has two network interfaces; one connected to the receive DVB link and the other connected by an Internet Service Provider to the terrestrial network.
DVB MUXs and Satellite Modem Set-TOP Box Up link Station HUB Router HUB Server Terrestrial Internet / ISDN Data Server Client Router /Gateway Client L A N

1 Introduction The growing use of multimedia-capable personnel computers to access the Internet and in particular, the use of world wide web, has resulted in a growing demand for Internet bandwidth. The emphasis has moved from basic Internet access to the expectation that connectivity may be provided whatever the location. This presents fresh challenges to the networking community, particularly as users become familiar with the benefits of high speed connectivity. Along with an increased use of the Internet, there has also been a revolution in television transmission, with the emergence of Digital Video Broadcasting (DVB) [1, 2]. Recent developments will allow packet data services to provide high speed transmission using a DVB satellite hub station (6-34 Mbps) to transmit packet data to the same small satellite dish as used for TV distribution. In many cases, the available low speed terrestrial infrastructure (e.g., dial-up modem, ISDN connection) may be used for the return link. This results in a network connection in which the capacity to a remote server differs from that from the server. Such an asymmetry in the network paths may be suited to user needs, since most Internet connections receive much more data than they need to send [3]. However, a high degree of asymmetry in the forward and return paths introduces a potential bottleneck in performance.

Figure 1: Configuration showing connection via a DVB satellite network

Most current Internet applications use TCP [4] for reliable data transfer. This has fundamental limitations (e.g., insufficient window size) which prevent an application achieving high throughput over a high bandwidth satellite network. This paper compares the performance of standard TCP with that using the Long Fat Network (LFN) extensions [5]. The need for and the benefit of the SACK extension is also examined. The particular case of asymmetric delivery is then reviewed and a set of potential extensions identified. The extensions have been implemented in a TCP/IP which contains a full implementation of TCP and a model for the DVB link [6-9]. 2 TCP over Satellite Links Since TCP was first introduced in early 1980, it has been improved to provide efficient and reliable operation over a variety of transmission media. This has lead to a series of TCP reference implementations. A satellite 23

link has distinguishable characteristics, not normally found in other transmission media, which adversely impact TCP performance. These are (i) long propagation delay, (ii) a large bandwidth-delay product, and (iii) bandwidth asymmetry. The congestion control (Slow Start) and congestion avoidance (multiplicative decrease) procedures [10-11] were first introduced by the Tahoe TCP implementation. This allows a TCP transmitter to transmit a limited number of bytes determined by the smallest value of the receiver advertised window or the congestion window (cwnd). The algorithm also uses a variable to keep the threshold value of the send window (ssthresh) which is set to one half of the current cwnd (multiplicative decrease) when a packet is lost. The cwnd is set to one segment when the slow start phase begins (when either the connection first starts or the retransmission timer expires), and increased by one segment (exponential increase) whenever an Acknowledgment (ACK) is received, until it reaches the ssthresh. At this point, the algorithm switches to the congestion avoidance (linear increase) phase. The high delay of a satellite link significantly slows the growth of the congestion window. Each ACK is delayed by the round-trip time over the satellite link, and only when received, does it allow the sender to increase the cwnd by one segment (MSS [11]), during the slow start phase, and by a fraction of the MSS (equivalent to one MSS per round trip delay) during the linear increase phase. The Slow Start procedure [10] was introduced to achieve network stability under Internet congestion. It is however, known to be unduly conservative [11], since on detection of congestion, it drains the pipe (i.e., waits until all transmitted packets have left the network by setting the cwnd to one segment) before transmission of any more data. The Fast Retransmission and Recovery algorithms [11] introduced by the Reno TCP implementation improve performance by draining only one half of the pipe and then recommencing transmission, assuming the reception of each duplicate ACK is an indication of a packet leaving the network. If two or more packets have been lost from the transmitted data (window), these algorithms use partial ACKs (which do not cover all data sent, when the packet lost is first detected) to recover multiple packet losses without waiting for retransmission time out [12, 8].

3 TCP Extensions for a LFN The standard TCP does not perform efficiently over a LFN due to: (i) insufficient window size, (ii) inaccuracy of the round trip delay estimation, (iii) inefficient packet loss recovery, and (iv) an inability to provide reliability at high transfer rates due to sequence number wrapping [5]. The IETF has recently standardised a set of TCP extensions to overcome these limitations [5]. These are: a window scale option, a time stamp option, Selective ACK (SACK), and an extension to Protect Against Wrapped Sequence numbers (PAWS) [5, 13]. The PAWS extension is only important for high speed transmission (e.g., >140Mbps) [9]. The TCP time stamp option adds an additional overhead (increasing the ACK packet size by 25%) [7]. This increase may be significant for a terrestrial return link in a DVB network. 3.1 The Window Scale Option Each TCP sender and receiver allocates a buffer space (typically up to 64KB). When receiving data, the receiver returns ACKs (using the 16 bit window field in the TCP packet header) advertising the available buffer space. This advertised value constrains the amount of data that the sender may transmit. To achieve optimum performance, a sender should transmit an amount of data equivalent to the bandwidth delay product of the network path. In a LFN, the maximum supported window size (216) may not be adequate for the sender to fully utilize the network. The window scale option [5] expands the TCP window size by using a scale factor without changing the TCP header. The scale factor is negotiated by an option in the initial (SYN) segment at the connection setup and maintained for the rest of the connection. When using the extension, both the send and receive windows are implemented using a 32 bit number. The window size advertised by a receiver is right shifted by the scale factor before being encoded in each out-going TCP header. When the packet is received, the window value is left shifted by the scale factor before updating the send window. A 34Mbps satellite link (figure 1) was simulated to investigate the need for using the window scale option. TCP was configured to use an MSS of 1024 bytes. Table 1 shows the average throughput of 3 simultaneous sessions for different window sizes: the maximum window size without using 24

window scaling (64 KB) and a range of windows with window scaling. (N.B. the limited bandwidth on the return is not considered here (see section 4), and all paths are assumed symmetrical).
Table 1: TCP performance for different window sizes and different terrestrial delays (satellite delay is 280ms)
Window Delay [ms ] 100 200 300
[KB]

TCP Throughput [kbps] 720 8775 8764 8495 512 8646 8305 6781 256 5258 4171 3408 128 64

2624 1298 2089 1037 1705 846.4

The results in table 1 suggest that TCP may transmit data from a server at the hub site at a throughput <1.3Mbps without using window scaling. The throughput would be correspondingly decreased when the data was sent from a remote Internet site (i.e. increasing the overall round trip delay). Figure 2 shows the variation with time of cwnd, ssthresh, maximum send and receive window size, and the number of bytes in transit at a time (only two cases were illustrated for simplicity). In both cases, the cwnd has been increased exponentially (slow start) up to ssthresh (section 2) before starting linear growth. If the data transfer does not last longer than the initial slow start period for a 64 Kbytes cwnd (i.e. for small transfers), the window size has no impact.
(C) Congestion Window, etc [Bytes] (B) (A) (D) (A) (B) (C) (D) (E) (F) (G) (H) (F) (E) (G)

curves for the cwnd and the number of bytes in transit overlap, showing that the receiver always advertises a sufficient window (i.e. greater than the estimated cwnd). Window scaling improves performance over an LFN when the bandwidth delay product exceeds 64 KB and the volume of data to be transferred is also more than 126 KB. It may therefore be useful for ftp and similar bulk transfer applications over DVB network. 3.2 Selective Acknowledgment Option The probability of multiple packet loss in a window is much greater for a LFN, where more packets are in flight. Although the new TCP implementations (e.g., Reno) are capable of recovering multiple packet losses without waiting for expiry of the retransmission timer, frequent packet loss may still not be efficiently recovered using such an implementation. The Selective ACKnowledgement (SACK) extension [13] has been recently proposed to improve TCP performance over such a network [8], but has not yet been widely implemented. The SACK option is triggered when the receiver buffer holds in-sequence data segments following a packet loss. The receiver then sends duplicate ACKs bearing the SACK option to inform the transmitter which segments have been correctly received. When the third duplicate ACK is received, a SACK TCP transmitter retransmits the lost packets starting with the sequence number acknowledged by the duplicate ACKs, and followed by any subsequent unacknowledged segments. The Fast Retransmission and Recovery algorithms are also modified to avoid retransmitting already SACKed segments. The explicit information carried by SACKs also enables the transmitter to accurately estimate the number of transmitted data packets that have left the network [8, 13].
Table 2: The SACK extension reduces number of retransmission and improves the TCP throughput (even a single retransmission is significant, since sender loses hardly earned cwnd with satellite delay).
Queue size (packets) 300 600 900 1200 8017

(H)

Figure 2: Window scaling and adequate buffer space improve TCP performance.

Throughput [kbps] 3422 4471 4536

SACK

For the 64 KB case, the transmitter rapidly uses up all the window space which then constrains the throughput. There is no such constraint for a window size of 720 Kbytes (approximately equivalent to the bandwidth delay product of a session). In this case, both

Retransmission [%] 0.15

0.11

0.05 0.0007 7997

Reno

Throughput [Mbps] 2378 2790 3884 Retransmission [%] 0.46 0.33

0.11 0.0007

A 34Mbps DVB network (with 280ms satellite delay and 100ms terrestrial network 25

delay) (figure 1) was simulated to investigate the need for using the SACK option. The maximum number of packets that can be held in the bottleneck router was restricted to the values shown in table 2 to simulate different congestion conditions in the network. TCP was configured to use an MSS of 1024 bytes, and had transmit and receive buffer sizes of 720 KB (sufficient for the DVB link). Table 2 compares the TCP performance (average throughput of 3 simultaneous sessions and the number of retransmissions per 100 transmitted packets) using Reno against SACK (i.e. Reno with the SACK option). A SACK transmitter selectively retransmits only the packets lost during the transmission resulting in few retransmissions of packets compared to Reno (table 2). It is also able to calculate the pipe size accurately using SACKed information to improve the transmission rate [8]. Therefore, the SACK option is able to sustain a high throughput over a network subject to high packet loss and is therefore desirable for bulk transfers over a DVB network. 4 TCP Extensions for Asymmetry A TCP receiver normally sends an ACK for each received packet which the TCP sender uses for two purposes. First, it indicates a previously transmitted packet has left the network, so the sender may increase the cwnd allowing it to transmit more data. Second, it indicates the receiver will accept more data. Therefore, the TCP throughput of a DVB link (figure 1) may become constrained by the rate at which the sender receives ACKs through the low speed return path (a phenomena we call ACK Congestion). During the slow start phase, the transmission rate is constrained by the first factor, and when the cwnd is sufficiently open, the transmission rate is constrained only by the second factor. TCP could be modified to overcome this limitation. Since TCP ACKs are cumulative [11], the throughput can be improved by reducing the number of ACKs generated by the receiver by delaying the transmission of each (cumulative) ACK until a number of packets arrive at the receiver (this is called ACK_delay, and usually delays only one packet [11]). However, this also reduces the growth of the cwnd during the initial phase. The problem of slow growth of the cwnd over a satellite network may be overcome using a larger initial cwnd [14] or using the byte count of the ACKed sequence numbers rather than ACK count itself to increase

cwnd. However, these modifications are not widely accepted, since they may impact the stability of TCP over a large network [15]. In addition, since TCP implementations exist on end systems (i.e., a part of the installed software on all user computers), modifications are undesirable and difficult to manage. Therefore, alternative solutions to reduce ACK Congestion are needed to suit the characteristics of a DVB Network. 4.1 TCP Header Compression TCP header compression [16] is widely used on modem links and may compress the TCP/IP header. An ACK packet may be reduced by 70% over the return link. However, this compression ratio may not be sufficient to prevent ACK congestion when the asymmetric ratio is higher than 80. 4.2 ACK Suppression Another way to avoid ACK congestion is to delete ACKs from the queue which builds up at the return interface. This technique is called ACK Suppression [17, 18]. Such a scheme may be implemented in the data link layer of the equipment at the user premises (similar to implementation of header compression [11]). Implementation at the user premises may also reduce any interaction with security and encryption procedures. The extra processing overhead performing ACK Suppression may be compensated for by avoiding modifications to standard TCP. To perform ACK suppression, each incoming packet forwarded to the link interfere queue is classified by flow (TCP packets are identified by the protocol types of the IP header, and TCP port numbers and IP addresses which uniquely identify a flow). The suppressor tags each packet with a flow identifier, and keeps a packet count for each flow. The queue of packets for each flow is allowed to increase up to a threshold value for suppression. A suitable limit (set according to the asymmetric ratio) improves the TCP performance, allowing the first ACKs of a flow to be received without suppression, allowing the sender to open the cwnd quickly. When the session exceeds the threshold value, packets belonging to this session will be deleted from the front of the queue (oldest ACK) leaving a single (cumulative) ACK. Some ACK packets also have other functions in the TCP protocol and must not be deleted to ensure normal operation of TCP. The suppressor must therefore not delete an ACK packet which carries any data, or has any 26

other TCP flags set (sync, reset, urgent, and final). In addition, the suppressor should avoid deleting a series of 3 duplicate ACKs which indicate the need for Fast Retransmission (section 2). ACK Suppression reduces the level of ACK Congestion. However, the throughput may still remain low. This is because the cwnd grows slowly, due to the small number of ACKs received - even though all data has been cumulatively acknowledged. (The sender congestion avoidance algorithm only increases cwnd on reception of an ACK and does not consider the cumulative nature). This is particularly significant, since most Internet connections are short lived [3] and therefore are always constrained by the cwnd. 4. 3 ACK Compaction A new ACK Compaction technique is proposed which eliminates the effect of ACK Congestion, but maintains an acceptable arrival rate of ACKs at the sender. A data link layer compactor first establishes an IP tunnel to a remote expander (e.g., at the hub site or an Internet Service Providers premises). IP tunneling (or encapsulating an IP packet with another IP header) is a commonly implemented technique in the Internet (e.g., MBONE [11] and DirectPC [18]). Tunneling may also be desirable for asymmetric networking [18] when the return packets of a connection are routed through a different network path than used to send the forward data (asymmetric routing). In the DVB case, a packet from the user (with the IP source of the users satellite interface) is encapsulated with an IP header (with the IP source address of the terrestrial interface and an IP destination address of the remote expander). The remote end of the tunnel extracts the encapsulated packet and forwards it through the Internet as if it were routed through the satellite link. The expander must also validate the received IP addresses to prevent spoofing. ACK Compaction uses the ACK Suppression technique with several modifications. When packets are deleted, the transmitter appends information to the remaining packet which is later used by the expander. The information contains the number of deleted ACKs (one byte) and the average number of bytes acknowledged (two bytes). The expander is stateless (i.e. processes each received packet independently of previously received packets). It uses the 3 bytes of prefixed information together with the

acknowledgment field in the TCP header of the received ACK to produce a burst of ACKs equivalent to the number of ACKs previously deleted by the compactor. These ACKs are then forwarded to the original destination (i.e. the TCP sender). This process is simple to implement and requires little modification to the tunneling software at the remote site. 4.4 Comparison of Techniques A DVB link was simulated to understand the impact of ACK Congestion and the performance of the three techniques. For simplicity a single session was considered limited to a maximum of 10 Mbps over the DVB channel. Most European telephone networks support 9.6kbps modems, although high speed modems are now widely used in some countries. This study considers a 9.6 kbps return link to analyse a network with asymmetric ratio of ~1000:1. TCP was configured to use an MSS of 1024 bytes, and had transmit and receive buffer sizes of 720 KB sufficient for the DVB link (see section 3.1). Figure 3 shows the variation of the cwnd with time for a data transfer from the hub server to a client. Figure 4 shows the time required to transfer a number of bytes over the DVB link.
800 Unmodified Using Compression Using Suppression Using Compaction

Congestion Window [Kbyts]

600

400

Using Suppression

200

Unmodified

10

Time [sec]

20

25

30

Figure 3: The variation of the cwnd with time.

ACK Congestion results when unmodified TCP is used, resulting in a slow increase in cwnd (figure 3) and a low TCP throughput (i.e. large transmission time) (figure 4). In addition, the large queuing delay causes the TCP timers at the sender to expire indicating potential data loss and triggering unnecessary retransmission of data. The TCP congestion control algorithms also trigger, reducing the transmission rate, and therefore throughput. Since ACKs arrive at the sender more quickly using header compression, the cwnd may grow more rapidly [16] (figure 3). However, 27

the compression ratio is not enough to eliminate ACK Congestion for a DVB network, which results in expiry of the TCP retransmission timer, and subsequent slow start. This results in only a small improvement in TCP throughput (figure 4).
28 Unmodified - (A) Using Compression - (B)

Further increases in throughput require the use of the TCP window scaling option. Network congestion has a serious impact on connections with a high bandwidth and delay. This effect may be significantly reduced by using the TCP SACK extension. 6 References
1. ETSI, DVB Specification for Data Broadcasting', Draft EN 301-192 V1.1.1, 1997. 2. DVB project, 'The Global Solution for Digital Television', hhtp://www.dvb.org, 1998. 3. V. Paxson, 'End-to-End Internet Packet Dynamics, SIGCOMM, ACM, France, 139-152 (1997). 4. J. Postel, 'Transmission Control Protocol', Information Sciences Institute, RFC 793, Sep 1981. 5. V. Jacobson, et al., 'TCP Extensions for High Performance', IETF-DRAFT, May 1997. 6. N. Samaraweera, et al., 'Robust Data Link Protocols for Connection-less Service over Satellite Links', Int J. Sat Coms, 14(5), 427-437 (1996). 7. N. Samaraweera and G. Fairhurst, 'Explicit Loss Indication and Accurate RTO Estimation for TCP Error Recovery using Satellite Links', IEE Proceedings - Coms, 144(1), 47-53 (1997). 8. N. Samaraweera and G. Fairhurst, 'Reinforcement of TCP Error Recovery for Wireless Communication', To be published in Computer Communication Review, ACM SIGCOMM, 28(2), 1998. 9. N. Samaraweera and G. Fairhurst, Integration of Internet Traffic with Digital Video, Fifteenth UK Teletraffic Symposium, IEE, UK, March 1998. 10. V. Jacobson, 'Congestion Avoidance and Control', SIGCOMM, ACM, USA, 314-329 (1988). 11. W. R. Stevens, TCP/IP Illustrated: The Protocols, Vol:1, Addison Wesley, New York, 1994. 12. J. C. Hoe, 'Improving the Start-up Behavior of a Congestion Control Scheme for TCP', SIGCOMM, ACM, USA, 270-280 (1996). 13. M. Mathis et al., 'TCP Selective Acknowledgment Options', IETF, RFC 2018, October 1996. 14. S. Floyd et al., 'Increasing TCP's Initial Window', IETF-DRAFT, July 1997. 15. T. Shepard and C. Partridge, 'When TCP Starts Up With Four Packets Into Only Three Buffers', IETF-DRAFT, 1997. 16. V. Jacobson, 'Compressing TCP/IP Headers for Low-Speed Serial Links', IETF, RFC1144,Feb 1990. 17. R.Durst, et al., TCP Extensions for Space Comms', MOBICOMM, ACM, 15-26 (1996). 18. A. D Falk, et al., Hybrid Internet Access, N A S A Centers for Commercial Development of Space. American Institute of Physics, USA,1995.

Number of Bytes Transmitted [Mbyts]

24 20 16 12

Using Suppression - (C) Using Compaction - (D)

(D) 8

(C) (B) (A)

4 0 0 5 10 Time [sec] 20 25 30

Figure 4: First sequence number of the transmitted packet against time

ACK Suppression achieves higher TCP performance compared to header compression by significantly reducing the queuing delay. The suppressor may inhibit suppression until the queue builds up to a threshold value (60 packets in this simulation). However, since the actual number of ACKs received by the sender is significantly reduced (constrained by the bottleneck speed), the growth of the cwnd still limits the throughput (figure 4). When compared with header compression, there is an initial slow growth of the cwnd (for first 6 secs) for ACK Compaction. This arises since compressed ACKs are received more rapidly than the initially non-compacted ACKs. Once ACK Compaction starts to take place, performance rapidly exceeds header compression and completely eliminates ACK Congestion. This therefore avoids TCP timer expiry. TCP throughput has therefore been significantly improved (figure 4). 5 Conclusions A DVB network may provide low cost universal Internet access at high speed. Throughput as high as 216 kbps may be achieved using unmodified TCP protocols and a 9.6 kbps return link. This performance is predominantly constrained by the effect of ACK Congestion on the slow speed return link. A number of extensions have been investigated which reduce or eliminate the effect of ACK Congestion resulting in significant performance improvement (up to 1.3 Mbps for a non-congested network).

Acknowledgments
The Authors wish to thank the European Space Agency (ESA), and R Donadio (ESA), P. Glover (ESA), K. Hodson, R Heron (Delta Communication), R. Eley and C. Yildez (Globecast Northern Europe) for their contribution to this project.

28

29

30