Provider and Peer Selection in the

Evolving Internet Ecosystem

Amogh Dhamdhere

Committee:
Dr. Constantine Dovrolis (advisor)
Dr. Mostafa Ammar
Dr. Nick Feamster
Dr. Ellen Zegura
Dr. Walter Willinger (AT&T Labs-Research)
 The Internet Ecosystem

 27,000 autonomous networks independently operated

and managed
 The “Internet Ecosystem”
 Different types of networks
 Interact with each other and with “environment”
 Network interactions
 Localized, in the form of interdomain links
 Competitive (customer-provider), symbiotic (peering)
 Distributed optimizations by each network
 Select providers and peers to optimize utility function
 The Internet ecosystem evolves
6/7/2019 2
 High Level Questions
 How does the Internet ecosystem evolve?
 What is the Internet heading towards?
 Topology
 Economics
 Performance
 How do the strategies used by networks affect their
utility (profits/costs/performance)?
 How do these individual strategies affect the global
Internet?

6/7/2019 3
 Previous Work
 Static graph properties  Dynamics of the evolving
 No focus on how the graph graph
evolves
 Birth/death
 “Descriptive” modeling  Rewiring
 Match graph properties
e.g. degree distribution  “Bottom-up”
Model the actions of
Homogeneity


individual networks
 Nodes and links all the
same  Heterogeneity
Networks with different
Game theoretic,


incentives
computational  Semantics of interdomain
 Restrictive assumptions links
6/7/2019 4
 Our Approach

“Measure – Model – Predict”

Measure the evolutionModel strategiesPredict the effects

of the Internet and incentives of
of provider and
Ecosystem different network
peer selection
Topological, but focus
types strategies
on business types and
rewiring
6/7/2019 5
 Outline

 Measuring the Evolution of the Internet Ecosystem

[IMC ’08]

 The Core of the Internet: Provider and Peer

Selection for Transit Providers [to be submitted]

 The Edge of the Internet: ISP and Egress Path

Selection for Stub Networks
[Infocom ‘06]

 ISP Profitability and Network Neutrality

[Netecon ’08]

6/7/2019 6
 Motivation

 How did the Internet ecosystem evolve during the

last decade?
 Is growth more important than rewiring?
 Is the population of transit providers increasing or
decreasing?
 Diversification or consolidation of transit market?

 Given that the Internet grows in size, does the

average path length also increase?
 Where is the Internet heading?

6/7/2019 7
 Approach
 Focus on Autonomous Systems (ASes)
 As opposed to networks without AS numbers
 Start from BGP routes from RouteViews and RIPE
monitors during 1997-2007
 Focus on primary links

 Classify ASes based on their business function

 Enterprise ASes, small transit providers, large transit
providers, access providers, content providers

 Classify inter-AS relations as “transit” and “peering”

 Transit link – Customer pays provider
Visibility Issue
 Peering link – No money exchanged

6/7/2019 8
 Internet growth

 Number of CP links and ASes showed initial exponential growth

until mid-2001
 Followed by linear growth until today
 Change in trajectory followed stock market crash in North
America in mid-2001
6/7/2019 9
 Path lengths stay constant

 Number of ASes has grown from 5000 in 1998 to 27000 in 2007

 Average path length remains almost constant at 4 hops

6/7/2019 10
 Rewiring more important than
growth

 Most new links due to internal rewiring and not birth (75%)

 Most dead links are due to internal rewiring and not death
(almost 90%)
6/7/2019 11
 Classification of ASes based on
business function
 Four AS types:
 Enterprise customers (EC)
 Small Transit Providers
(STP)
 Large Transit Providers
(LTP)
 Content, Access and Hosting
Providers (CAHP)
 Based on customer and peer
degrees
 Classification based on
decision-trees
 80-85% accurate
6/7/2019 12
 Evolution of AS types

 LTPs: constant population (top-30 ASes in terms of customers)

 Slow growth of STPs (30% increase since 2001)
 EC and CAHP populations produce most growth
 Since 2001: EC growth factor 2.5, CAHP growth factor 1.5
6/7/2019 13
 Multihoming by AS types

 CAHPs have increased their multihoming degree significantly

 On the average, 8 providers for CAHPs today

 Multihoming degree of ECs almost constant (average < 2)

 Densification of the Internet occurs at the core
6/7/2019 14
 Conjectures on the Evolution of
peering

 Peering by CAHPs has increased significantly

 CAHPs try to get close to sources/destinations of content

6/7/2019 15
 Conclusions
Where is the Internet heading?
 Initial exponential growth up to mid-2001, followed
by linear growth phase
 Average path length practically constant
 Rewiring more important than growth
 Need to classify ASes according to business type
 ECs contribute most of the overall growth
 Increasing multihoming degree for STPs, LTPs and
CAHPs
 Densification at the core
 CAHPs are most active in terms of rewiring, while ECs
are least active
6/7/2019 16
 Outline

 Measuring the Evolution of the Internet Ecosystem

 The Core of the Internet: Provider and Peer

Selection for Transit Providers

 The Edge of the Internet: ISP and Egress Path

Selection for Stub Networks

 ISP Profitability and Network Neutrality

6/7/2019 17
 Modeling the Internet Ecosystem
 From measurements: Significant rewiring activity
 Especially by transit providers
 Networks rewire connectivity to optimize a certain
objective function
 Distributed
 Localized spatially and temporally
 Rewiring by changing the set of providers and peers
 What are the global, long-term effects of these
distributed optimizations?
 Topology and traffic flow
 Economics
 Performance (path lengths)
6/7/2019 18
 The Feedback Loop

Interdomain Routing Cost/price

TM parameters

Interdomain Traffic Per-AS

topology flow profit

Provider
selection

Peer
selection

When does it converge?

When no network has the incentive to
change its connectivity – “steady-state”
6/7/2019 19
 Impact of provider/peer selection
strategies

P1 P1P2
open to
C No and P3
peering
C
peering
peer with
CPs

P2 P3

S S
C S C
S

6/7/2019 20
 Our Approach

 What is the outcome when networks use certain

provider and peer selection strategies?
 Model the feedback loop in the Internet ecosystem
 Real-world economics of transit, peering, operational costs
 Realistic routing policies
 Geographical constraints
 Provider and peer selection strategies
 Computationally find a “steady-state”
 No network has further incentive to change connectivity
 Measure properties of the steady-state
 Topology, traffic flow, economics
6/7/2019 21
 Network Types

 Enterprise Customers (EC)

 Stub networks at the edge (e.g. Georgia Tech)
 Either sources or sinks
 Small Transit Providers (STP)
 Provide Internet transit
 Mostly regional in presence (e.g. France Telecom)
 Large Transit Providers (LTP)
 Transit providers with global presence (e.g. AT&T)
 Content Providers (CP)
Provider and of
 Major sources peer selection
content for STPs and LTPs
(e.g. Google)

6/7/2019 22
 What would happen if..?
 The traffic matrix consists of mostly P2P traffic?
 P2P traffic benefits STPs, can make LTPs
unprofitable

 LTPs peer with content providers?

 LTPs could harm STP profitability, at the expense of
longer end-to-end paths

 Edge networks choose providers using path lengths?

 LTPs would be profitable and end-to-end paths
shorter
6/7/2019 23
 Provider and Peer Selection

 Provider selection strategies

 Minimize monetary cost (PR)
 Minimize AS path lengths weighted by traffic (PF)
 Avoid selecting competitors as providers (SEL)

 Peer selection strategies

 Peer only if necessary to maintain reachability (NC)
 Peer if traffic ratios are balanced (TR)
 Peer by cost-benefit analysis (CB)

 Peer and provider selection are related

6/7/2019 24
 Provider and Peer Selection are
Related
A  Restrictive
peering
X
? C

A B
 Peering by
necessity
C B
A U  Level3-Cogent
peering dispute
U
6/7/2019 C 25
 Economics, Routing and Traffic
Matrix
 Realistic transit, peering and operational costs
 Transit prices based on data from Norton
 Economies of scale

 BGP-like routing policies

 No-valley, prefer customer, prefer peer routing policy

 Traffic matrix
 Heavy-tailed content popularity and consumption by sinks
 Predominantly client-server: Traffic from CPs to ECs
 Predominantly peer-to-peer: Traffic between ECs
6/7/2019 26
 Algorithm for network actions

 Networks perform their actions sequentially

 Can observe the actions of previous networks
 And the effects of those actions
 Network actions in each move
 Pick set of preferred providers
 Attempt to convert provider links to peering links “due to
necessity”
 Evaluate each existing peering link
 Evaluate new peering links
 Networks make at most one change to their set of
peers in a single move
6/7/2019 27
 Solving the Model

 Determine the outcome as each network selects

providers and peers according to its strategy
 Too complex to solve analytically: Solve
computationally
 Typical computation
 Proceeds iteratively, networks act in a predefined sequence
 Pick next node n to “play” its possible moves
 Compute routing, traffic flow, AS fitness
 Repeat until no player has incentive to move
 “steady-state” or equilibrium

6/7/2019 28
 Properties of the steady-state

 Is steady-state always reached?

 Yes, in most cases

 Is steady-state unique?
 No, can depend on playing sequence
 Different steady-states have qualitatively similar properties

 Multiple runs with different playing sequence

 Average over different runs
 Confidence intervals are narrow

6/7/2019 29
 Canonical Model

 Parameterization of the model that resembles real

world
 Traffic matrix is predominantly client-server (80%)
 Impact of streaming video, centralized file sharing services
 20% of ECs are content sources, 80% sinks
 Heavy tailed popularity of traffic sources
 Edge networks choose providers based on price
 5 geographical regions
 STPs cheaper than LTPs

6/7/2019 30
 Model Validation

 Reproduces almost constant average path length

 Activity frequency: How often do networks change
their connectivity?
 ECs less active than providers – Qualitatively similar to
measurement results
6/7/2019 31
 Results – Canonical Model

LTP  Hierarchy of STPs

S1 Trafficcan bypass LTPs –

LTPs unprofitable
S2 CP
STPs should not peer
EC EC
with CPs
CP CP Resist the temptation!

6/7/2019 32
 Results – Canonical Model

LTP What-if: LTPs peer with

CP CP
CPs
CP S1
Generaterevenue from
downstream traffic
S2

Can harm STP fitness

EC EC
Long paths
6/7/2019 33
 Deviation 1: P2P Traffic matrix

CP LTP CP P2P traffic helps STPs

Smaller traffic volume
CP S1 from CPs to Ecs

More EC-EC traffic =>

S2 S3
S2 balanced traffic ratios
More opportunities for

EC ECEC EC EC EC
STPs to peer
Peering by “traffic
ratios” makes sense
6/7/2019 34
 Conclusions

 A model that captures the feedback loop between

topology, traffic and fitness in the Internet

 Considers effects of
 Economics
 Geography
 Heterogeneity in network types

 Predict the effects of provider and peer selection

strategies
 Topology, traffic flow, economics, and performance
6/7/2019 35
 Outline

 Measuring the Evolution of the Internet Ecosystem

 The Core of the Internet: Peer and Provider

Selection for Transit Providers

 The Edge of the Internet: ISP and Egress Path

Selection for Stub Networks

 ISP Profitability and Network Neutrality

6/7/2019 36
 The Edge of the Internet

 Sources and sinks of content

 Content Providers (CP): sources
 Enterprise Customers (EC): sinks
 From measurements:
 ECs connect increasingly to STPs
 Cost conscious ?
 CPs connect increasingly to LTPs
 Performance ?
 Increasing multihoming (about 60% of stubs)
 Redundancy, load balancing, cost effectiveness
 How should stub networks choose their providers?
6/7/2019 37
 Major Questions
 How to select the set of
upstream ISPs ?
 Low monetary cost
 Short AS paths to major
destinations
 Path diversity to major traffic
destinations – robustness to
network failures
 How to allocate egress traffic
to the set of selected ISPs ?
 Objective: Avoid congestion on the
upstream paths
 Also maintain low cost
6/7/2019 38
 ISP Selection
 Select k ISPs out of N

 Let C be a subset of k ISPs out of N

 Total cost of a selection of ISPs C: Weighted sum of

monetary, path length and path diversity costs

 Select combination C with minimum total cost

 Feasible to enumerate all combinations

6/7/2019 39
 Monetary and Path Length Cost
 For set of ISPs C, what is the monetary and path
length cost of routing egress flows?
 Find the minimum cost mapping G* of flows to ISPs
(Bin Packing)
 Flows = items
 ISPs = bins
 NP hard !
 Use First Fit Decreasing (FFD) heuristic
 Mapping G* very close to optimal
 Monetary and path length costs of C are calculated
using the mapping G*

6/7/2019 40
 Path Diversity
 selection C gives K paths to
each destination d
 K-shared link to d: link shared
by all K paths to d
 If a K-shared link fails,
destination d is unreachable
 Minimize the number of K-
shared links
 Path diversity metric: The number
of k-shared links to destination d
averaged over all destinations
 Gives the best resiliency to
single-link failures
6/7/2019 41
 Summary
 Algorithms for ISP selection
 Choosing best set of upstream ISPs
 Objectives are minimum monetary cost, short AS paths and
high path diversity

 ISP selection for monetary and performance

constraints
 Formulated as a bin-packing problem
 Heuristic gives solution very close to optimal

 ISP selection for path diversity

 Returns set of ISPs with best path diversity to the set of
major destinations
6/7/2019 42
 Outline

 Measuring the Evolution of the Internet Ecosystem

 The Edge of the Internet: ISP and Egress Path

Selection for Stub Networks

 The Core of the Internet: Peer and Provider

Selection for Transit Providers

 ISP Profitability and Network Neutrality

6/7/2019 43
 The debate
 Recent evolution trend: Large amounts of video and
peer-to-peer traffic
 Content providers (CP) generate the content
 Provide content and services “over the top” of the basic
connectivity provided by ISPs
 Profitable (think Google)
 Access Providers (AP) deliver content to users
 Recent trend: Not profitable
 Commoditization of basic Internet access
 Want a share of the pie
 Tension between AP and CPs: “Network neutrality”
6/7/2019 44
 A Technical View

 Previous work
 Mostly non-technical
 Highly emotional debates in the press
 Legislation/policy aspects: Do we need network neutrality
legislation?
 But what about the underlying problem: Non-
profitability of Access Providers?
 Our approach: A quantitative look at AP profitability
 Investigate reasons for non-profitability
 Evaluate strategies for remaining profitable

6/7/2019 45
 Modeling AP Profitability

 Three AS types: AP, CP

and transit provider (TP)
 Focus on the AP
 AS links
 customer-provider
(customer pays provider)
 peering (no payments)
 AP and CPs can transfer
traffic either through
customer-provider or
peering links

6/7/2019 46
 AP Profitability

 Reasons why APs can be unprofitable

 AP users
 The impact of video traffic
 AP strategies: Pricing
 Heavy hitter charging
 Heavy hitter blocking
 Non-neutral charging
 AP strategies: Connection
 Caching CP content
 Peering selectively with CPs

6/7/2019 47
 Major Findings

 Variability in AP users can cause large variability in

costs
 Video traffic: Increases costs for AP

 AP strategies based on differential/non-neutral

pricing may not succeed
 Have to account for user departure due to competition

 AP strategies based on connection are promising

 Caching content from CPs
 Peering selectively with large CPs
6/7/2019 48
 Contributions of this Thesis
 A measurement study of the evolution of the
Internet ecosystem

 Modeling the evolution of the Internet ecosystem

 “what-if” questions about possible evolution paths

 Optimizations at the edge of the Internet

 Algorithms for provider selection and egress routing

 A technical view of the network neutrality debate

 Strategies for ISP profitability
6/7/2019 49
 Future Directions

 Measurements: Investigate the evolution of the

connectivity for monitor ASes
 We can observe all links for such ASes
 Focus on transitions between peering and customer-provider
links

 Measurements: What does the interdomain traffic

matrix really look like?
 Can we use measurements from a large Tier-1 provider?
 Can we augment that data with information about the
interdomain topology?

6/7/2019 50
 Future Directions

 What is the best strategy for different types of

providers?
 Strategies for classes of providers
 Strategies for individual providers

 Do the distributed optimizations by networks solve a

centralized problem?
 E.g., minimizing path lengths

6/7/2019 51
 Other things I’ve been up to
 Router buffer sizing
 “Buffer Sizing For Congested Internet Links” [Infocom ‘05]
 “Open Issues in Router Buffer Sizing” [CCR ‘06]

 Network troubleshooting
 “NetDiagnoser: Troubleshooting network disruptions using
end-to-end probes and routing data” [CoNext ‘07]

 Network monitoring
 “Route monitoring from passive data plane measurements”
[In progress]
 Measurement
 “Poisson vs. Periodic Path Probing” [IMC ‘05]

 “Bootstrapping in Gnutella” [PAM ‘04]

6/7/2019 52
 Thank You !

6/7/2019 53
Issue-1: remove backup/transient links
 Each snapshot of the Internet topology captures 3
months
 40 snapshots – 10 years
 Perform “majority filtering” to remove backup and
transient links from topology
 For each snapshot, collect several “topology samples”
interspersed over a period of 3 weeks
 Consider an AS-path only if it appears in the majority of the
topology samples
 Otherwise, the AS-path includes links that were active for
less than 11 days (probably backup or transient links)

Samples

Snapshot
6/7/2019 54
6/7/2019
 Issue-2: variable set of BGP monitors
 Some observed link births may be links revealed due to increased
monitor set
 Similarly for observed link deaths
 We calculated error bounds for link births and deaths
 Relative error < 10% for CP links
 See paper for details

6/7/2019 55
Issue-3: visibility of ASes, Customer-Provider (CP) and
Peering (PP) links
 Number of ASes and CP links is robust to number of monitors
 But we cannot reliably estimate the number of PP links

6/7/2019 56
 Global Internet trends

6/7/2019 57
Transit (CP) vs Peering (PP) relations

 The fraction of peering links has been increasing steadily

 But remember: this is just a lower bound
 At least 20% of inter-AS links are of PP type today
6/7/2019 58
 The Internet gets larger but not
longer

 Average path length remains almost constant at 4 hops

 Average multihoming degree of providers increases faster than
that of stubs
 Densification at core much more important than at edges
6/7/2019 59
6/7/2019 60
 Regional distribution of AS types

 Europe is catching up with North America w.r.t the population of

ECs and LTPs
 CAHPs have always been more in Europe
 More STPS in Europe since 2002
6/7/2019 61
 Evolution of Internet transit:
the customer’s perspective

6/7/2019 62
 Customer activity by region

 Initially most active customers were in North America

 After 2004-05, customers in Europe have been more active
 Due to increased availability of providers?
 More competitive market?
6/7/2019 63
 How common is multihoming among AS
species?

 CAHPs have increased their multihoming degree significantly

 On the average, 8 providers for CAHPs today
 Multihoming degree of ECs has been almost constant (average < 2)
 Densification of the Internet occurs at the core
6/7/2019 64
 Who prefers large vs small transit
providers?

 After 2004, ECs prefer STPs than LTPs

 Mainly driven by lower prices or regional constraints?
 CAHPs connect to LTPs and STPs with same probability
6/7/2019 65
 Customer activity by region

 Initially most active customers were in North America

 After 2004-05, customers in Europe have been more active
 Due to increased availability of providers?
 More competitive market?
6/7/2019 66
 Evolution of Internet transit:
the provider’s perspective

6/7/2019 67
Attractiveness (repulsiveness) of transit providers

 Attractiveness of provider X: fraction of new CP links that connect to X

 Repulsiveness, defined similarly
 Both metrics some positive correlation with customer degree
 Preferential attachment and preferential detachment of rewired links
6/7/2019 68
 Evolution of attractors and
repellers

 A few providers (50-60) account for 50% of total

attractiveness (attractors)
 The total number of attractors and repellers increases
 The Internet is NOT heading towards oligopoly of few large players
 LTPs dominate set of attractors and repellers
 CAHPs are increasingly present however
6/7/2019 69
 Correlation of attractiveness and
repulsiveness

 Timeseries of attractiveness and repulsiveness for each provider

 Calculate cross-correlation at different lags
 Most significant correlation values at lags 1,2 and 3
 Attractiveness precedes repulsiveness by 3-9 months
6/7/2019 70
 Evolution of Internet peering
(conjectures)

6/7/2019 71
 Evolution of Internet Peering

 ECs and STPs have low peering frequency

 Aggressive peering by CAHPs after 2003
 Open peering policies to reduce transit costs

6/7/2019 72
 Which AS pairs like to peer?

 Peering by CAHPs has increased significantly

 CAHPs try to get close to sources/destinations of content
 Peering by LTPs has remained almost constant (or declined)
 “Restrictive” peering by LTPs
6/7/2019 73
 Conclusions
Where is the Internet heading towards?

 Initial exponential growth up to mid-2001, followed

by linear growth phase
 Average path length practically constant
 Rewiring more important than growth
 Need to classify ASes according to business type
 ECs contribute most of the overall growth
 Increasing multihoming degree for STPs, LTPs and
CAHPs
 Densification at core
 CAHPs are most active in terms of rewiring, while ECs
are least active
6/7/2019 74
6/7/2019
 Conclusions
Where does the Internet head toward?

 Positive correlations between attractiveness &

repulsiveness of provider and its customer degree
 Strong attractiveness precedes strong repulsiveness
by period of 3-9 months
 Number of attractors and repellers between shows
increasing trend
 The Internet market will soon be larger in Europe
than in North America
 In terms of number of transit providers and CAHPs
 Providers from Europe increasingly feature in the set
of attractors and repellers
6/7/2019 75
 Multihoming
 Multihoming: Connection of a
stub network to multiple ISPs
 x% of stub networks are
multihomed
 Redundancy
 primary/backup relationships
 Load Balancing
 Distribute outgoing traffic among
ISPs
 Cost Effectiveness
 Lower cost ISP for bulk traffic,
higher cost ISP for performance-
sensitive traffic
 Performance
 Intelligent Route Control
6/7/2019 76
 ISP selection
 Should consider both monetary cost and
performance
 Minimum monetary cost
 Estimate the cost that “would be” incurred if a set of
ISPs was selected
 Minimum AS path lengths
 Longer paths: delays, interdomain routing failures
 Measure AS path length offline using Looking Glass
Servers
 Maximum Path diversity
 AS-level paths to destinations should be as “different”
as possible

6/7/2019 77
 Problem Definition
 Two phases
 Phase I – ISP Selection:
 Select K upstream ISPs
 K depends on monetary and performance constraints
 “Static” operation
 Change only when major changes in the traffic destinations or
ISP pricing
 Phase II – Egress Path Selection
 Allocate egress traffic to selected ISPs
 Avoid long term congestion and minimize cost
 “Semi-static” operation, performed every few hours or days

6/7/2019 78
 Evaluation – Path Diversity
 AS-level paths and traffic
rates are input to simulator
 9 ISPs, 250 destinations
 Given K, find the selection
C* with the minimum path
diversity cost
 For each selection C, find
u(C) = total traffic lost due
to the failure of each link in
topology
 Calculate Δu(C) = u(C) – u(C*)  Single link failures:
for each selection C C* is the optimal
selection
6/7/2019 79
 Egress Path Selection
 After Phase-I, S has K upstream ISPs
 Problem: How to map outgoing traffic to the ISPs
 M flows: KM mappings of flows to ISPs
 Some mappings may cause congestion to flows !
 Flows can be congested at access links or further upstream
 Objective: Find the loss-free mapping with the
minimum cost
 Challenges:
 Upstream topology and capacities are unknown
 Iterative routing approaches required
 Propose an iterative routing based on simulated annealing

6/7/2019 80
 Evaluation – Path Diversity
 AS-level paths and traffic
rates are input to simulator
 9 ISPs, 250 destinations
 Given K, find the selection
C* with the minimum path
diversity cost
 For each selection C, find
u(C) = total traffic lost due
to the failure of each link in
topology
 Calculate Δu(C) = u(C) – u(C*) 
 Single
2,3 linklink failures:
failures:
for each selection C C*
C* is
is the
closeoptimal
to the
selection
optimal selection
6/7/2019 81
 Provider and Peer Selection

 Detailed model for provider and peer selection

 Complex real-world decisions
 Provider selection objectives
 Monetary cost
 AS path lengths
 Peer selection
 Minimize transit costs
 Maintain reachability
 Constraints
 Only local knowledge
 Geographical constraints
6/7/2019 82
 Peering Federation

A B C

 Traditional peering links: Not transitive

 Peering federation of A, B, C: Allows mutual transit
 Longer chain of “free” traffic
 Incentives to join peering federation?
 What happens to tier-1 providers if smaller providers form
federations?
6/7/2019 83
 Why can the AP be unprofitable?
 Variability of users =>
high variability in the
costs incurred by AP
 Variability increases
with the access speed
 Increase in video
traffic: higher transit
payment by AP

6/7/2019 84
 Baseline model
 AP and CP connect to the TP as customers
 N users of AP, charged a flat rate R ($/month)
 Transit pricing: 95th percentile of traffic volume,
concave transit pricing functions
 95th / mean = 2:1 for normal traffic, 4:1 for video1
 More video means higher transit payment by AP
 AP users: Heavy tailed distribution of content
downloaded per month
 High variability in AP costs

1Norton’06: Internet Video: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem

6/7/2019 85
 AP Strategies
 Charging strategies
 AP charges “heavy hitters”
according to volume
downloaded
 AP caps heavy hitters
 AP charges CP (non-network
neutral)
 Charging strategies are
disruptive
 AP cannot control customer
departure probability

6/7/2019 86
 AP Strategies

 Charging heavy hitters

 download amount D,
threshold T, flat rate R
 c(D) = D*R/T
 AP’s profit is sensitive to
customer departure prob
 Capping heavy hitters
and non-neutral charging
would not work for the
same reason

6/7/2019 87
 AP Strategies

 Connection Strategies
 AP caches content from CPs
 AP peers with CPs
 Non-disruptive
 Caching can reduce transit costs of AP
 But depends on the amount of content cacheable
 Selective peering with CPs can improve profitability
 Peering cost depends on CP
 Cost/benefit analysis for each CP
 CP with large network: low cost of peering

6/7/2019 88
 AP Strategies
 Connection Strategies
 AP caches content from CPs
 AP peers with CPs
 Non-disruptive
 Cost-benefit analysis for
peering
 Peering cost depends on CP
(easy/medium/hard)
 r = saving/cost (both
estimated)
 Peer if r > R
 AP controls the factor R
6/7/2019 89
 AP Strategies
 Charging heavy hitters
 download amount D,
threshold T, flat rate R
 c(D) = D*R/T
 AP’s profit is sensitive to
customer departure prob
 Non-neutral charging
 Customer departure prob
 “How discriminatory is my
AP?”
 AP’s profit is sensitive to
customer departure prob
6/7/2019 90
 Why Study Internet Evolution?

 “Bottom-up” models (more later)

 Understand how local actions lead to emerging properties

 Performance of protocols over time

 “How would BGP perform 10 years from now?”

 Clean slate vs. evolutionary design

 After initial design, both must evolve !
 Understanding evolution of the current Internet can help
design

6/7/2019 91