Performance
Sprint ATL
May 2002
Introduction
• Tier-1 ISPs interested in providing Voice-
Over-IP (VoIP)
• Need to provide quality
– Voice quality and availability
• Possible causes of degradation
– Congestion (what is this?)
– Link failures (what is this?)
– Routing instabilities (what is this?)
• Goal of this work is to study the frequency of
these events (at Sprint) and assess their
impact on VoIP performance
Introduction
• Use passive monitoring for congestion
– Assess loss plus delay
– Can’t get routing information
• Use active measurement
– on two well-connected locations
– Across one IS-IS boundary
• “We find …”
– Sprint IP backbone ready for toll-quality VoIP
• Congestion effect is negligible
– Link failures impact availability
• Cause routing instability for 10s of minutes
Outline
• Introduction (done)
• Related Work (next)
• Measurements
• Voice Call Rating
• Results
• Conclusion
Related Work
• Introduction (done)
• Related Work (done)
• Measurements (next)
• Voice Call Rating
• Results
• Conclusion
Measurement
• Passive
– Via Sprint infrastructure
• Active
– Induce own data
Passive Measurements
• Sprint has a passive measurement architecture
– traces on more than 30 links in POPs
– Includes 44 byte IP packet and timestamp via GPS
reference signal
• Use traces from OC-12 (622 Mbps)
– Jul 24th, 2001; Sep 5th, 2001; Nov 8th, 2001
– Compute delays across backbone
• But
– Can’t get loss since leave out non-monitored links
– Can’t control traffic source
Active Measurements
• Introduction (done)
• Related Work (done)
• Measurements (done)
• Voice Call Rating (next)
• Results
• Conclusion
Voice Call Rating – The E-model
• Combine loss and delay into single rating
• Use to compute Mean Opinion Score (MOS)
– ITU recommendation
• Below 60 unacceptable
• Above 70 is “toll” quality
• Above 90 is excellent
The E-Model
R = R0 – Is – Id – Ie + A
• R0 is effects of noise
• Is is impairments in signal (quantization)
• Id is impairment from mount-to-ear delay
• Ie is impairment from distortion (loss)
• A is advantage factor (tolerance)
– Different for different systems
– Example: wireless is a “10”
– Since not agreed upon, drop further
• (Ok, but how does it map to transport layer?)
The E-model at the Transport Layer
R = 94.2 – Id – Ie
The E-model at the Transport Layer
• Id includes expression encompassing entire
telephone system
• Simplify
– All delays collapse into one: mouth-to-ear
– Use defaults [4] for all save for IP network delay
Id = 0.024d + 0.11(d-177.3)H(d-177.3)
• d is mouth-to-ear delay
– Encoding (packetization)
– Network (transmission, propagation and queuing)
– Playout (buffering)
• H() is “heavyside” function
– H(x) = 0 if x < 0
– H(x) = 1 if x > 0
The E-model at the Transport Layer
• No analytic model for Ie (impairment)
– Must use subjective measurements
– Appendix includes samples for different
encodings
• Focus on G.711 (uses concealment)
• Summary R-factor:
R = 94.2 - 0.11(d-177.3)H(d-177.3) –
- 0.024d - 30 * ln(1 + 15 * e)
• Introduction (done)
• Related Work (done)
• Measurements (done)
• Voice Call Rating (done)
• Results (next)
– Delay
– Failures
– Voice Quality
• Conclusion
Passive Delay Measurements
Mean 28.5ms
Variation 200 sec
Almost speed of fiber so
almost no queuing
Active Delay Measurements
Min is 30.95 ms
Avg is 31.38 ms
99.9% under 32.85 ms
Same as active
• Introduction (done)
• Related Work (done)
• Measurements (done)
• Voice Call Rating (done)
• Results (next)
– Delay (done)
– Failures (next)
– Voice Quality
• Conclusion
Impact of Failures on Data Traffic
Route changes
Loss from Route Changes
Packet Sequence Numbers during Route
Changes
No out of order
Indicates from route change
Routers involved in Failure
Solid is primary
Dashed is backup
R4 has problems
Router Messages
• Recommendations
– Not from IS to IS protocol (so MPLS would not
help)
– Engineers should work on improving reliability
of hardware and software
Outline
• Introduction (done)
• Related Work (done)
• Measurements (done)
• Voice Call Rating (done)
• Results (next)
– Delay (done)
– Failures (done)
– Voice Quality (next)
• Conclusion
Voice Quality (Does not
include failure)
• More experiments
– Want overall likelihood of link failure
• Compare network availability with telephone
availability
– FCC defines standards that affect 90k lines for
more than 30 minutes
– Difficult to define for IP since no “lines”,
customer count tough, and outage could be
from non-network (ie- DNS) cause