Performance Workshop

Alcatel-Lucent Chicago July 18, 2007

All Rights Reserved Alcatel-Lucent 2007

Agenda

1. Introductions (All) 2. RAMSES (Benjamin Williams) 3. Systems Engineering: Performance Analysis (Paul Boudreaux) 4. Voice Quality (Falguni Sarkar) 5. Q & A (All)

All Rights Reserved Alcatel-Lucent 2007

RAMSES

Alcatel-Lucent July 16, 2007

All Rights Reserved Alcatel-Lucent 2007

Agenda

1. KPI Team Members/Roles 2. Typical RAMSES KPI Reporting 3. RAMSES Tool Overview 4. Specific examples from RAMSES KPI Reports

All Rights Reserved Alcatel-Lucent 2007

KPI Team Members / Roles

All Rights Reserved Alcatel-Lucent 2007

KPI Team Members/Roles

KPI Team
- Ben Williams - Imron Hussaini - Vrushali Nidgundi - Anand Sane - Francis B. Smith Team Leader, Central Region Prime, XML Prime Northeast Region Prime, GSM/GeoProbe co-Prime South Region Prime, GSM/GeoProbe co-Prime UMA Prime West Region Prime

All Rights Reserved Alcatel-Lucent 2007

Typical RAMSES KPI Reporting

All Rights Reserved Alcatel-Lucent 2007

Typical RAMSES KPI Reporting Weekly reports  GSM Weekly Report (reviewed 2nd and 4th Tuesdays of each month via teleconference)
 Covers all (~50) TMO Atriums providing GSM service (i.e. GMSCs, DMSCs: GSM-only + Phase 2 UMA)  Summarizes TCBH (Time Consistent Busy Hour) for M-F of previous week  MS Word format and includes:
Dashboard Summary, Active Subscribers, Traffic/Events per Subscriber Traffic Profile, BHCA, CPU Utilization GOS for Calls, SMS, CAMEL, Paging & Auth., Loc. Update, and HandOver Trunk Group Report, Call Blocking Report, SS7 Utilization Report (STP and BSC) Separate Weekly Trend Report for the above GOS Trends

All Rights Reserved Alcatel-Lucent 2007

Typical RAMSES KPI Reporting (cont.)

Weekly Reports (cont.)  UMA Weekly Report (reviewed each Monday via teleconference)
 Covers all (12) TMO Atriums providing UMA service (i.e. Phase 1 UMA + Phase 2 UMA DMSCs)  Summarizes TCBH (Time Consistent Busy Hour) for Su-Sa of previous week  MS Word format and includes:
Active Subscribers, Traffic/Events per Subscriber Traffic Profile, BHCA, CPU Utilization GOS for Calls, SMS, CAMEL, Paging & Auth., Loc. Update, and HandOver Trunk Group Report, Call Blocking Report, SS7 Utilization Report (STP and BSC) UMA Discovery & Registration, UMA Paging, LDAP Queries KPIs for other UMA Nodes (SGW, GGW, GMLC, SMLC, etc.)

Daily Reports (as needed)

 FOAs, Switch launches, UMA service launch, etc.  Typically MS Word format, though sometimes MS Excel  Typically single switch per report  Typically shows all (15-minute) samples for the day in line graph form for each GOS indicator

All Rights Reserved Alcatel-Lucent 2007

RAMSES Tool Overview

All Rights Reserved Alcatel-Lucent 2007

RAMSES Tool Overview

RAMSES is an ALU internal tool which is composed of three main parts: - RAMSES Data Collector Cycles through all supported switches, FTPing all .tar files in the kpi directory which are not already on the RAMSES server - RAMSES database A flat-file database containing the post-processed contents from the .tar files collected from each switch since collection started for that switch (back to late 2005 for some switches) - RAMSES Report Generator GUI
- Enables customized (.rtf, .xls, or .html) reports for one or more switches (selected at run time through the RAMSES GUI) - Reports are defined in XML and are structured as combinations of individual charts (histograms, tables, line graphs, etc.) based on KPI definitions - KPIs are defined in XML and are based on the Atrium performance counters - Ad-hoc charts can be generated from individual KPIs or counters as needed

All Rights Reserved Alcatel-Lucent 2007

Specific examples from RAMSES KPI Reports

All Rights Reserved Alcatel-Lucent 2007

Specific examples from RAMSES KPI Reports

Example #1: Histogram chart Active Subscribers (all GSM DMSCs)

All Rights Reserved Alcatel-Lucent 2007

Specific examples from RAMSES KPI Reports (cont.)

Example #2: Table chart Grade of Service UMA Paging (for all UMA switches)

Ne

UNC Paging Attempts

UMA Paging Success Rate 93.9 % 94.9 % 85.0 % 94.7 % 97.8 % 95.9 %

TCBH

ATMSS996_UNC CHMSS966_UNC CSMSS949_UNC DAMSS987 DEMSS931 DNMSS935

84.7 evt/h 78.6 evt/h 162 evt/h 64.9 evt/h 59.6 evt/h 41.4 evt/h

[21-22] [17-18] [12-13] [17-18] [17-18] [17-18]

All Rights Reserved Alcatel-Lucent 2007

...

Specific examples from RAMSES KPI Reports (cont.)

Example #3: Line chart Loc. Update Attempts vs. Failure Rate for 12-Jul-2007 at newly launched CLMSS981

All Rights Reserved Alcatel-Lucent 2007

Performance Analysis
Analysis Of Radio Issues On BSCs Re-Homed To Atrium MSC

Xavier Ligeret (xavier.ligeret@alcatel-lucent.com) T-mobile Systems Engineering North America RSC July 10, 2007

All Rights Reserved Alcatel-Lucent 2007

Description Of The Issues

Known SRs And Symptoms Known SRs and DCR/RDS: - SDCCH drop counter, T3260 Timer - TCH poor performance (SR 200610002564) - Increase SDCCH drop after BSC rehome (SR 200704003534) - Increase paging failure after BSC rehome (SR 200704003538) - TCH spikes on NEBSC01 and NEBSC97 (SR 200702014483) - Ciphering timer configurable from the WEM (DCR 173/RDS 6917) Impacted networks = All networks with BSCs re-homed to Atrium MSC Symptoms: - Reported increase of call drops, paging failures - Statistics suggest poor performance (on BSC counters) on re-homed BSCs compared with statistics on BSCs connected to legacy MSCs; though the call drops statistics dont exceed 3%

All Rights Reserved Alcatel-Lucent 2007

Presentation Of The Issues

Observations Made By TMO (Dinko Dinkov and Frankie Chen)

Issues seem to appear mostly at the borders of areas covered by the re-homed BSCs, with higher probability of RF issues (lower signal strength) Atrium MSC triggers more requests towards the MS compared with legacy MSCs (more Location Updates and more ciphering), thus:  More signaling on the Radio  The probability where the network tries to contact the MS, while it is in a poor radio coverage, is higher with Atrium, and eventually the risk of call drop increases MSC implementation differences found between legacy MSCs and Atrium MSC:  Different handling of DTAP signaling messages during Assignment Procedure  Higher Ciphering frequency (Atrium)  Higher TMSI-reallocation frequency

All Rights Reserved Alcatel-Lucent 2007

Analysis

We first checked that the Atrium MSC was 3GPP compliant, regarding:  The frequencies of the Ciphering and TMSI reallocation procedures, though Atrium follows the standards in a very strict way  The call flow and the global design to handle scenarios, though bugs may still exist Re-homing BSCs creates islands of small Location Areas managed by the Atrium MSC, surrounded by legacy MSCs, and increases Signaling messages The system seen as a whole (from the MSC to the MS) evolves in some extreme situations:  Limit of the radio coverage  With the MS probably often in unstable states (in search for radio signal, )  With message race situations  Where 3GPP can be interpreted in different ways This is NOT:  A load or dimensioning issue

All Rights Reserved Alcatel-Lucent 2007

Analysis
Summary More Signaling on the Radio increases the probability that the Network tries to reach the MS, while it is in a zone of bad radio coverage, and increases the chances to drop the MS

Existing RF issues

+ +

More signaling on the Radio

Reported more connection failures (according to counters)

Some implementation differences in the MSC have direct impacts on the BSC counters

Existing RF issues

A different MSC implementation

Poor statistics on the Network Access

All Rights Reserved Alcatel-Lucent 2007

Conclusion

The degradation of the statistics on re-homed BSCs: Can only be mitigated, but not totally fixed Shows that there are radio issues, that were partly masked while using the legacy MSCs because of the way the BSCs pegged their counters ALU will: Help TMO to fine tune the parameters to improve the statistics Fix the bugs found, if any Address any change request in the MSC implementation only as an ultimate solution There are clearly radio issues that need further investigations as well

All Rights Reserved Alcatel-Lucent 2007

Backup
Detailed Analysis

All Rights Reserved Alcatel-Lucent 2007

Analysis Of The TCH Issue

Example Of MSC Implementation Impacting BSC Counters (1/2): Ericssons MSC
Implementation (3GPP compliant): During the Assignment Procedure: - The MSC suspends the transmission of any CC or MM signaling messages - Received messages are queued or discarded (we dont know exactly yet) At the end, if the Assignment Procedure is successful: - The TCH setup is supposedly counted as successful by the BSC - If the Disconnect is processed by the MSC after the Assignment Procedure, it would supposedly be counted as a normal Disconnect Main Benefit: Better statistics on the BSC if the Assignment Procedure is successful Main Drawback(s): - If the MSC processes the Disconnect: waste of Radio resources (TCH immediately released after setup) - If the MSC has dropped the Disconnect: it would be a 3GPP violation

All Rights Reserved Alcatel-Lucent 2007

Analysis Of The TCH Issue

Example Of MSC Implementation Impacting BSC Counters (2/2): Atrium MSC
Implementation (3GPP compliant): - During the Assignment Procedure, the MSC responds to any signaling messages from the MS. - The MSC lets the BSC suspend the transmission of those messages (otherwise the BSC would violate 3GPP) At the end, the scenario leads to a Call Disconnection before the completion of the Assignment Procedure Main Drawback: TCH setup is not completed, the BSC supposedly counts it as a failure Main Benefit: Radio resource saved, since no TCH setup

Conclusion: At the end, both Ericsson and Atrium implementations lead ultimately to the Disconnection of the Call, but with different impacts on the BSC counters

All Rights Reserved Alcatel-Lucent 2007

T-Mobile/ALU Members Recommendations to fine tune the parameters to adjust to the BSCs and the TMO traffic and support A dedicated team is working on all these issues since 5/21/07 with a weekly or bi-weekly meeting:  On TMOs side:
 TMO coordinator: Pradeep Singh  Tier2 RAN Ericsson: Dinko Dinkov  Tier2 Atrium: Michele Kinsey

 On ALUs side:
 RSC POC: David McNemar and Sean Simmons  RSC System Engineering: Xavier Ligeret  War Room POC: Jim Morris

 Other participants:
 TMO: R. K. Kumar, Munar Le Roy, Homer Filart,  ALU: Brett Schwarz, Robbie Espiritu,

All Rights Reserved Alcatel-Lucent 2007

Atrium Voice Quality Verification T-Mobile USA Norfolk/DC Field Test

Voice and Data Quality/Wireless Performance and RF Core Support
Ping Yu, Wonho Yang, Ahmed Tarraf

July 13, 2007

All Rights Reserved Alcatel-Lucent 2007

Contents
Voice Quality Test Scenarios Voice Quality Test Setup Voice Quality Test Results Executive Summary Next Steps

All Rights Reserved Alcatel-Lucent 2007

Purpose
The Voice and Data Quality and Performance team performed Voice Quality field verification for the GSM/Atrium system in the T-Mobile market from June 11th to June 14th 2007. Field verification included: Voice quality for GSM Mobile-to-Land, Land-toMobile and Mobile-to-Mobile calls Echo Test for Mobile-to-Land calls

All Rights Reserved Alcatel-Lucent 2007

Voice Quality Test Scenarios

Stationary Tests
 Voice Quality Test: Land-to-Mobile Mobile-to-Land Mobile-to-Mobile  Echo Test (ESM1 card) Mobile-to-Land

Drive Tests
 Voice Quality Test: Land-to-Mobile Mobile-to-Mobile

Echo Test (ESM1 card)

Mobile-to-Land
All Rights Reserved Alcatel-Lucent 2007

Voice Quality Measurement Setup

Mobile-Mobile
Atrium

Land-Mobile
Atrium

Signaling link

Signaling link

MGW

MGW

MGW

BSC

BSC

BSC RF Tool Kit

BTS

BTS

PSTN Switch

BTS Mobile

Player

Mobile

Mobile

ROMA

Player

Analog Phone

Mobile

ROMA

Speech Path

Speech Path

All Rights Reserved Alcatel-Lucent 2007

Voice Quality Measurement Tool

 Voice Quality was measured using Alcatel-Lucent Real Time Objective MOS Analyzer (ROMA)  ROMA (Real Time Objective MOS Analyzer)
 real-time intrusive tool developed by ALU  based on ITU-T PESQ and on ALU expertise on objective voice quality evaluation since 1993  is highly correlated with Subjective MOS (~0.93)  runs on a laptop and interfaces with the Mobile via regular sound card and the 2.5mm audio jack at the mobile  performs automatic detection / segmentation of talk spurts

All Rights Reserved Alcatel-Lucent 2007

MOS = Mean Opinion Score

 ITU-T P.800

Rating Speech Quality

5 4 3 2 1 Excellent Good Fair Poor Bad

Listening Effort
Complete Relaxation possible Attention necessary, little effort Moderate effort required Considerable effort required No meaning understood

All Rights Reserved Alcatel-Lucent 2007

Voice Quality Test Setup

Stationary Tests
 Originating mobile, terminating mobile and Land line phone were at the same location (Switch room location in Norfolk, Virginia)- good RF location  BSC 26 was used, ESM1 card used  Land-to-Mobile:
Speech injected at Land line (using audio interface box) Speech recorded at Nokia 6010 mobile (using audio interface box)

 Mobile-to-Land
Speech injected at Motorola V330 Speech recorded at land line phone

 Mobile-to-Mobile:
Speech injected at Motorola phone Speech recorded at Nokia phone
All Rights Reserved Alcatel-Lucent 2007

Voice Quality Test Setup (cont.)

Drive Tests
 Originating mobile was at van  Drive routes included I464 and 64, experienced both intra BSC and inter BSC handover (BSCs 23, 26, 28)  ESM1 card used  Land-to-Mobile:
Speech injected at Land line (using audio interface box) Speech recorded at Nokia 6010 mobile (using audio interface box)

 Mobile-to-Land
Live talk test using MotorolaV330 phone and subjective echo at the same mobile was carefully tested.

ROMA (Realtime Objective MOS Analyzer), developed by VDQPAG, is used as the EMOS/VQ measurement tool

All Rights Reserved Alcatel-Lucent 2007

Voice Quality Test Results: MOS

Stationary Tests Land-to-Mobile: Call 1 (duration 6+ minutes) - 49 MOS samples

Statistics Mean Median Max Min Standard Dev

VQ is good

MOS 3.47 3.46 3.64 2.82 0.14

 Constant background noise impairment of about 0.2 MOS attributable to mobile audio interface and landline devices.
Background noise impact was a constant during the testing
Background Noise Sample
All Rights Reserved Alcatel-Lucent 2007

Executive Summary
 ALU Voice and Data Quality and Performance team performed a Voice Quality evaluation on the Atrium Call Server in Norfolk  Atrium Voice Quality was determined good as measured by EMOS
Stationary Atrium Testing Land to Mobile Expected MOS for Good Results 3.4 Measured MOS (Mean) 3.46

Mobile to Mobile Mobile to Land

3.1 3.4

3.25 3.54

 Expected MOS for Good Results based on typical good voice quality score of wireless network regardless of the technology

All Rights Reserved Alcatel-Lucent 2007

Executive Summary
 Observed VQ degradations:
 Constant background noise impairment of about 0.2 MOS attributable to mobile audio interface and landline devices.  Infrequent BER audio degradation which is consistent with RF impairments

 A sample drive test provided Land to Mobile MOS of 3.21  Next Steps
 Run VQ/MOS tests on ESM1 and ESM4 cards in T-M Lab environment.  Run Echo tests on ESM1 and ESM4 cards in T-M Lab environment.

All Rights Reserved Alcatel-Lucent 2007

Performance Workshop Q&A

Alcatel-Lucent July 16, 2007

All Rights Reserved Alcatel-Lucent 2007

www.alcatel-lucent.com

All Rights Reserved Alcatel-Lucent 2007