LTE Optimization Principles

[RL60]
Module 02

KPI Overview

1 © Nokia 2014 - RA47042EN60GLA0

Module Objectives

• After completing this module, you will be able to:

• Distinguish different classes of KPIs: network and e2e
• Name the documentation for detailed definitions of NSN KPIs
• Explain how to derive network KPIs based on performance counters
• Explain the measurement of e2e KPIs

3 © Nokia 2014 - RA47042EN60GLA0

Index

• NSN counter-based KPIs

• NSN field test KPIs
• Field KPIs – testing methods

4 © Nokia 2014 - RA47042EN60GLA0

Air interface KPIs LTE
• KPIs are categorized to distinguish different performance aspects.
• The names indicate the respective category according to 3GPP32.410 and 3GPP32.450.
Integrity:
Average CQI
Accessibility: Average Latency
RRC Setup Success Rate RLC PDU
E-RAB Setup Success Retransmission
Usage:
Rate
eNodeB Cell Availability eNodeB2
S1 Setup Success
1 Resource Block
Call Setup Success Rate
Usage
PDCP/RLC Layer
Throughput
Inter RAT
X redirection
UE X2

X
Retainability:
RRC Drop Rate Mobility:
E-RAB Drop Rate Intra eNodeB HO
Success Rate
Inter eNodeB HO
5 © Nokia 2014 - RA47042EN60GLA0
Success Rate
 General Measurement Overview (RL60)

Radio network measurements • LTE Handover (M8021)

• S1AP (M8000) • LTE X2AP (M8022)
• Cell load (M8001) • LTE UE and Service Differentiation (M8023)
• Transport load (M8004) • LTE Handover to eHRPD per eHRPD bandclass
• Power and Quality UL (M8005) measurement (M8025)
• EPS bearer (M8006) • LTE QoS measurement (M8026)
• Radio Bearer (M8007) Transmission measurements
• RRC (M8008) • IP Statistics (M51120)
• Intra eNodeB handover (M8009) • PHB Statistics (M51121)
• Power and Quality DL (M8010) • Ethernet Link (M51123)
• Cell resource (M8011) • IP Sec (M51125)
• Cell throughput (M8012) • IP Filtering (M51126)
• UE state (M8013) • VLAN IP Statistics (M51127)
• Inter eNodeB handover (M8014) • VLAN PHB Statistics (M51128)
• Neighbor cell related handover (M8015) • LTE VLAN Statistics Measurement (M51129)
• LTE Inter System Handover (M8016) • Ethernet Interface Statistics (M51130)
• LTE Inter System Handover to UTRAN per Neighbor • LTE L2 Switch Statistics (M51131)
Cell (M8017) • LTE TWAMP Statistics (M51132)
• eNodeB load (M8018) • LTE TAC Statistics (51136)
• LTE Inter System Handover to GSM per Neighbor • LTE TOP FreqSync Statistics (51137)
Cell measurement (M8019) • LTE TOP PhaseSync Statistics measurement
• Cell availability (M8020) (M51138)
6 © Nokia 2014 - RA47042EN60GLA0
Accessibility

E-UTRAN E-RAB Setup Success Ratio LTE_5017a

The KPI describes the setup success ratio of the elementary E-RAB setup
procedure used to setup the E-RAB between MME and UE.
It indicates the E-UTRAN contribution to network accessibility for the end-
user, not the whole end-to-end service accessibility.

Summarisation
Logical formula formula (PI ID) Summarisation formula (Abbreviation)
E-RAB SSR=(E-RAB 100*sum([M8006C1]) / 100*sum([EPS_BEARER_SETUP_COMPLETIONS])
setup successes / E- sum([M8006C0]) / sum([EPS_BEARER_SETUP_ATTEMPTS])
RAB setup
attempts)*100%

7 © Nokia 2014 - RA47042EN60GLA0

Accessibility – related counter measurement

E-UTRAN E-RAB Setup Success Ratio

E-RAB SSR=(E-RAB 100*sum([M8006C1]) / 100*sum([EPS_BEARER_SETUP_COMPLETIONS])
setup successes / E- sum([M8006C0]) / sum([EPS_BEARER_SETUP_ATTEMPTS])
RAB setup
attempts)*100%

8 © Nokia 2014 - RA47042EN60GLA0

 Retainability

E-RAB Normal Release Ratio, User perspective LTE_5023d

This KPI describes the ratio of normally released E-RABs from user
perspective. This KPI is corresponding to a Connection Completion Ratio.

Summarisation
Logical formula formula (PI ID) Summarisation formula (Abbreviation)
E-RAB NRR 100*sum([M8006C6]+[ 100*sum([EPC_EPS_BEARER_REL_REQ_NORM]
UP=(normal E-RAB M8006C7]+[M8006C15] +[EPC_EPS_BEARER_REL_REQ_DETACH]
releases user + [M8006C10] +
perspective / all E-RAB [M8006C174] + +[ENB_EPS_BEAR_REL_REQ_RNL_REDIR
releases)*100% [M8006C175]) / ]+[ENB_EPS_BEARER_REL_REQ_NORM]
sum([M8006C6]+[M800
6C7]+[M8006C8]+ + [PRE_EMPT_GBR_BEARER] +
[M8006C9] + [PRE_EMPT_NON_GBR_BEARER]) /
[M8006C15]+ sum([EPC_EPS_BEARER_REL_REQ_NORM]
[M8006C10] + +[EPC_EPS_BEARER_REL_REQ_DETACH]
[M8006C12] +[EPC_EPS_BEARER_REL_REQ_RNL]
+[M8006C14] +[EPC_EPS_BEARER_REL_REQ_OTH]
+[M8006C13] + +[ENB_EPSBEAR_REL_REQ_RNL_REDIR]
[M8006C174] + +[ENB_EPS_BEARER_REL_REQ_NORM]
[M8006C175]) +[ENB_EPS_BEARER_REL_REQ_RNL]
+[ENB_EPS_BEARER_REL_REQ_TNL]
+[PRE_EMPT_GBR_BEARER]
+ [PRE_EMPT_NON_GBR_BEARER])
9 © Nokia 2014 - RA47042EN60GLA0
Mobility

E-UTRAN HO Success Ratio, intra eNB LTE_5035a

This KPI describes the success ratio for the handover execution phase,
when the source eNB receives information that the UE successfully is
connected to the target cell within own eNB.

Summarisation Summarisation formula

Logical formula formula (PI ID) (Abbreviation)
Intra HO SR =(number of successful 100*sum([M8009C7]) 100*sum([SUCC_INTRA_ENB_HO]) /
intra eNB HOs) / / sum([M8009C6]) sum([ATT_INTRA_ENB_HO])
(number of intra eNB HO
attempts)*100%

10 © Nokia 2014 - RA47042EN60GLA0

Mobility

E-UTRAN HO Success Ratio, inter eNB X2 based LTE_5048b

This KPI describes the success ratio for the inter eNB X2 based handover
execution phase, when the source eNB receives information that the UE
successfully is connected to the target cell within target eNB.

Summarisation Summarisation formula

Logical formula formula (PI ID) (Abbreviation)
Inter X2 based HO SR =(number of 100*sum([M8014C7]) 100*sum([SUCC_INTER_ENB_HO]) /
successful inter eNB X2 based HOs) / / sum([M8014C6]) sum ([ATT_INTER_ENB_HO])
(number of inter eNB X2 based HO
attempts)*100%

11 © Nokia 2014 - RA47042EN60GLA0

 Top level KPIs RL60

Contains most important KPIs for air interface.

Category Key Performance Indicator KP ID
Accessibility Initial E-RAB Accessibility LTE_5060f
Total E-UTRAN RRC Connection Setup Success Ratio LTE_5218d
Initial E-RAB Setup Success Ratio LTE_5112b
Retainability E-RAB Drop Ratio, RAN View (pre-emptions excluded) LTE_5025d
E-RAB Retainability Rate, RAN View, RNL Failure with UE Lost LTE_5581b
Integrity/Quality IP Scheduled Throughput in DL LTE_5350a – 5358a
IP Scheduled Throughput in UL LTE_5359a – 5367a
PDCP SDU Delay in DL LTE_5471a-5479a
Average Latency Uplink LTE_5137a
Mobility Total HO Success Ratio, intra eNB LTE_5043a
Total HO Success Ratio, inter eNB X2 based LTE_5058b
Total HO Success Ratio, inter eNB S1 based LTE_5084a
Usage Average PDCP Layer Active Cell Throughput DL LTE_5292d
Average PDCP Layer Active Cell Throughput UL LTE_5289d
Average Active UEs with data in the buffer DL LTE_5800d
Average Active UEs with data in the buffer UL LTE_5801d
12 Cell Availability
© Nokia 2014 - RA47042EN60GLA0 Ratio LTE_5750a
New KPIs for RL60

• RRC Connection Setup Attempts (LTE_5380a, LTE_5381a)

• RRC Connection Setup Success Ratio per Establishment Cause (LTE_5382a – LTE_5386a)
• E-UTRAN RRC Connection Setup Failure Ratio per Cause (LTE_5590a)
• RRC Connection Setup Rejection Ratio per Establishment Cause(LTE_5387a – LTE_5392a)
• E-UTRAN E-RAB Setup Success Ratio for High Priority Calls (LTE_5393a)
• Average Number of Simultaneous E-RABs per QCI Characteristic (LTE_5394a – LTE_5402a)
• E-UTRAN Average number of UEs utilizing UL intra-eNB CoMP (LTE_5403a)
• Cell Resource Group Utilization Ratio (LTE_5411a – LTE_5418a)
• PDCP SDU Loss rate (LTE_5450a – LTE_5459a)
• Inter RAT Handover to eHRPD (LTE_5461a – LTE_5469a)
• PDCP SDU Delay in DL(LTE_5471a – LTE_5479a)
• Averaged IP End User Throughput (LTE_5503a – LTE_5520a)
• Average Active UEs with Scheduled Data in DL (LTE_5501a)
• Average Active UEs with Scheduled Data in UL (LTE_5502a)

13 © Nokia 2014 - RA47042EN60GLA0

Deleted KPIs at RL60

• RRC Connection Setup Failure Ratio per Cause LTE_5231b

• RRC Connection Establishment Attempts due to others LTE_5235a
• E-UTRAN PDCP SDU Loss Ratio in the DL, nonGBR LTE_5309a
• E-UTRAN PDCP SDU Loss Ratio in the UL, nonGBR LTE_5315a
• Average Latency Downlink LTE_5134a,
• Average Latency Downlink for QCI1 LTE_5138a,
• Average Latency Downlink for Non GBRs LTE_5139a,
• Average Latency Downlink for QCI2 LTE_5556a
• Average Latency Downlink for QCI3 LTE_5557a
• Average Latency Downlink for QCI4 LTE 5558a

14 © Nokia 2014 - RA47042EN60GLA0

Index

• NSN counter-based KPIs

• NSN field test KPIs

15 © Nokia 2014 - RA47042EN60GLA0

Field KPI
Attach Time

•With Attach, the mobile terminal registers at the LTE network. At the end of the procedure
the UE is authenticated, and a default (nGBR) bearer is established.
•The Attach Time is the interval between the connection request and the acknowledgement
of the positive response by the UE.

•Mean value and 95% from all measured samples

•Formula : Attach Time [ms] = tAttach Complete – tAttach Request

•Related E-UTRAN Counters & KPIs

- None

16 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Detach Time

•With an explicit Detach request the UE informs the LTE network that it does
not want to access the EPS any longer. At the end of the procedure all EPS
bearers of the UE are released.
•The Detach Time is the interval between the Detach Request and the
reception of a Detach Accept message by the UE. No Detach Accept is sent
by the network if the cause for Detach is switching the UE off.

•Formula : Detach Time [ms] = tDetachRequest – tDetachAccept

•Related E-UTRAN Counters & KPIs

- None

17 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Attach Success Rate

•The Attach Success Rate is defined as the ratio between the number of successful
registrations and the number of all requests. This is the probability that a user can attach to
the LTE network at any moment of time.
•The calculated success ratio figure excludes attach requests, which are rejected by
authentication failures. On the other hand, network attach requests which are terminated by
timer expiry (due to the unavailability of some LTE resource) are considered as
unsuccessful registrations.
•If the success rate is calculated on the eNB by counting incoming RRC requests, RRC
CONNECTION REQUEST retries are to be excluded, since they would increase the overall
number of establishment attempts, and thus reduce the success ratio.
•Formula:
number _ of _ successful l _ attach
LTENwAttSR   100 %
number _ of _ attach _ attempt

•Related E-UTRAN Counters & KPIs

•PM counters: LTE EPS Bearer - M8006;
•KPIs: Initial E-RAB Setup Success Ratio (ISSR) - LTE_5112a
18 © Nokia 2014 - RA47042EN60GLA0
Field KPIs
Service Request Time: UE Initiated

•It is the time taken by the LTE network to setup an EPS bearer on request by the UE. The EPS bearer
can be new (dedicated), or an existing one (e.g. the default EPS bearer). The latter is needed to reassign
Uu radio and S1 bearer resources to the existing EPS bearer of a previously Idle UE.
•The EPS bearer has to be created or activated before IP packets can be exchanged, i.e. the Service
Request creates the IP link dynamically over the mobile access (LTE) network.
•The UE Initiated EPS Bearer Setup Time is the interval between the submission of the message RRC
CONNECTION REQUEST (see References 26. [3GPP36.331]) carrying a NAS: Service Request
message in its body, and the reception of the RRC CONNECTION RECONFIGURATION response by
the UE. The Bearer Setup procedure is based on the Service Request procedure, see References 3.
[3GPP23.401] and 20. [3GPP36.300].

19 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Service Request Time: UE Initiated Continued

•Mean value and 95% from all measured samples

•Formula:
•Service Request Time [ms] = tRRC_Reconfig – tRRC_Request

•Related E-UTRAN Counters

•Mean EPS Setup Time: SAEB.EstabTimeMean.QCI
•Max EPS Setup Time: SAEB.EstabTimeMax.QCI

20 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Service Request Time: Network Initiated

•It is the time taken by the LTE network to set up an EPS bearer on request by the P-GW.
The EPS bearer has to be created before IP packets can be sent (DL) to the UE if the UE
has no proper EPS bearer for the given IP packet flow. The network initiated Service
Request
•Time includes a Paging Time (ref. to Paging Time) if the UE is idle. The EPS Bearer Setup
procedure is based on the Paging and Service Request procedures according to
References 26. [3GPP36.331] and 3.[3GPP23.401].
•Mean value and 95% from all measured samples
•Formula: Service Request Time [s] = tRRC_Reconfig – tRRC_Request

•Related E-UTRAN Counters (same as UE Initiated)

•Mean EPS Setup Time: SAEB.EstabTimeMean.QCI
•Max EPS Setup Time: SAEB.EstabTimeMax.QCI

21 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Service Request Success Rate

•This KPI is defined as the ratio between successfully established EPS bearers compared
to the overall number of EPS bearer establishment attempts. It corresponds to the
probability that a user or the LTE network can establish an EPS bearer at any moment in
time.
•Requests that are terminated by timer expiry (due to the non-accessibility of some LTE
resource) are considered as unsuccessful attempts. Authentication errors (requests
rejected by the MME) are included in the total number of failures.
•Only the first RRC CONNECTION REQUEST is to be considered, since counting retries of
the same message would increase the overall number of bearer establishment attempts,
and thus reduce the success ratio.

22 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Service Request Success Rate Continued

•Formula:
number_of (RRC_CONN_ RECONFIGUR ATION_COMP LETE)
EPSSR   100 %
number_of (RRC_CONNE CTION_REQU EST)

•Related E-UTRAN Counters & KPIs

• Group of related PM counters: LTE EPS Bearer - M8006
• KPI: Additional E-RAB Setup Success Ratio (ASSR) - LTE_5113a

23 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Service Drop Rate

•It is the ratio between abnormally released bearers and the overall
number of established EPS bearers. An abnormal release is defined as
any EPS bearer termination that was not triggered by the mobile user
(from UE side). Thus, it reflects the probability that an established bearer
is aborted due to insufficient network resources.
•Dropping the bearer becomes visible to the end-user if an application
service is actively using it. If the application automatically re-establishes
the bearer, it remains unnoticed by the user.
•Formula:
number_of (dropped_c alls)
EPSBearerD R   100 %
number_of (successfu ll_calls)

24 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Testing methods for network wide testing

• Target:
• Few selected KPIs that can be post-processed automatically from drive test logs.
• Network wide drive testing executable with one drive (=one test case/script) per
cluster.

• Different test methods:

1. Each session started from EMM-DEREGISTERED state
2. Each session started from RRC-IDLE EMM-REGISTERED state
3. Each session started from RRC-CONNECTED state

25 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Testing methods

• 1. Each session started from EMM-DEREGISTERED state

• Attach – Application data – Detach
• Several data applications can be included in the same session (FTP DL/UL, HTTP,
Ping)
• Possible to measure Success Rates/times: Attach, ERAB setup, RRC setup,
Application setup, HO
• Not reflecting end user behavior, as users typically start sessions from EMM-
Time Subchannel Direction Message

REGISTERED state. sesssion1 13:56:34.094

13:56:34.094
Uplink
Uplink
ATTACH_REQUEST
PDN_CONNECTIVITY_REQUEST
13:56:34.097 CCCH Uplink RRCConnectionRequest
13:56:34.127 CCCH Downlink RRCConnectionSetup
13:56:34.132 DCCH Uplink RRCConnectionSetupComplete
13:56:34.366 DCCH Downlink SecurityModeCommand
13:56:34.368 DCCH Uplink SecurityModeComplete

•
13:56:34.369 DCCH Downlink UECapabilityEnquiry
Example script 13:56:34.390
13:56:34.410
DCCH
DCCH
Uplink
Downlink
UECapabilityInformation
RRCConnectionReconfiguration
13:56:34.414 DCCH Uplink RRCConnectionReconfigurationComplete
• Attach 13:56:34.416 Downlink ATTACH_ACCEPT
13:56:34.416 Downlink ACTIVATE_DEFAULT_EPS_BEARER_CONTEXT_REQUEST

• FTP DL (5MB) 13:56:34.424

13:56:34.458
DCCH Uplink
Uplink
MeasurementReport
ATTACH_COMPLETE
13:56:34.458 Uplink ACTIVATE_DEFAULT_EPS_BEARER_CONTEXT_ACCEPT
• FTP UL (3MB) 13:56:34.459 DCCH Uplink ULInformationTransfer
13:56:35.464 DCCH Uplink MeasurementReport

• HTTP page DL (x5) 13:56:56.606

13:56:56.607 DCCH
Uplink
Uplink
DETACH_REQUEST
ULInformationTransfer
13:56:56.662 DCCH Downlink DLInformationTransfer
• Ping 32-1024B (x10) 13:56:56.663 DCCH Downlink RRCConnectionRelease
Session 1 stop 13:56:56.665 Downlink DETACH_ACCEPT

• Detach 13:56:57.142
13:56:57.142
BCCH_BCH Downlink
BCCH Downlink
SYSTEM_INFORMATION_BCH
MASTER_INFORMATION_BLOCK

26 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Testing methods

• 2. Each session started from RRC IDLE, EMM-REGISTERED state

• RRC setup - Application data – Wait – Release due to inactivity
• Several data applications can be included in the same session (FTP DL/UL, HTTP,
Ping)
• Possible to measure Success Rates/times: Service request, ERAB setup, RRC setup,
Application setup, HO
• Challenges in practical measurement setup, especially with inactivity timer >10s.
Requires specific firewall settings to measurement laptop to block all outgoing
Time Subchannel Direction Message
packages during wait period. sesssion1 14:25:52.537
14:25:52.540
CCCH
DCCH
Downlink
Uplink
RRCConnectionSetup
RRCConnectionSetupComplete
14:25:52.574 DCCH Downlink SecurityModeCommand
14:25:52.576 DCCH Uplink SecurityModeComplete
14:25:52.577 DCCH Downlink RRCConnectionReconfiguration
14:25:52.581 DCCH Uplink RRCConnectionReconfigurationComplete
14:25:52.594 DCCH Uplink MeasurementReport

• Example script 14:25:53.634

14:26:06.540
DCCH
DCCH
Uplink
Downlink
MeasurementReport
RRCConnectionRelease
session2 14:26:17.957 CCCH Uplink RRCConnectionRequest
• RRC setup/service request 14:26:18.021
14:26:18.023
CCCH
DCCH
Downlink
Uplink
RRCConnectionSetup
RRCConnectionSetupComplete
14:26:18.059 DCCH Downlink SecurityModeCommand
• FTP DL (5MB) 14:26:18.061 DCCH Uplink SecurityModeComplete
14:26:18.061 DCCH Downlink RRCConnectionReconfiguration
• FTP UL (3MB) 14:26:18.066
14:26:18.077
DCCH
DCCH
Uplink
Uplink
RRCConnectionReconfigurationComplete
MeasurementReport
14:26:19.117 DCCH Uplink MeasurementReport
• HTTP page DL (x5) 14:26:57.044 DCCH Downlink RRCConnectionRelease
session3 ….
• Ping 32-1024B (x10)
• Wait until released by eNB
27 © Nokia 2014 - RA47042EN60GLA0
Field KPIs
Testing methods

• 3. Each session started from RCC Connected, EMM-REGISTERED state

• Application data only
• Several data applications can be included in the same session (FTP DL/UL, HTTP,
Ping)
• Possible to measure Success Rates/times: Application setup, HO
• Only mobility and re-establishment/drop related signaling is seen.

• Example script
• FTP DL (5MB)
• FTP UL (3MB)
• HTTP page DL (x5)
• Ping 32-1024B (x10)

28 © Nokia 2014 - RA47042EN60GLA0

Field KPIs
Testing methods

• Example list of KPIs that could be measured with different methods:

KPIs 1 2 3 Comment
Attach SR x Attach request -> Attach complete
Attach time x Attach request -> Attach complete
Session setup SR from Deregistetered x RRCC request -> ULinformation transfer (Attach complete)
Session setup time from Deregistered x RRCC request -> ULinformation transfer (Attach complete)
Session setup time from RRC idle x RRCC request -> RRCC reconf. Complete
Session setup SR from RRC idle x RRCC request -> RRCC reconf. Complete
LTE intra freq HO SR x x x
LTE inter freq HO SR x x x
Call/Session drop rate x x
# drops/minute x x x
Re-establishment success rate x x x
FTP/HTTP/PING SR x x x
RTT PING x x x
FTP DL throughput x x x
FTP UL throughput x x x
HTTP page download time x x x

Recommended method for mass testing, closest match to user experience

Could be used, if Attach success rate measurements are mandatory

29 © Nokia 2014 - RA47042EN60GLA0

30 © Nokia 2014 - RA47042EN60GLA0