the requirements given in the ETSI documents TS 102250 point by point. Each sheet contains the table of contents of the corresponding document with the compliance comments in the same line.
The ETSI document TS 102250 is a multipart deliverable and is Technical Specification (TS) that has been produced by ETSI Technical Committee Speech Processing, Transmission and Quality Aspects (STQ). It covers the QoS aspects for popular services in GSM and 3G networks, as identified below: The contents of the different parts is outlined below: Part 1: "Identification of Quality of Service aspects"; Part 2: "Definition of Quality of Service parameters and their computation"; Part 3: "Typical procedures for Quality of Service measurement equipment"; Part 4: "Requirements for Quality of Service measurement equipment"; Part 5: "Definition of typical measurement profiles"; Part 6: "Post processing and statistical methods". Part 1 identifies QoS aspects for popular services in GSM and 3G networks. For each service chosen QoS indicators are listed. They are considered to be suitable for the quantitatively characterization of the dominant technical QoS aspects as experienced from the end-customer perspective. Part 2 defines QoS parameters and their computation for popular services in GSM and 3G networks. The technical QoS indicators, listed in part 1, are the basis for the parameter set chosen. The parameter definition is split into two parts: the abstract definition and the generic description of the measurement method with the respective trigger points. Only measurement methods not dependent on any infrastructure provided are described in the present document. The harmonized definitions given in the present document are considered as the prerequisites for comparison of QoS measurements and measurement results.
Part 3 describes typical procedures used for QoS measurements over GSM, along with settings and parameters for such measurements. Part 4 defines the minimum requirements of QoS measurement equipment for GSM and 3G networks in the way that the values and trigger-points needed to compute the QoS parameter as defined in Part 2 can be measured following the procedures defined in part 3. Test-equipment fulfilling the specified minimum requirements, will allow to perform the proposed measurements in a reliable and reproducible way. Part 5 specifies test profiles which are required to enable benchmarking of different GSM or 3G networks both within and outside national boundaries. It is necessary to have these profiles so that when a specific set of tests are carried out then customers are comparing "like for like" performance. Part 6 describes procedures to be used for statistical calculations in the field of QoS measurement of GSM and 3G networks using probing systems.
Section 5 5.1 5.2 5.2.1 5.2.1.1 5.3 5.3.1 5.3.1.1 5.3.1.2 5.3.2 5.3.2.1 5.3.3 5.3.3.1 5.4 5.4.1 5.4.1.1 5.4.1.2 5.4.2 5.4.2.1 5.4.2.2 6 6.1 6.1.1 6.1.2 6.1.3 6.1.3.1 6.1.3.2 6.1.3.3
Indicator Name
compliant
comment
defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2
defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2
QoS Indicator: MMS Retrieval Time (MT) [s] QoS Aspect: Service retainability QoS Indicator: MMS Notification Failure Ratio [%] QoS Indicator: MMS Notification Time [s] QoS Indicator: MMS End-to-end Failure Ratio [%] QoS Indicator: MMS End-to-end Delivery Time (MO/MT) [s]
defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2
Section 4.2 4.2.1 4.2.2 4.3 4.3.1 4.3.2 4.3.3 4.3.4 4.3.5 4.4 4.4.1 4.4.2 4.4.3 4.4.4 4.5 4.5.1 4.5.2 4.5.3 4.5.4 4.6 4.6.1 4.6.1.1 4.6.1.2 4.6.1.3 4.6.1.4 4.6.1.5 4.6.1.6 4.6.1.7
Indicator Name Service Independent Network Accessibility Circuit Switched (NA - CS) Network Accessibility Packet Switched (NA - PS) Telephony Service Accessibility-Telephony (SA-T) Setup Time Telephony (ST-T) Speech Quality on call basis (SpQ-C) Speech Quality on sample basis (SpQ-S) Call Completion Rate Circuit Switched Telephony (CCR-CS-T) Short Message Service (SMS) Service Accessibility SMS MO (SA SMS MO) Access Delay SMS MO (AD SMS-MO) End-to-end Delivery Time SMS (DT SMS) Completion Rate SMS Circuit Switched (CR SMS CS) Circuit Switched Data Service Service Accessibility, Circuit Switched Data (SA - CSD) Set-up Time (ST - CSD) Data Quality (DQ-CSD) Completion Rate Circuit Switched Data (CR-CSD) Packet Switched Data Services Key Performance Indicators Method A {Service} Service Accessibility Ratio (SA-PSD) [%] {Service} Setup Time (ST-PSD) [seconds] {Service} IP-Service Access Ratio (IPSA-PSD) {Service} IP-Service Setup Time (IPST-PSD) {Service} Completed Session Ratio (CoSeR-PSD) [%] {Service} Session Time (SeT-PSD) {Service} Mean Data Rate (MDR-PSD) [kBit/s]
possible
4.6.1.8 4.6.2 4.6.2.1 4.6.2.2 4.6.2.3 4.6.2.4 4.6.2.5 4.6.2.6 4.6.2.7 4.6.2.8 4.6.3 4.6.3.1 4.6.3.2 4.6.3.3 4.6.3.4 4.6.3.5 4.6.3.6 4.6.3.7 4.7 4.7.1 4.7.2 4.7.3 4.7.4 4.7.5 4.7.6 4.7.7 4.7.8 4.8 4.8.4 4.8.5 4.8.6 4.8.7 4.8.8 4.8.9 4.8.10 4.8.11
{Service} Data Transfer Cut-off Ratio (DTCoR-PSD) [%] Key Performance Indicators Method B {Service} Service Accessibility Ratio (SA-PSD) [%] {Service} Setup Time (ST-PSD) [seconds] {Service} IP-Service Access Ratio (IPSA-PSD) {Service} IP-Service Setup Time (IPST-PSD) {Service} Completed Session Ratio (CoSeR-PSD) [%] {Service} Session Time (SeT-PSD) {Service} Mean Data Rate (MDR-PSD) [kBit/s] {Service} Data Transfer Cut-off Ratio (DTCoR-PSD) [%] Performance Indicators Unavailability [%] Attach Failure Ratio [%] Attach Setup Time [seconds] {Service} PDP Context Activation Failure Ratio [%] {Service} PDP Context Activation Time [seconds] {Service} PDP Context Cut-off Ratio [%] {Service} Round Trip Time [milliseconds] Multimedia Messaging Service (MMS) MMS send failure ratio (MO) [%] MMS retrieval failure ratio (MT) [%] MMS send time (MO) [s] MMS retrieval time (MT) [s] MMS notification failure ratio [%] MMS notification time [s] MMS end-to-end failure ratio [%] MMS End-to-end Delivery Time (MO/MT) [s] Streaming Streaming Service Non-Accessibility [%] Streaming Service Access Time [s] Streaming Reproduction Cut-off Ratio [%] Streaming Audio Quality [] Streaming Video Quality Streaming Audio/Video De-Synchronisation Streaming Reproduction Start Failure Ratio [%] Streaming Reproduction Start Delay [s]
20201,20301,20306,20400,20405 10320,10321,10420 10320,10321,10420 10307,10308,10407 10307,10308,10407 10211,10310,10311,10410 10211,10310,10311,10410 30307,30308,30407 20307,20308,20407 16200 15100 15100 15200 15200 25200 21000 20800 20801 20800 30804 30801 30801 30800 30800 10600 10600 20600 SquadLQ VQuad lipsync 30600 30600
4.9 4.9.3 4.9.4 4.9.5 4.9.6 4.9.7 4.9.8 4.9.9 4.9.10 4.9.11 4.9.12
Video Telephony VT Service Non-Accessibility [%] VT Service Access Time [s] VT Audio/Video Setup Failure Ratio [%] VT Audio/Video Setup Time [s] VT Cut-off Call Ratio [%] VT Speech Quality on Call Basis [MOS-LQO] VT Speech Quality on sample basis [MOS-LQO] VT Video Quality VT End-To-End Mean One-Way Transmission Time [s] VT Audio/Video Synchronization [%] Notes: Numbers represent KPI numbers (see KPI Users Guide) Numbers in brackets represent KPIs that do not match exactly the definition compliant: Used definition matches exact ETSI spec possible : Not defined but possible within the KPI framework Failure ratio and success rate is considered equivalent
10100 10100 (10105) (10105) 20100 (Call Quality) SquadLQ,P.862.1 VQuad RTT/2 lipsync
Section 5 5.1 6 6.1 6.2 6.3 6.3.1 6.3.2 6.3.3 6.3.3.1 6.3.3.1.1 6.3.3.1.2 6.3.3.2 6.3.3.2.1 6.3.3.2.2 6.3.3.3 6.3.3.4 7 7.1 7.1.1 7.1.2 7.2 7.2.1 7.2.1.1 7.2.1.2 7.2.2 7.2.2.1
Indicator Name
Telephony measurements
General aspects Speech telephony Video telephony Measurement set-up and control Call Types Call transaction phases Call set-up Service accessibility Setup time Call connect Speech quality Call completion rate Call clear-down Pause
SMS measurements
General aspects SMS General aspects of SMS measurement Testing modes SMS-MT Event flow Specific QoS objects SMS-MO Event flow
compliant yes void void yes yes partly yes yes yes yes yes yes void yes yes partly yes
comment
network accessability
Idle phase
network accessability
7.2.2.2 7.3 7.3.1 8 8.1 8.1.1 8.1.2 8.1.3 8.1.4 8.1.5 8.2 8.3 9 9.1 9.2 9.3 9.3.1 9.3.2 9.3.3 9.3.4 9.3.5
Data measurements
Common aspects Target for data access Test data content Outer conditions to be kept constant for testing Definition of transaction Assessing transactions Circuit switched Packet switched (GPRS) Speech measurements SMS measurements Data measurements HTTP FTP E-mail (SMPT/POP3) WAP UDP
no partly void yes yes yes yes partly no yes yes partly partly yes partly yes partly yes
Notes: void partly not defined yet no definition missing condition in the comment
Section 5 5.1 5.2 6 6.1 6.2 6.3 6.4 6.5 6.5.1 6.5.2 6.6 6.6.1 6.6.2 6.7 6.8 7 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 8
Indicator Name
General requirements
General requirement for data logging Required information for logging
compliant yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes no yes yes yes
comment
no push services
8.1 8.2 8.2.1 8.2.2 8.2.2.1 8.2.2.2 8.2.2.3 8.2.2.4 8.2.3 8.3 8.3.1 8.3.2 8.3.3 8.3.4 Notes: void partly no
Overview Description of standard parameter set Geo data QoS data Telephony SMS CSD PSD Measurement system info Data selection and file format Data selection box Export File 1 Export File 2 Export File 3
not defined yet no definition missing condition in the comment missing condition in the comment
Section 4 4.1 4.2 4.2.1 4.2.1.1 4.2.1.2 4.2.2 4.2.2.1 4.2.2.2 4.2.3 4.2.3.1 4.2.3.2 4.2.3.3 4.3 4.3.1 4.3.2 4.3.3 4.3.4 4.3.5 4.3.5.1 4.3.5.2 4.3.5.3
Indicator Name
Measurement profiles
Classification of measurement environments Service profiles Telephony Voice telephony Video Telephony Messaging Services SMS MMS Data services Circuit switched Packet switched Streaming Usage Profiles for Data Sessions Web browsing using HTTP E-Mail access File Transfer using FTP File Sharing using UDP Synthetic tests UDP ICMP TCP Notes:
compliant yes yes yes yes yes partly partly partly partly yes yes yes void void void
comment collectcion name call windows definable call windows definable call windows definable no E2E timeout defined no E2E timeout defined
void partly no
not defined yet no definition missing condition in the comment missing condition in the comment
Section 4 4.1 4.2 4.3 4.4 5 5.1 5.2 5.3 5.3.1 5.3.2 5.4 5.5 5.6 5.6.1 5.6.1.1 5.6.1.1.1 5.6.1.1.2 5.6.1.1.3 5.6.1.2 5.6.1.2.1 5.6.1.3 5.6.1.4 5.6.1.5 5.6.1.6 5.6.2 5.6.2.1
Indicator Name Important measurement data types in mobile communications Data with binary values Data out of time-interval measurements Measurement of data throughput Data concerning quality measures Distributions and moments Introduction Continuous and discrete distributions Definition of density function and distribution function Probability Distribution Function (PDF) Cumulative Distribution Function (CDF) Moments and quantiles Estimation of moments and quantiles Important distributions Continuous distributions Normal distribution Standard normal distribution Central limit theorem Transformation to normality Log-Normal distribution Use-case: transformations Exponential distribution Weibull distribution Pareto distribution Extreme distribution (Fisher-Tippett distribution) Testing distributions Chi-square distribution with n degrees of freedom
compliant
comment
5.6.2.1.1 5.6.2.1.2 5.6.2.2 5.6.2.2.1 5.6.2.3 5.6.2.3.1 5.6.2.3.2 5.6.2.3.3 5.6.3 5.6.3.1 5.6.3.2 5.6.3.3 5.6.3.4 5.6.4 5.6.4.1 5.6.4.2 5.6.4.3 5.6.5 5.6.6 5.6.6.1 5.6.6.1.1 5.6.6.1.2 5.6.6.1.3 5.6.6.2 5.7 5.7.1 5.7.1.1 5.7.1.2 5.7.1.3 5.7.1.3.1 5.7.1.3.2 5.7.1.3.3 5.7.1.4 5.7.1.4.1 5.7.1.4.2 5.7.1.4.3
Further relations Relation to empirical variance Student t-distribution Relation to normal distribution F distribution Quantiles Approximation of quantiles Relations to other distributions Discrete distributions Bernoulli distribution Binomial distribution Geometric distribution Poisson distribution Transitions between distributions and appropriate approximations From binomial to Poisson distribution From binomial to Normal distribution From Poisson to Normal distribution Truncated distributions Distribution selection and parameter estimation Test procedures Chi-square test Kolmogorov-Smirnov test Shapiro-Wilk test Parameter estimation methods Evaluation of measurement data Statistical tests Formulation of statistical tests Classes of statistical tests Tests for normal and binomial data One-sample tests for normal data Two-sample tests for normal data Test for binomial data Distribution-free tests for location Sign tests Sign rank test Wilcoxon rank sum test
5.7.2 5.7.2.1 5.7.2.2 5.7.3 6 6.1 6.1.1 6.1.2 6.1.3 6.1.4 6.2 6.2.1 6.2.2 6.2.3 7 7.1 7.1.1 7.1.2 7.1.3 7.1.3.1 7.1.3.1.1 7.1.3.1.2 7.1.3.1.3 7.1.3.2 7.1.3.3 7.1.3.3.1 7.1.3.3.2 7.1.4 8 8.1 8.2 8.2.1 8.2.1.1 8.2.1.2 8.2.2 8.3
Confidence Interval Binomial distribution Normal (Gaussian) distribution Required sample size for certain confidence levels Visualization Techniques Visualization of static data Histograms Barplots QQ-Plots Boxplots Visualization of dynamic data Line Diagrams Temporal changing Boxplots MMQ-Plots Time series modelling Descriptive characterization Empirical moments Decomposition of time series Determination of the trend component Trend function types Linear trend function Polynomial trend function Non-linear trend models Trend estimation Transformation of time series by filtering Linear filters Exponential filters Seasonal component Data aggregation Basic data aggregation operators Data sources, structures and properties Raw data Performance data Event data Key Performance Indicators / Parameters Aggregation Hierarchies
8.3.1 8.3.2 8.4 8.4.1 8.4.2 8.4.3 8.4.4 8.5 8.5.1 8.5.2 8.6 8.6.1 8.6.2 9 9.1 9.2 9.2.1 9.2.1.1 9.2.1.2 9.2.2 9.3 9.3.1 9.3.2
Temporal Aggregation Spatial Aggregation Parameter estimation methods Projection method Substitution method Application of estimation methods Attributes of aggregation operators Weighted aggregation Perceived QoS Weighted quantiles Additional data aggregation operators MAWD and BH AVGn Assessment of performance indices Estimation of performance parameters based on active service probing systems Monitoring concepts Control charts Shewhart control charts CUSUM and EWMA charts Other alarming rules Methods for evaluation of objectives Desirability functions Loss functions
No
Comment Network Accessability is not independently measured identical number of attempts (no extra attempts in case of failure)