Anda di halaman 1dari 17

The present document describes the compliance of SwissQual measurement system (data collection and post processing) with

the requirements given in the ETSI documents TS 102250 point by point. Each sheet contains the table of contents of the corresponding document with the compliance comments in the same line.

The ETSI document TS 102250 is a multipart deliverable and is Technical Specification (TS) that has been produced by ETSI Technical Committee Speech Processing, Transmission and Quality Aspects (STQ). It covers the QoS aspects for popular services in GSM and 3G networks, as identified below: The contents of the different parts is outlined below: Part 1: "Identification of Quality of Service aspects"; Part 2: "Definition of Quality of Service parameters and their computation"; Part 3: "Typical procedures for Quality of Service measurement equipment"; Part 4: "Requirements for Quality of Service measurement equipment"; Part 5: "Definition of typical measurement profiles"; Part 6: "Post processing and statistical methods". Part 1 identifies QoS aspects for popular services in GSM and 3G networks. For each service chosen QoS indicators are listed. They are considered to be suitable for the quantitatively characterization of the dominant technical QoS aspects as experienced from the end-customer perspective. Part 2 defines QoS parameters and their computation for popular services in GSM and 3G networks. The technical QoS indicators, listed in part 1, are the basis for the parameter set chosen. The parameter definition is split into two parts: the abstract definition and the generic description of the measurement method with the respective trigger points. Only measurement methods not dependent on any infrastructure provided are described in the present document. The harmonized definitions given in the present document are considered as the prerequisites for comparison of QoS measurements and measurement results.

Part 3 describes typical procedures used for QoS measurements over GSM, along with settings and parameters for such measurements. Part 4 defines the minimum requirements of QoS measurement equipment for GSM and 3G networks in the way that the values and trigger-points needed to compute the QoS parameter as defined in Part 2 can be measured following the procedures defined in part 3. Test-equipment fulfilling the specified minimum requirements, will allow to perform the proposed measurements in a reliable and reproducible way. Part 5 specifies test profiles which are required to enable benchmarking of different GSM or 3G networks both within and outside national boundaries. It is necessary to have these profiles so that when a specific set of tests are carried out then customers are comparing "like for like" performance. Part 6 describes procedures to be used for statistical calculations in the field of QoS measurement of GSM and 3G networks using probing systems.

Section 5 5.1 5.2 5.2.1 5.2.1.1 5.3 5.3.1 5.3.1.1 5.3.1.2 5.3.2 5.3.2.1 5.3.3 5.3.3.1 5.4 5.4.1 5.4.1.1 5.4.1.2 5.4.2 5.4.2.1 5.4.2.2 6 6.1 6.1.1 6.1.2 6.1.3 6.1.3.1 6.1.3.2 6.1.3.3

Indicator Name

compliant

comment

QoS Indicator List


Overview Service independent QoS Aspect: Network Access QoS Indicator: Network Accessibility (NA) Telephony QoS Aspect: Service Access QoS Indicator: Service Accessibility Telephony (SA-T) QoS Indicator: Setup Time Telephony (ST-T) QoS Aspect: Service integrity QoS Indicator: Speech Quality (SpQ) QoS Aspect: Service retainability QoS Indicator: Call Completion Ratio circuit switched (CCR-CS) Short Message Service QoS Aspect: Service Access QoS Indicator: Service Accessibility SMS MO (SA SMS MO) QoS Indicator: Access Delay SMS MO (AD SMS MO) QoS Aspect: Service integrity SMS QoS Indicator: End-to-end Delivery Time SMS (DT SMS) QoS Indicator: Completion Rate SMS circuit switched (CR SMS CS)

defined in Part 102250-2

defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2

defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2

Multimedia Messaging Service (MMS)


Void Preconditions Reservation due to implementation Overview of indicators for QoS measurement of MMS QoS Aspect: Service Access QoS Indicator: MMS Send Failure Ratio (MO) [%] QoS indicator: MMS Send Time (MO) [s] QoS Indicator: MMS Retrieval Failure Ratio (MT) [%]

defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2

6.1.3.4 6.1.4 6.1.4.1 6.1.4.2 6.1.4.3 6.1.4.4

QoS Indicator: MMS Retrieval Time (MT) [s] QoS Aspect: Service retainability QoS Indicator: MMS Notification Failure Ratio [%] QoS Indicator: MMS Notification Time [s] QoS Indicator: MMS End-to-end Failure Ratio [%] QoS Indicator: MMS End-to-end Delivery Time (MO/MT) [s]

defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2 defined in Part 102250-2

Section 4.2 4.2.1 4.2.2 4.3 4.3.1 4.3.2 4.3.3 4.3.4 4.3.5 4.4 4.4.1 4.4.2 4.4.3 4.4.4 4.5 4.5.1 4.5.2 4.5.3 4.5.4 4.6 4.6.1 4.6.1.1 4.6.1.2 4.6.1.3 4.6.1.4 4.6.1.5 4.6.1.6 4.6.1.7

Indicator Name Service Independent Network Accessibility Circuit Switched (NA - CS) Network Accessibility Packet Switched (NA - PS) Telephony Service Accessibility-Telephony (SA-T) Setup Time Telephony (ST-T) Speech Quality on call basis (SpQ-C) Speech Quality on sample basis (SpQ-S) Call Completion Rate Circuit Switched Telephony (CCR-CS-T) Short Message Service (SMS) Service Accessibility SMS MO (SA SMS MO) Access Delay SMS MO (AD SMS-MO) End-to-end Delivery Time SMS (DT SMS) Completion Rate SMS Circuit Switched (CR SMS CS) Circuit Switched Data Service Service Accessibility, Circuit Switched Data (SA - CSD) Set-up Time (ST - CSD) Data Quality (DQ-CSD) Completion Rate Circuit Switched Data (CR-CSD) Packet Switched Data Services Key Performance Indicators Method A {Service} Service Accessibility Ratio (SA-PSD) [%] {Service} Setup Time (ST-PSD) [seconds] {Service} IP-Service Access Ratio (IPSA-PSD) {Service} IP-Service Setup Time (IPST-PSD) {Service} Completed Session Ratio (CoSeR-PSD) [%] {Service} Session Time (SeT-PSD) {Service} Mean Data Rate (MDR-PSD) [kBit/s]

compliant (corresponding KPI No) 10100 10100

possible

X 30100 20100 20900 20900 30900 30900 -

X X 10200,10201,10300,103001,10400 10200,10201,10300,103001,10400 10211,10310,10311,10410 10211,10310,10311,10410 30201,30306,30405

4.6.1.8 4.6.2 4.6.2.1 4.6.2.2 4.6.2.3 4.6.2.4 4.6.2.5 4.6.2.6 4.6.2.7 4.6.2.8 4.6.3 4.6.3.1 4.6.3.2 4.6.3.3 4.6.3.4 4.6.3.5 4.6.3.6 4.6.3.7 4.7 4.7.1 4.7.2 4.7.3 4.7.4 4.7.5 4.7.6 4.7.7 4.7.8 4.8 4.8.4 4.8.5 4.8.6 4.8.7 4.8.8 4.8.9 4.8.10 4.8.11

{Service} Data Transfer Cut-off Ratio (DTCoR-PSD) [%] Key Performance Indicators Method B {Service} Service Accessibility Ratio (SA-PSD) [%] {Service} Setup Time (ST-PSD) [seconds] {Service} IP-Service Access Ratio (IPSA-PSD) {Service} IP-Service Setup Time (IPST-PSD) {Service} Completed Session Ratio (CoSeR-PSD) [%] {Service} Session Time (SeT-PSD) {Service} Mean Data Rate (MDR-PSD) [kBit/s] {Service} Data Transfer Cut-off Ratio (DTCoR-PSD) [%] Performance Indicators Unavailability [%] Attach Failure Ratio [%] Attach Setup Time [seconds] {Service} PDP Context Activation Failure Ratio [%] {Service} PDP Context Activation Time [seconds] {Service} PDP Context Cut-off Ratio [%] {Service} Round Trip Time [milliseconds] Multimedia Messaging Service (MMS) MMS send failure ratio (MO) [%] MMS retrieval failure ratio (MT) [%] MMS send time (MO) [s] MMS retrieval time (MT) [s] MMS notification failure ratio [%] MMS notification time [s] MMS end-to-end failure ratio [%] MMS End-to-end Delivery Time (MO/MT) [s] Streaming Streaming Service Non-Accessibility [%] Streaming Service Access Time [s] Streaming Reproduction Cut-off Ratio [%] Streaming Audio Quality [] Streaming Video Quality Streaming Audio/Video De-Synchronisation Streaming Reproduction Start Failure Ratio [%] Streaming Reproduction Start Delay [s]

20201,20301,20306,20400,20405 10320,10321,10420 10320,10321,10420 10307,10308,10407 10307,10308,10407 10211,10310,10311,10410 10211,10310,10311,10410 30307,30308,30407 20307,20308,20407 16200 15100 15100 15200 15200 25200 21000 20800 20801 20800 30804 30801 30801 30800 30800 10600 10600 20600 SquadLQ VQuad lipsync 30600 30600

4.9 4.9.3 4.9.4 4.9.5 4.9.6 4.9.7 4.9.8 4.9.9 4.9.10 4.9.11 4.9.12

Video Telephony VT Service Non-Accessibility [%] VT Service Access Time [s] VT Audio/Video Setup Failure Ratio [%] VT Audio/Video Setup Time [s] VT Cut-off Call Ratio [%] VT Speech Quality on Call Basis [MOS-LQO] VT Speech Quality on sample basis [MOS-LQO] VT Video Quality VT End-To-End Mean One-Way Transmission Time [s] VT Audio/Video Synchronization [%] Notes: Numbers represent KPI numbers (see KPI Users Guide) Numbers in brackets represent KPIs that do not match exactly the definition compliant: Used definition matches exact ETSI spec possible : Not defined but possible within the KPI framework Failure ratio and success rate is considered equivalent

10100 10100 (10105) (10105) 20100 (Call Quality) SquadLQ,P.862.1 VQuad RTT/2 lipsync

Section 5 5.1 6 6.1 6.2 6.3 6.3.1 6.3.2 6.3.3 6.3.3.1 6.3.3.1.1 6.3.3.1.2 6.3.3.2 6.3.3.2.1 6.3.3.2.2 6.3.3.3 6.3.3.4 7 7.1 7.1.1 7.1.2 7.2 7.2.1 7.2.1.1 7.2.1.2 7.2.2 7.2.2.1

Indicator Name

General aspects for all types of services


Set-up and control

Telephony measurements
General aspects Speech telephony Video telephony Measurement set-up and control Call Types Call transaction phases Call set-up Service accessibility Setup time Call connect Speech quality Call completion rate Call clear-down Pause

SMS measurements
General aspects SMS General aspects of SMS measurement Testing modes SMS-MT Event flow Specific QoS objects SMS-MO Event flow

compliant yes void void yes yes partly yes yes yes yes yes yes void yes yes partly yes

comment

network accessability

Idle phase

network accessability

7.2.2.2 7.3 7.3.1 8 8.1 8.1.1 8.1.2 8.1.3 8.1.4 8.1.5 8.2 8.3 9 9.1 9.2 9.3 9.3.1 9.3.2 9.3.3 9.3.4 9.3.5

Specific QoS objects Common QoS objects MMS

Data measurements
Common aspects Target for data access Test data content Outer conditions to be kept constant for testing Definition of transaction Assessing transactions Circuit switched Packet switched (GPRS) Speech measurements SMS measurements Data measurements HTTP FTP E-mail (SMPT/POP3) WAP UDP

no partly void yes yes yes yes partly no yes yes partly partly yes partly yes partly yes

bit error detection

no indication of the kind of error no circuit switched tests

Information to be logged for reproducibility of measurements


no notification type logged no OS,no MTU, no logical Location, no DNS no client type no access type

Notes: void partly not defined yet no definition missing condition in the comment

Section 5 5.1 5.2 6 6.1 6.2 6.3 6.4 6.5 6.5.1 6.5.2 6.6 6.6.1 6.6.2 6.7 6.8 7 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 8

Indicator Name

General requirements
General requirement for data logging Required information for logging

Mobile QoS Test-equipment (MQT)


Overview Test-MS Antennas Controller/processor/storage Man Machine Interface (MMI) MQT-LC MQT-RC Geographical positioning Format of geographical co-ordinates Accuracy Time sources Environnemental conditions

Fixed QoS Test-equipment (FQT)


General FQT for user-to-user services FQT for store-and-forward services FQT for information (download) services FQT for push services Controller Time-sources Environmental conditions

compliant yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes yes no yes yes yes

comment

no push services

QoS parameter export

8.1 8.2 8.2.1 8.2.2 8.2.2.1 8.2.2.2 8.2.2.3 8.2.2.4 8.2.3 8.3 8.3.1 8.3.2 8.3.3 8.3.4 Notes: void partly no

Overview Description of standard parameter set Geo data QoS data Telephony SMS CSD PSD Measurement system info Data selection and file format Data selection box Export File 1 Export File 2 Export File 3

yes yes yes yes no yes yes no no no no

no circuit switched data

export tool neccessary report neccessary report neccessary report neccessary

not defined yet no definition missing condition in the comment missing condition in the comment

Section 4 4.1 4.2 4.2.1 4.2.1.1 4.2.1.2 4.2.2 4.2.2.1 4.2.2.2 4.2.3 4.2.3.1 4.2.3.2 4.2.3.3 4.3 4.3.1 4.3.2 4.3.3 4.3.4 4.3.5 4.3.5.1 4.3.5.2 4.3.5.3

Indicator Name

Measurement profiles
Classification of measurement environments Service profiles Telephony Voice telephony Video Telephony Messaging Services SMS MMS Data services Circuit switched Packet switched Streaming Usage Profiles for Data Sessions Web browsing using HTTP E-Mail access File Transfer using FTP File Sharing using UDP Synthetic tests UDP ICMP TCP Notes:

compliant yes yes yes yes yes partly partly partly partly yes yes yes void void void

comment collectcion name call windows definable call windows definable call windows definable no E2E timeout defined no E2E timeout defined

not all timeouts definable

void partly no

not defined yet no definition missing condition in the comment missing condition in the comment

Section 4 4.1 4.2 4.3 4.4 5 5.1 5.2 5.3 5.3.1 5.3.2 5.4 5.5 5.6 5.6.1 5.6.1.1 5.6.1.1.1 5.6.1.1.2 5.6.1.1.3 5.6.1.2 5.6.1.2.1 5.6.1.3 5.6.1.4 5.6.1.5 5.6.1.6 5.6.2 5.6.2.1

Indicator Name Important measurement data types in mobile communications Data with binary values Data out of time-interval measurements Measurement of data throughput Data concerning quality measures Distributions and moments Introduction Continuous and discrete distributions Definition of density function and distribution function Probability Distribution Function (PDF) Cumulative Distribution Function (CDF) Moments and quantiles Estimation of moments and quantiles Important distributions Continuous distributions Normal distribution Standard normal distribution Central limit theorem Transformation to normality Log-Normal distribution Use-case: transformations Exponential distribution Weibull distribution Pareto distribution Extreme distribution (Fisher-Tippett distribution) Testing distributions Chi-square distribution with n degrees of freedom

compliant

comment

5.6.2.1.1 5.6.2.1.2 5.6.2.2 5.6.2.2.1 5.6.2.3 5.6.2.3.1 5.6.2.3.2 5.6.2.3.3 5.6.3 5.6.3.1 5.6.3.2 5.6.3.3 5.6.3.4 5.6.4 5.6.4.1 5.6.4.2 5.6.4.3 5.6.5 5.6.6 5.6.6.1 5.6.6.1.1 5.6.6.1.2 5.6.6.1.3 5.6.6.2 5.7 5.7.1 5.7.1.1 5.7.1.2 5.7.1.3 5.7.1.3.1 5.7.1.3.2 5.7.1.3.3 5.7.1.4 5.7.1.4.1 5.7.1.4.2 5.7.1.4.3

Further relations Relation to empirical variance Student t-distribution Relation to normal distribution F distribution Quantiles Approximation of quantiles Relations to other distributions Discrete distributions Bernoulli distribution Binomial distribution Geometric distribution Poisson distribution Transitions between distributions and appropriate approximations From binomial to Poisson distribution From binomial to Normal distribution From Poisson to Normal distribution Truncated distributions Distribution selection and parameter estimation Test procedures Chi-square test Kolmogorov-Smirnov test Shapiro-Wilk test Parameter estimation methods Evaluation of measurement data Statistical tests Formulation of statistical tests Classes of statistical tests Tests for normal and binomial data One-sample tests for normal data Two-sample tests for normal data Test for binomial data Distribution-free tests for location Sign tests Sign rank test Wilcoxon rank sum test

5.7.2 5.7.2.1 5.7.2.2 5.7.3 6 6.1 6.1.1 6.1.2 6.1.3 6.1.4 6.2 6.2.1 6.2.2 6.2.3 7 7.1 7.1.1 7.1.2 7.1.3 7.1.3.1 7.1.3.1.1 7.1.3.1.2 7.1.3.1.3 7.1.3.2 7.1.3.3 7.1.3.3.1 7.1.3.3.2 7.1.4 8 8.1 8.2 8.2.1 8.2.1.1 8.2.1.2 8.2.2 8.3

Confidence Interval Binomial distribution Normal (Gaussian) distribution Required sample size for certain confidence levels Visualization Techniques Visualization of static data Histograms Barplots QQ-Plots Boxplots Visualization of dynamic data Line Diagrams Temporal changing Boxplots MMQ-Plots Time series modelling Descriptive characterization Empirical moments Decomposition of time series Determination of the trend component Trend function types Linear trend function Polynomial trend function Non-linear trend models Trend estimation Transformation of time series by filtering Linear filters Exponential filters Seasonal component Data aggregation Basic data aggregation operators Data sources, structures and properties Raw data Performance data Event data Key Performance Indicators / Parameters Aggregation Hierarchies

8.3.1 8.3.2 8.4 8.4.1 8.4.2 8.4.3 8.4.4 8.5 8.5.1 8.5.2 8.6 8.6.1 8.6.2 9 9.1 9.2 9.2.1 9.2.1.1 9.2.1.2 9.2.2 9.3 9.3.1 9.3.2

Temporal Aggregation Spatial Aggregation Parameter estimation methods Projection method Substitution method Application of estimation methods Attributes of aggregation operators Weighted aggregation Perceived QoS Weighted quantiles Additional data aggregation operators MAWD and BH AVGn Assessment of performance indices Estimation of performance parameters based on active service probing systems Monitoring concepts Control charts Shewhart control charts CUSUM and EWMA charts Other alarming rules Methods for evaluation of objectives Desirability functions Loss functions

No

Issue 1 Network Accessability 2 Timing pattern

Comment Network Accessability is not independently measured identical number of attempts (no extra attempts in case of failure)

Anda mungkin juga menyukai