Web Service Modeling

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 1

Web Service Key Points of the User's Experience
Step 1: User inputs the URL Step 2: Some parts of the Step 3: User can see the
in the browsing address bar, website start showing in the whole page of the website.
and click ENTER interface.

“Can I Access to the Website?” “Can I see the whole page?”

“How long will it take to access “Is it fast when I see the whole
to the website?” page?”

Key Experience Points Accessibility Integrity

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 2

Web Service Key Points of the Working Flow
User Experience: User inputs the URL in the
browsing address bar, and click ENTER
① UE actives PDP context. This step is not needed if the PDP
context already existed when browsing website.
② DNS Query: translate the URL to the server IP address.
③ TCP Establishment: the browser connects to the website by
using the IP address.
④ First HTTP GET message: the first service message to the
website server.
HTTP 200 OK: server response to the browser successfully.

User Experience: Some parts of the website start

showing in the interface.
⑤ HTTP Data 1~n: the browser obtains the web page
information from the website by means of GET/POST and
displays the web page information

User Experience: User can see the whole page of

the website.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 3

WEB KQI Hierarchy KQI

KPI

Signaling KPI
Accessibility Integrity
Page Response Page Response Page Browsing Page Browsing Page Download
Success Rate Delay Success Rate Delay Throughput

DNS Query TCP Connect DNS Query DNS Query Page Average
Success Rate Delay Success Rate Delay Size

TCP Connect
TCP Connect DNS Query TCP Connect Page Download
Success Rate
Success Rate Delay Delay Delay
Page Content
Browsing GET/POST
First GET First GET Page Content
Success Rate Retransmission
Success Rate Response Delay Browsing Delay
Rate
GET Success
Rate TCP
Retransmission
Rate
POST Success
Rate

Attach Success Rate Auth&Cipher Success Rate Active PDP Success Rate RAB Success Rate SGSN DNS Query Success Rate

Attach Delay Auth&Cipher Delay Active PDP Delay RAB Delay SGSN DNS Query Delay
AAA Connection Success
Create PDP Success Rate Create PDP Delay AAA Connection Delay TCP RTT
Rate

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 4

KQI - Page Response Success Rate & Delay
vs. ETSI Formula:
Page Response Successes
Page Response
100%
Success Rate Page Requests

DNS Query Success Rate = DNS Query Successes/DNS Query Requests (Point 1~ 2)
TCP Connect Success Rate = TCP Connect Successes/TCP Connect Requests (Point 3~ 4)
First GET Success Rate = First GET Successes/ First GET Requests (Point 5~ 6)

Point 1：Page Request

Point 6：Page Response Success vs. ETSI formula result：
Point 1: DNS Query Request
ETSI begin with TCP SYN message while
Point 2: DNS Query Success SEQ begin with DNS Query message. So
Point 3：TCP Connect Request
Point 4：TCP Connect Success SEQ modeling is more consistent
Point 5：First GET Request with customer experience.
Point 6：First GET Success

vs. ETSI Formula: Page Response Delay

Page Response Success Time Page Request Time

DNS Query Delay = DNS Query Success Time – DNS Query Request Time (Point 1~ 2)
TCP Connect Delay = TCP Connect Success Time – TCP Connect Request Time (Point 3~ 4)
First GET Response Delay = First GET Success Time – First GET Request Time (Point 5~ 6)

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 5

KQI - Page Browsing Success Rate & Delay
Page Browsing Successes vs. ETSI Formula:
Page Browsing
100%
Success Rate Page Requests
Page Content Browsing Success Rate = Page Browsing Successes / First GET Successes (Point
6~ 9)
GET Success Rate = GET Successes / GET Requests (Point 5~ 6, and Point 7~8)
POST Success Rate = POST Successes / POST Requests

Point 1： Page Request

vs. ETSI formula result：
Point 9： Page Browsing Success ETSI begin with TCP SYN message while
SEQ begin with DNS Query message. So
SEQ modeling is more consistent
Point 1: DNS Query Request
Point 2: DNS Query Success with customer experience.
Point 3：TCP Connect Request
Point 4：TCP Connect Success
Point 5：First GET Request
Point 6：First GET Success
Point 7: GET Request Page Browsing Delay
Point 8：Get Success

Page Browsing Success Time Page Request Time

Page Content Browsing Delay = Page Browsing Success Time - First GET
Success Time (Point 6~ 9)

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 6

KQI - Page Download Throughput
Pages Size
Page Download
Throughput vs. ETSI
Page Download Delay
Formula:
Pages Size = Page Download Total Packets (Point 1~ 4)
Page Download Delay = ∑1n (Last Packet Time – First Packet Time) (Point 1~2, and
Point 3~4)

vs. ETSI formula result：

ETSI begin with Reception of the first
data packet containing content (point 1)
and end with the last packet (point 4).
Point 1: First Packet for First GET
Point 2: Last Packet for First GET But theoretically, data is not transferred
Point 3: First Packet for Following GET all the time (e.g. from point 2~ point 3). This
Point 4: Last Packet for Following GET duration can not be included in the
downloading time and must be moved away.

So SEQ modeling is more reasonable.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 7

 Streaming Service Modeling

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 8

Streaming Service Key Points of the User's Experience
Step1: User chooses the video. Step2: Video starts to buffer. Step3: Video begin to play.

Step4: Video stalls. Step5: Video stalls successfully Step6: Video Disconnect.
and plays again.

“Can I open the video?” “Does the video play smoothly?”

“Why video stalls so much frequently?”

“How long will it take to
start to open the video?” “Will it take long time for the stalling?”
Key Experience Points

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 9

Streaming Service Key Points of the Work Flow
① User opens the main website.
This step is a web browsing work flow, and not belongs to
video streaming.

② User chooses a video and click the PLAY button.

This is the begin of user experience. DNS query process get
the video server IP address, and TCP establish process built
a connection between client and that server.

③ Server returns a response and the window shows

buffering. Client begins downloading video data.

④ Video starts to play when media data is enough.

⑤ Video stalls when the data down loading rate is less than
playing rate, and the media data will be consumed soon.

⑥ Video plays again when media data is enough.

⑦ Connection ends when video finished, or some abnormal

interruption happened.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 10

Streaming KQI Hierarchy KQI

KPI

Signaling KPI
Accessibility Integrity

Video Streaming
Video Streaming Video Streaming Video Streaming Video Streaming
Plays
Start Success Rate Start Delay Stall Frequency Stalled Time Rate
Disconnection Rate
TCP Download TCP Download RST
DNS Query DNS Query
Retransmission Retransmission Disconnection
Success Rate Delay
Rate Rate Rate

Video Streaming Video Streaming FIN

TCP Connect TCP Connect
Download Download Disconnection
Success Rate Delay
Throughput Throughput Rate

Initial Buffer GET Response

Success Rate Delay

Initial Buffer
Delay

Attach Success Rate Auth&Cipher Success Rate Active PDP Success Rate RAB Success Rate SGSN DNS Query Success Rate

Attach Delay Auth&Cipher Delay Active PDP Delay RAB Delay SGSN DNS Query Delay
AAA Connection Success
Create PDP Success Rate Create PDP Delay AAA Connection Delay TCP RTT
Rate

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 11

KQI - Video Streaming Start Success Rate
Initial Buffer Successes
Video Streaming 100%
Start Success Rate Play Requests

DNS Query Success Rate = DNS Query Successes / DNS Query Requests (Point 1~ 2)
TCP Connect Success Rate = TCP Connect Successes / TCP Connect Requests (Point 3~ 4)
Initial Buffer Success Rate = Initial Buffer Successes / Initial Buffer Requests (Point 7 ~8)

vs. ETSI
Formula:

Point 1: Play Request

Point 8: Initial Buffer Success

Point 1: DNS Query Request

Point 2: DNS Query Success
Point 3: TCP Connect Request vs. ETSI formula result：
Point 4: TCP Connect Success ETSI begin with TCP SYN message while SEQ begin with DNS Query
Point 5: GET Request
Point 6: GET Success
message. So SEQ modeling is more consistent with customer
Point 7: Initial Buffer Request experience.
Point 8: Initial Buffer Success

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 12

KQI - Video Streaming Start Delay
Video Streaming Start Delay

Initial Buffer Success Time Play Request Time

DNS Query Delay = DNS Query Success Time – DNS Query Request Time (Point 1~ 2)
TCP Connect Delay = TCP Connect Success Time – TCP Connect Request Time (Point 3~ 4)
GET Response Delay = GET Success Time – GET Request Time (Point 5~ 6)
Initial Buffer Delay = Initial Buffer Success - Initial Buffer Request (Point 7~ 8)

vs. ETSI
Formula:
Point 1: Play Request
Point 8: Initial Buffer Success

Point 1: DNS Query Request

Point 2: DNS Query Success
Point 3: TCP Connect Request vs. ETSI formula result：
Point 4: TCP Connect Success ETSI begin with TCP SYN message while SEQ begin with DNS Query
Point 5: GET Request
Point 6: GET Success
message. So SEQ modeling is more consistent with customer
Point 7: Initial Buffer Request experience.
Point 8: Initial Buffer Success

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 13

KQI - Video Streaming Stall Frequency & Stalled Time Rate

Total Stall Times Total Stall Duration

Stall Stalled
100%
Frequency Total Play Duration Time Rate
Total Play Duration
Total Stall Duration
= ∑1n (Streaming Play Time – Streaming Stall Time) (Point 2 ~ 3)
Total Stall Duration
= Streaming Disconnect Time – Streaming Begin Time (Point 1 ~ 4)
TCP Download Retransmission Rate vs. ETSI
= TCP Download Retransmission Packets / TCP Download Packets
Formula:
Video Streaming Download Throughput
= Video Streaming Download Packets / Video Streaming Download Delay

vs. ETSI formula result:

1. ETSI modeling is based on streaming over RTP/RTCP
protocol, while Huawei modeling is based on streaming
Point 1: Streaming Begin over HTTP protocol. So the equation will be a little
Point 2: Streaming Stall different.
Point 3: Streaming Play
Point 4: Streaming Disconnect 2. ETSI provides KPI for each re-buffering (ratio and delay),
but there maybe several stalling during a whole video.
So Huawei provides KQI (stalling frequency and stall
duration) which are more in accordance with user
experience.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 14

KQI - Video Streaming Plays Disconnection Rate
Streaming Disconnection Times
Plays Disconnection Rate 100%
Streaming Begin Times

RST Disconnection Rate = RST Disconnection Times / Streaming Begin Times (Point 4 ~ 1)
FIN Disconnection Rate = FIN Disconnection Times / Streaming Begin Times (Point 4 ~1) vs. ETSI
Formula:

vs. ETSI formula result:

1. ETSI KPI is a parameter describing the probability that
before starting successfully the is ended by a cause
other than the intentional termination by the user.
Point 1: Streaming Begin
Point 2: Streaming Stall 2. Huawei KQI is a parameter describing the probability
Point 3: Streaming Play that after starting successfully the is ended by the
Point 4: Streaming Disconnect cause that the client or server fails to process the
multimedia service efficiently or the network
transmission quality is poor.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 15

P3 Introduction
What is P3 Standard P3 According to
P3 is a professional network optimization  P3 defines KQIs in conformance with the ETSI TS 102 250 
consulting and service company in Europe. specifications. The test items and KQIs used in the tests 
vary slightly depending on the countries where the tests 
Flagships for P3
are conducted.
 Connect magazine from Germany
It conducts annual competitive tests among operators  P3 Test Range
Germany, Austria and Switzerland and releases the 
Covers voice and data services, focusing on:
test results on Connect, the most influential magazine 
of its kind in Europe.   Web browsing
 Competition test between VDF subnet and   YouTube video (HD and SD)
other operators   HTTP‐based file uploading and downloading 
VDF group compares it’s subnet with other competitors  (10s or 1/3 MB UL/DL file transfer)
from the same city through P3, and publish the results  The main approach used in P3 test is proactive 
to the local media.
drive test. 
Influence of P3
It’s influence spreads gradually from Europe to Asia,  Tools P3 Uses
Africa, and Latin America. Cooperation with industry‐leading road test tool 
P3 also serves as the editor (president) of TCP‐based  manufacturers, such as R&S, SwissQual (also belong to 
video (such as YouTube) technical report on service quality  R&S now).
standardization (TR 101 578) released by ETSI.

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 16

Scene 1: Web Service Testing in China Mobile
Testing Scenario: 4G testing result
 60 times web browsing in 4G Page Page Response Page Response Page Browsing Page Download
 202 times web browsing in 3G Requests Success Rate (%) Delay (ms) Delay(ms) Throughput(kbps)
SwissQual 60 100% 717.655 2480.724 398.333
Visited Websites:
http://www.jd.com SEQ 58 100% 913.483 2406.66 387.630
http://www.taobao.com
conformity 96.67% 100% 91.38% 94.83%
http://www.sohu.com
http://3g.qq.com
http://www.baidu.com
http://www.sina.com.cn
2G/3G testing result
http://www.ganji.com.cn Page Page Response Page Response Page Browsing Page Download
http://m.tmall.com Requests Success Rate (%) Delay (ms) Delay(ms) Throughput(kbps)
http://3g.mop.com
SwissQual 202 96.63% 1109.693 8985.866 111.35
http://m.58.com
http://l.9game.cn SEQ 202 100% 1141.208 9003.308 102.27
conformity 100% 96.63% 94.55% 98.5%
Testing compare to:
SwissQual
Observed Testing Indicators:  In summary, conformity of two systems achieves more than 95%.
Page Requests   The difference of delays because of the statistic manner and formula. SwissQual accords to
Page Size ETSI while SEQ accord to TMF.
Page Response Success Rate (%)   Throughput of SwissQual will be little more than SEQ because some page was transferred in
Page Response Delay (ms)  gzip. So the web page size will be larger for SwissQual.
Page Browsing Delay(ms) 
Page Download Throughput(kbps) 

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 17

Scene 2: Web Service Testing in Danmark
Testing result
Testing Scenario:
Page Page Response Page Response Page Browsing Page Download
 2663 times web browsing in 4G Requests Success Rate (%) Delay (ms) Delay(ms) Throughput(kbps)
SwissQual 2663 99.58% 212.51 1429.355 31658.04
Visited Websites: SEQ 2663 100% 263.35 1549.71 2495.44
www.google.com
conformity 100% 99.58% 99.40% 99.85%
www.youtube.com
www.tdc.dk
portal.etsi.org

 In summary, conformity of two systems achieves more than 99%.

 The difference of delays because of the statistic manner and formula. SwissQual accords to
Testing compare to: ETSI while SEQ accord to TMF.
SwissQual  Throughput of SwissQual will be more than SEQ because web page was transferred in
gzip. So the web page size will be larger for SwissQual.

Observed Testing Indicators: Website Ratio_Pagesize=SEQ/P3

Page Requests 
google.com 0.0016
Page Size
Page Response Success Rate (%)  youtube.com 0.3500
Page Response Delay (ms)  portal.etsi.org 0.3081
Page Browsing Delay(ms)  tdc.dk 0.5445
Page Download Throughput(kbps) 

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 18

Scene 3: Streaming Service Testing in Danmark
Testing Scenario:
Testing result
Play Video Streaming Start Video Streaming Video Streaming
 477 times video streaming in 4G Requests Success Rate Start Delay Stall Times
(%) (ms)
SwissQual 477 100% 379 1

Visited Video : SEQ 431 100% 1406 0

Youtube conformity 90.36% 100% 98.5% 99.75%

 90.36% conformity for Play Requests
Analysis: 46 times video streaming were based on HTTPS protocol.  SEQ streaming modeling 
only based on HTTP protocol at this stage.
Testing compare to:
SwissQual

Observed Testing Indicators:  Most of the difference for Video Streaming Start Delay is about 1 second.

Analysis: Because of the statistic manner  difference, SwissQual get the data from handset 
Play Requests  directly while SEQ calculates the delay from parameters of the original packets.
Video Streaming Start Success Rate
Video Streaming Start Delay
Video Streaming Stall Times

HUAWEI TECHNOLOGIES CO., LTD. Huawei Confidential 19

 Thank you
www.huawei.com

Copyright©2011 Huawei Technologies Co., Ltd. All Rights Reserved.

The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and
operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and
developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for
reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice.