Speed5g-D6.2 Hybrid Simulations With Hardware in The Loop v1.0

Grant Agreement No.
: 671705
Research and Innovation action
Call Identifier: H2020-ICT-2014-2
quality of Service Provision and capacity Expansion through

Extended-DSA for 5G
D6.2: Hybrid simulations with hardware in the loop

Version: v1.0
Deliverable type Report

Dissemination level Public
Due date 31/12/2017
Submission date 22/12/2017
Lead editor Benoit Miscopein (CEA)
Authors Harald Weigold (R&S); Evangelos Kosmatos, Andreas Georgakopoulos,
Ioannis-Prodromos Belikaidis, Panagiotis Vlacheas, Dimitrios Kelaidonis,
Voula Vassaki, Athina Ropodi, Panagiotis Demestichas, Aristotelis
Margaris, George Paitaris, Kostas Tsagkaris, Evaggelika Tzifa, Aikaterini
Demesticha, Vera Stavroulaki (WINGS); Keith Briggs (BT); Benoit
Miscopein, Antonio de Domenico, Jérémy Estavoyer, David Miras (CEA).
Reviewers Uwe Herzog (EURESCOM); Shahid Mumtaz (IT)
Work package, Task WP 6, T6.2
Keywords Hardware in the loop, system-level simulations, offline samples, network
emulation, building block validation
Abstract
This deliverable reports on the hardware-in-the-loop simulation setups and results that are used to
validate two important objectives. First, the relevance of the project approach along the hierarchical
RRM framework based on FBMC waveform using heterogeneous traffic in dense networks is shown
by using parameters and models coming from hardware measurements. Secondly, several building
blocks of the project testbed have been validated (video testbed, cRRM communication, channel
selection based on reinforcement learning) before their integration in the project proof-of-concepts.
Document revision history

Version Date Description of change List of contributor(s)
V0 2017-10-26 ToC created and first draft form Benoit Miscopein
MS 14 documentation
V0.1 2017-10-31 BT section updates Keith Briggs
V0.2 2017-10-31 Updates to Chapter 1 WINGS
V0.5 2017-11-28 Updates to BT chapter Keith Briggs
V0.6 2017-11-30 More updates to BT chapter Keith Briggs
V0.7 2017-12-01 CEA section complete Benoit Miscopein
V0.8 2017-12-03 Updates to R&S chapter Harald Weigold
V0.9 2017-12-11 Yet more updates to BT chapter Keith Briggs
V0.10 2017-12-13 Updates to R&S chapter Harald Weigold
V0.11 2017-12-14 Wings section complete Vangelis Kosmatos
V0.12 2017-12-14 Abstract, Exec summary, Benoit Miscopein
Introduction and conclusion
complete
V0.13 2017-12-20 Implementation of modifications BM, HW, KB, VK
after internal review
V0.14 2017-12-21 Review by S. Vahid and S. Vahid and B. Miscopein
modifications by BM
V0.15 2017-12-21 Final Harald Weigold
V1.0 2017-12-22 Final editorial check and Uwe Herzog
submission
Disclaimer
This report contains material which is the copyright of certain SPEED-5G Consortium Parties and may
not be reproduced or copied without permission.
All SPEED-5G Consortium Parties have agreed to publication of this report, the content of which is
licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License1.
Neither the SPEED-5G Consortium Parties nor the European Commission warrant that the
information contained in the Deliverable is capable of use, or that use of the information is free from
risk, and accept no liability for loss or damage suffered by any person using the information.
Copyright notice
© 2015 - 2017 SPEED-5G Consortium Parties
1
http://creativecommons.org/licenses/by-nc-nd/3.0/deed.en_US
© 2015 - 2017 SPEED-5G Consortium Parties Page 2 of 53

Executive Summary
This deliverable details hardware-in-the-loop (HIL) simulations which have been run in order to serve
two objectives. First, it allows validating the project technical approach on Filter Bank Multi Carrier
(FBMC) using the concept of hierarchical RRM (hRRM). Secondly, this work prepares the integration
of some key components of the project testbed which will be presented with a number of Proof of
Concepts PoCs). The considered PoCs which will be impacted by this work are the following ones:
- PoC1: FBMC-MAC and hierarchical RRM
- PoC2: DCS MAC (Dynamic Channel Selection MAC)
- PoC4: Communication protocol for RRM
- PoC5: Video quality measurements in 5G networks
- PoC6: Integrated demonstration
The first objective is reached by means of system-level simulations (SLS) which have been run using
FBMC models coming from measurements performed on the project real-time MAC implementation,
and injected in the SLS tool using look-up tables. This allows to capture the essential characteristics
of the FBMC waveform and to show the gains of the SPEED-5G approach, compared to the legacy LTE
solution. Using simulation scenarios which capture heterogeneous traffic patterns on dense
heterogeneous networks, we show that the proposed solution, which will be showcased in PoC 1 and
PoC 6 can provide performance improvement of 10 to 35% on per-user throughput depending on the
type of service.
The second objective, related to the validation of key components before their integration in PoCs, is
reached for the following components: first, a reinforcement learning algorithm for channel selection
which will run on top pf the FBMC-MAC in PoC1 and PoC6 is validated by means of SLS which use
offline samples of spectrum sensing measured real signals. It shows that the channel selection
algorithm manages to provide a very good probability (around 90%) of selecting a free channel even
in the case of varying interference caused by neighbouring systems. Secondly, extensive
measurements on the video Quality of Service (QoS) and Quality of Experience (QoE) test bench of
the project have been made on both TCP and UDP traffic, simulating communication impairments
like packet loss, jitter, and latency. It shows that the video test bench is resilient on most of the
considered impairments, even if the video throughput is a sensitive parameter. The simulations
performed will help in tuning the test bench in an appropriate way to support the different
conditions of operations in PoC1, 2 and 6. Finally, this report shows how the centralised RRM (cRRM),
which is a part of the hRRM framework, has been exercised by using real hardware devices. By doing
so, the cRRM communication protocol is validated before being integrated into PoC1, PoC2 and
PoC6.

Table of Contents
Executive Summary ...................................................................................................................... 3
Table of Contents ......................................................................................................................... 4
List of Figures ............................................................................................................................... 6
List of Tables ................................................................................................................................ 8
Introduction ................................................................................................................................. 9
1 Capacity expansions through FBMC MAC and hRRM ..................................................... 11
1.1 hRRM and FBMC MAC overall approach ............................................................................... 11
1.2 Linkage to the SPEED-5G PoCs ............................................................................................... 11
1.3 Platform description .............................................................................................................. 12
1.3.1 Hardware-in-the-Loop statistics ............................................................................................ 13
1.4 Evaluation results................................................................................................................... 13
1.4.1 Simulation scenarios .............................................................................................................. 13
1.4.2 Simulation results .................................................................................................................. 15
2 HIL validation of the channel selection algorithm based on reinforcement learning ....... 17
2.1 Motivation and scope ............................................................................................................ 17
2.2 Linkage to the PoCs................................................................................................................ 17
2.3 Algorithm and hardware in the loop descriptions ................................................................. 17
2.3.1 Algorithm description ............................................................................................................ 17
2.3.2 Hardware-in-the-Loop platform ............................................................................................ 19
2.4 Validation procedure ............................................................................................................. 20
2.5 HIL simulation results ............................................................................................................ 21
2.5.1 Interference scenarios ........................................................................................................... 21
2.5.2 Simulation results .................................................................................................................. 22
3 Appropriate tuning of video QoS/QoE test bench .......................................................... 26
3.1 Motivation and Scope ............................................................................................................ 26
3.2 Linkage to the PoCs................................................................................................................ 26
3.3 Scenario/use-case/feature description ................................................................................. 26
3.4 Evaluation Criteria and Performance measures (KPIs) .......................................................... 27
3.5 Capabilities of radio technologies considered in the SPEED-5G PoCs ................................... 28
3.6 Configuration/Test Setup ...................................................................................................... 31
3.7 Measurements on a TCP video transmission......................................................................... 31
3.7.1 QoS parameters as a function of end-to-end latency ........................................................... 32
3.7.2 QoS parameters as a function of packet jitter ....................................................................... 33
3.7.3 QoS parameters as a function of packet loss ratio ................................................................ 34
3.7.4 QoS parameters as a function of packet corruption ............................................................. 40

3.7.5 QoS parameters as a function of packet duplication ............................................................ 41

3.7.6 QoS parameters as a function of packet re-ordering ............................................................ 41
3.8 Measurements on a UDP video transmission........................................................................ 41
3.8.1 QoS parameters as a function of end-to-end latency ........................................................... 42
3.8.2 QoS parameters as a function of packet jitter ....................................................................... 42
3.8.3 QoS parameters as a function of packet loss ........................................................................ 43
3.8.4 QoS parameters as a function of packet corruption ............................................................. 46
3.8.5 QoS parameters as a function of packet duplication ............................................................ 47
3.8.6 QoS parameters as a function of packet re-ordering ............................................................ 47
3.8.7 Findings and summary ........................................................................................................... 47
3.8.8 Tuning .................................................................................................................................... 48
4 RRM communication protocol evaluation ..................................................................... 49
4.1 Motivation and Scope ............................................................................................................ 49
4.2 Hardware-in-the-Loop platform ............................................................................................ 49
4.3 Parameters and statistics derived from hardware ................................................................ 49
4.4 Validation procedure and steps ............................................................................................. 50
4.5 Test results ............................................................................................................................. 51
4.6 Future work ........................................................................................................................... 51
5 Conclusion ................................................................................................................... 52
References ................................................................................................................................. 53

List of Figures
Figure 1: hRRM and FBMC MAC overall approach................................................................................ 12
Figure 2: Hardware-in-the-Loop (HIL) platform .................................................................................... 13
Figure 3: User throughput ..................................................................................................................... 15
Figure 4: File/report download time ..................................................................................................... 16
Figure 5: FBMC-MAC superframe structure.......................................................................................... 18
Figure 6: Channel selection algorithm overview ................................................................................... 19
Figure 7: Hardware in the loop set-up for channel selection algorithm validation .............................. 20
Figure 8 Scenario 1 – static interference conditions for channel selection validation ......................... 21
Figure 9: Scenario 2 – semi-static interference conditions for channel selection validation ............... 21
Figure 10: Scenario 3 – dynamic interference conditions for channel selection validation ................. 22
Figure 11: Per-channel policy indexes for the static case scenario ...................................................... 23
Figure 12: Per-channel policy indexes for the semi-static case scenario.............................................. 23
Figure 13: Per-channel policy indexes for the dynamic case scenario ................................................. 24
Figure 14: Aggregated count of successful accesses in the 3 channels (aggregated reward) .............. 24
Figure 15: Average free channel access probability (normalized reward)............................................ 25
Figure 16: Test setup of PoC 5 and PoC 6 in the focus of video QoS/QoE measurements ................... 26
Figure 17: Block error rates on FBMC (for each modulation and coding scheme) ............................... 30
Figure 18: Video Server and Monitoring System with Network Impairments Simulator ..................... 31
Figure 19: Web-GUI of WANem Network Impairments Emulator ........................................................ 31
Figure 20: Picture failure points with different throughput data rate conditions as function of packet
loss......................................................................................................................................................... 38
Figure 21: Throughput data rate requirement of a 10Mbps video ....................................................... 38
Figure 22: Picture failure points with different video throughputs as a function of SNR ..................... 39
Figure 23: Relevant test range below 6% BLER with FBMC .................................................................. 39
Figure 24: Range of error-free video transmission below 1% BLER with FBMC ................................... 40
Figure 25: Picture failure points with different data rate conditions as a function of packet corruption
............................................................................................................................................................... 40
Figure 26: Picture failure points with different throughput data rate conditions as a function of
packet duplication ................................................................................................................................. 41
Figure 27: QoS with packet loss (UDP, proMPEG FEC).......................................................................... 45
Figure 28: Picture quality reduction @ 0.01% packet loss (no FEC) ..................................................... 45
Figure 29: Picture quality reduction @ 3% packet loss (proMPEG FEC) ............................................... 46
Figure 30: Picture failure points in a UDP transmission (with and without FEC) as a function of SNR 46
Figure 31: Picture failure points in a UDP transmission (with and without FEC) as a function of packet
corruption ............................................................................................................................................. 47

Figure 32: Block diagram of the setup for Wifi tests. The red rectangle indicates the removable
absorbing material used to impair the channel. The 60 GHz link is used to provide an unimpaired
duplex channel for reference purposes only. It is not strictly part of the current test. ....................... 50
Figure 33: The Wifi testbed. The channel is impaired by covering the tin shielding boxes with
conducting foam. .................................................................................................................................. 50
Figure 34: Screenshot of terminal of the demo during channel impairment. Note that the TX bitrate
drops to zero near the top of the screen (red arrow).. ......................................................................... 51

List of Tables
Table 1: Summary of simulation parameters ........................................................................................ 14
Table 2: IP transmission related KPIs .................................................................................................... 27
Table 3: Video QoS related KPIs ............................................................................................................ 28
Table 4: Estimated MAC throughput on FBMC (for each modulation and coding scheme) ................. 29
Table 5: MAC throughput on FBMC (for each modulation and coding scheme) .................................. 29
Table 6: MAC KPIs on LTE ...................................................................................................................... 30
Table 7: Picture freeze ratio vs. symmetrical latency ........................................................................... 32
Table 8: Picture freeze ratio with uplink latency .................................................................................. 33
Table 9: QoS with jitter ......................................................................................................................... 33
Table 10: Video QoS variation with packet loss ratio (TCP, throughput headroom 100%) .................. 35
Table 11: Video QoS variation with packet loss ratio (TCP, throughput headroom 25%) .................... 36
Table 12: Video QoS variation with packet loss ratio (TCP, throughput headroom 8%) ...................... 37
Table 13: FBMC PHY throughput data rate ........................................................................................... 39
Table 14: FBMC SNR for error-free video.............................................................................................. 40
Table 15: QoS reduction due to latency (UDP/RTP).............................................................................. 42
Table 16: QoS with jitter ....................................................................................................................... 42
Table 17: QoS with packet loss (UDP no FEC) ....................................................................................... 43
Table 18: QoS with packet loss (UDP, proMPEG FEC) ........................................................................... 44

Introduction
This document reports the work performed for the last step of the SPEED-5G project before the
actual demonstration phase. It consists of Hardware in the Loop (HIL) simulations which aim at
bridging the design and simulation work in WP4 and WP5 on RRM and MAC respectively, and the
realisation of the proof of concepts (PoCs) of the project. These simulations will contribute to the
project achievements by achieving two important goals. On the one hand, The HIL simulations help
with evaluating a combination of key outcomes in system-level-simulations (SLS), wen injecting
parameters and models coming from hardware implementations which will be used in PoCs. By doing
this, this work allows extrapolating the expected gains provided by the SPEED-5G approach when
considering large-scale networks, which will not be possible in PoCs due to the limited number of
hardware devices involved. Compared to the simulations performed in WP4 and WP5, this work
allows to capture i) effects coming from implementation in general and ii) how networks based on
FBMC (Filter Banck Multi Carrier) modulation compare to legacy LTE networks on selected KPIs.
On the other hand, the work reported in this document prepares the integration of Hardware and
software (HW/SW) building blocks in the project testbed and PoCs. The hybrid simulations with HW-
in-the-loop aim at validating these blocks under realistic assumptions, which are representative of
the real-life operation. The outcomes of this work will be used in the following PoCs, as defined in the
project and reported in the SPEED-5G D6.1 deliverable:
- PoC1: FBMC-MAC and hierarchical RRM
- PoC2: DCS MAC (Dynamic Channel Selection MAC)
- PoC4: Communication protocol for RRM
- PoC5: Video quality measurements in 5G networks
- PoC6: Integrated demonstration
Whether it relates to the first or second objective, the different work items described in this
deliverable relates to a particular PoC (or a set of PoCs), which is (are) identified in task 6.3, in order
to feed the project workflow and to help with i) managing the successful integration of different
blocks and ii) validating the relevance of the project achievements.
This report is composed of 4 main parts.

Chapter 1 investigates by means of SLS how the hierarchical RRM framework developed in WP4 can
take advantage of the FBMC modulation characteristics to maximise the capacity of dense small cells
networks, assuming heterogeneous traffic patterns. This is done by modelling FBMC characteristics
using look-up-tables in SLS tools, coming from measurements performed on the platform where
FBMC-MAC has been implemented. This works complies with the first objectives as it allows
anticipating the results one can expect if the system, which will be showcased in PoC 1 and PoC 6, is
deployed on a large scale basis in real life. The other chapters relate to the second objective of PoCs
building block validation.
Chapter 2 describes the channel selection algorithm based on reinforcement learning, which will run
on top of the FBMC-MAC in PoC 1 and PoC 6. This work is based on system-level simulations, which
use files containing offline channel sensing results performed on the FleX board on real signals, to
mimic the interference of neighbouring systems. This work is intended to demonstrate that the
algorithm developed in WP4 is indeed able to operate in realistic environments.
Chapter 3 reports extensive HIL simulations on the video quality of service and quality of experience
(QoS, QoE) test bench, which will be used in PoC5 and PoC6. This chapter investigates by means of
inserting a software tool between the video server and client, which is able to emulate the behaviour
of and impairments induced by a real communication network. By simulating packet jitter, packet
losses, latency, or packet duplication, this work allows anticipating the tolerance of the QoS/QoE test

bench to the impairments and/or limitations of the communications systems involved in PoC6, and
provides appropriate tuning of the video server, depending on the test cases.
Finally, chapter 4 shows how the centralised RRM can support real radio communication devices,
showcasing a simplistic RRM use-case. This allows for the validation of the cRRM-cell communication
protocol before its integration in the different PoCs.

1 Capacity expansions through FBMC MAC and hRRM
1.1 hRRM and FBMC MAC overall approach

The scope of the proposed HIL platform is the evaluation of a selected set of proposed SPEED-5G
components (e.g. RRM algorithms, MAC protocols) using a combination of system-level simulation
tool and real hardware. This approach enables us the execution of a set of system level simulation
scenarios including a large number of small cells (SCs) and macro cells, UEs and devices (which
cannot be realised in real hardware PoCs) using models and parameter values based on statistics
collected from real hardware devices (USRPs2, Flex board from CEA). Therefore, we combine the
realistic values and models coming from hardware implementations with the scalability (high number
of elements) afforded by the system level simulation tools in order to present the gains of the
proposed SPEED-5G system based on the evaluation of the proposed FBMC-MAC against the LTE-A.
1.2 Linkage to the SPEED-5G PoCs

This work will mainly feed PoC 1, where FBMC-MAC is used in the unlicensed 5 GHz spectrum. The
overall scenario of the related PoC (PoC#1) is presented as a sequence diagram in Fig.1. This diagram
shows the different entities involved, while the interactions between them are illustrated. According
to this scenario, initially, the network can anticipate a new situation based on the collection and
analysis of measurements (e.g. a sudden increase of the load in a specific area). This information can
be alternatively reported to the network by some components outside of the network (e.g. vertical
mechanisms as illustrated in Figure 1). In response to the new situation, the dRRM is initially
responsible to make the appropriate decision to handle this new situation (e.g. decides on the set of
channels that can be selected by the MAC). The decision of dRRM is delivered to the cRRM for
further acceptance and handling. In detail, based on the assumed scenario, either dRRM just informs
cRRM about its decision (cRRM just updates its database of allocated resources) or ask cRRM to
contribute in the final decision (e.g. cRRM may decide to exempt a channel which knows to have low
quality). In addition, dRRM informs the fronthaul/backhaul about the new situation, so that resource
optimization can also be performed by the backhaul segment. Finally, dRRM communicates with the
MAC in order to provide the set of appropriate solutions to be used (e.g. a set of channels that can be
selected by the MAC). At MAC level, the Higher MAC (HMAC, see [1]) is responsible to make the final
decision on channel selection (e.g. select the channel with the best quality from the channels
reported by the dRRM).
2
USRP boards from Ettus Research™ (https://www.ettus.com/product/details/UB200-KIT)

Figure 1: hRRM and FBMC MAC overall approach
Indirectly, this work will also feed PoC 6 (SPEED-5G integrated PoC) as PoC 6 is partly based on PoC 1.
1.3 Platform description

The proposed HIL platform that was implemented is illustrated in Figure 2. The platform includes
both HW and SW elements. The HW components include a) a set of FLEX boards realizing the FBMC
MAC and PHY layers; b) a set of USRPs equipped with the OpenAirInterface (OAI) software which
realizes the LTE-A network. The SW components include mainly the system level simulation tool.
The HW components (USRPs and FLEX boards) are responsible for collecting measurements from the
real network and generate a set of statistics specific for each technology. These statistics are defined
in detail in section 1.3.1. The hardware components are responsible for the delivery of the collected
statistics and the provision of the configuration parameters values of the hardware component to
the system level simulation tool.
The system level simulation tool has been appropriately extended by the implementation of modules
which creates a set of modeling abstractions using the statistics collected by the hardware
components. In the current implementation, an abstraction for the LTE-A and FBMC are integrated,
capturing the following aspects of both LTE-A and FBMC:
 Spectral efficiency (BLER vs SINR) performance
 Number of Resource Blocks (RBs) per TTI
The abstractions, in the form of Look-Up-Tables (LUTs), model the spectral efficiency performance by
mapping BLER values to SINR values, as well as mapping the specific technology with a number of RBs
per TTI.
In addition to the aforementioned abstraction the hierarchical RRM is implemented to the tool,
which provides a representative set of SPEED-5G technical components.
Then, the system level simulation tool is used to execute a selected set of simulation scenarios. The
configuration information received by the hardware components is used by the system level
simulation tool to configure in the same way the components of the simulation environment (e.g. to
appropriately configure FBMC transceivers, SCs, etc.).

The system level simulation tool generates a set of performance results which represent the
performance of the system against a set of KPIs. These are used in order to compare the proposed
system (e.g. SPEED-5G RRM and MAC) against the baseline technology (LTE-A). The performance
results are presented and discussed in section 1.4.
System level
simulation tool
FBMC statistics
LTE-A statistics LTE
FBMC
abstractions
abstractions
LTE-A USRPs FLEX Boards
Simulation
results
Figure 2: Hardware-in-the-Loop (HIL) platform
1.3.1 Hardware-in-the-Loop statistics
The statistics that are collected by the hardware components in order the simulation tool to generate
the appropriate abstractions are the following in the FBMC case:
 Spectral efficiency curves
 BLER vs SINR curves for different MCSs
 Transport block size (how many bits in 1 TTI per MCS) for different configuration of FBMC
PHY
In the LTE-A case, the statistics are the following:
 Spectral efficiency curves
 Transport block size (how many bits in 1 TTI per MCS)
1.4 Evaluation results

In order to evaluate the proposed system (SPEED-5G with hRRM and FBMC MAC) a set of simulation
scenarios were executed using the system level simulation tool of the HIL platform. Before the
execution, the HIL statistics (described in the previous section) were integrated into the system level
simulation tool by implementing appropriate abstractions (e.g. integration of look-up-tables) in order
to capture the statistics derived from the real hardware capabilities.
1.4.1 Simulation scenarios
In order to evaluate the proposed SPEED-5G system against the baseline scenario (LTE-A), a set of
simulation scenarios were executed. Two scenario sets were defined:
 LTE-A (baseline scenario)
 SPEED-5G including FBMC-MAC and hRRM

The hRRM algorithm deployed in the system-level simulator is described in detail in deliverables D4.2
and D4.3 of the project. It utilizes a number of macro and small cells at 2GHz and 3.5GHz bands
respectively, each with different number of channels. Specifically, the simulation consists in7 macro
base stations (BS) each with three cells and also 30 SCs per BS, resulting to a total number of 210 SCs
and 231 cells in the network. In addition, we have utilized 5 channels, each of 10MHz bandwidth for
every cell. We discriminate between three different user types consuming three different service
types (eMBB, mMTC and URLLC). The eMBB users are executing FTP download of 2Mbyte files. The
mMTC users are devices which download small reporting messages of 10Kbytes, while URLLC users
which have strict requirements in terms of latency, also download small messages of 10Kbytes. The
traffic mix (user distribution) between the mMTC, eMBB and URLLC users are 94.4%, 4.6% and 1%
respectively. During the simulations, the number of users/devices are kept constant (252 URLLC
users, 1316 eMBB users and 26770 mMTC devices), while the inter-arrival time between requests is
configured appropriately in order to generate the targeted offered load. The total traffic is shared
among the macro BSs and SCs and ranges between 21-22% for the macro cells and 78-79% for the
SCs in our test cases.
Table 1 presents the simulation parameters of the system level simulation with the different loads of
the traffic expressed with a variety of inter-arrival requests from the different users resulting to
different offered loads spanning between 40Mbps to 63Mbps per cell.
Parameter Value
Macro BSs (with 3 cell each) 7
Inter-site distance 500 m
Small BSs 210
Total RAT Devices 231
Total Number of UEs 28338
94.4% mMTC, 4.6% eMBB
Traffic Mix (user mix)
and 1% URLLC
Offered load / cell 40Mbps to 63Mbps
Traffic model – eMBB FTP download
Traffic model – mMTC Data reporting (downlink)
Traffic model – URLLC Data reporting (downlink)
File size (KBytes) - eMBB users 2000
File size (KBytes) - mMTC users 10
File size (KBytes) - URLLC users 10
Exponentially distributed
Application layer packet inter-arrival
(eMBB, mMTC, URLLC)
5 (20MHz each - 10MHz
Total Number of channels
uplink/10MHz downlink)
A number of resource blocks: 50 PRBs per UL/DL channel
Transmission Time Interval (TTI) length : 1ms
Simulation time 60s
Table 1: Summary of simulation parameters

1.4.2 Simulation results
The performance evaluation of the SPEED-5G system (hRRM and FBMC MAC) against the LTE-A was
realised in terms of capacity (user throughput) and latency (download file time) results for all the
three different service types.
Figure 3 illustrates the performance of proposed SPEED-5G system againg the LTE-A in terms of user
throughput. From this graph, it is evident that SPEED-5G exhibits improvements for all the different
cases of offered load and for all three different service types.
Specifically, the gains in terms of throughput per user for the high priority users (URLLC users) are
around 8% to 10% for all the different offered loads. In the case of the eMBB users, SPEED-5G shows
improvements of the user throughput of around 19% (in the undertullised scenarios) to 24% (in the
overutilised scenarios). In the mMTC cases, the improvements of SPEED-5G remains dependent on
the offered load. At low loads, SPEED-5G shows some improvements of 14%, and reaching up to 35%
in the over utilised scenarios (63Mbps/cell).
User Throughput (URLLC) User Throughput (eMBB)

350 50
300 LTE-Α 45 LTE-Α
40
User Throughput (Mbps)
250 SPEED5G
35 SPEED5G
200 30
25
150
20
100 15
10
50
5
0 0
40 45 50 55 60 65 40 45 50 55 60 65
Offered Load (Mbps/Cell) Offered Load (Mbps/Cell)
User Throughput (mMTC)

3.0
2.5 LTE-Α
2.0 SPEED5G
1.5
1.0
0.5
0.0
40 45 50 55 60 65
Offered Load (Mbps/Cell)
Figure 3: User throughput
Figure 4 illustrates the performance of proposed SPEED-5G system against the LTE-A in terms of the
file and report download time. This time in the eMBB case is the time between the initialisation of
the FTP download request to the reception of the last packet of the file, while in the mMTC and
URLLC cases is the time for the successful download of the report files from the SC or macro cell to
the user terminal or device. The superiority of the proposed FBMC-MAC scheme in combination with
an hRRM algorithm is evident in the illustrated figure. The SPEED-5G system shows performance
improvements (lower file and report download time values) for all the scenarios under examination,
both in underutilised and over utilised cases and for all type of services.

Latency (Report download Time) (mMTC) Latency (File download Time) (eMBB)
50 2.0
45 LTE-Α 1.8
LTE-Α
40 1.6
Latency (ms)
35 SPEED5G 1.4 SPEED5G
30
Latency (s)
1.2
25 1.0
20 0.8
15 0.6
10 0.4
5 0.2
0 0.0
40 45 50 55 60 65 40 45 50 55 60 65
Offered Load (Mbps/Cell) Offered Load (Mbps/Cell)
Latency (Report download Time) (URLLC)

4.0
3.5
3.0
2.5
Latency (ms)
2.0
1.5 LTE-Α
1.0 SPEED5G
0.5
0.0
40 45 50 55 60 65
Offered Load (Mbps/Cell)
Figure 4: File/report download time

2 HIL validation of the channel selection algorithm based on

reinforcement learning
2.1 Motivation and scope

This topic is related to the testing and validation of a channel selection algorithm with inputs coming
from real measurement. The considered channel selection algorithm relies on a Multi-Armed Bandit
approach (MAB); it is expected to select the best channel in a shared spectrum band in which
neighbouring systems are active. This algorithm is based on a reinforcement learning approach,
where a reward is associated with a given channel when the transmission using selected channel
proves successful. As a matter of fact, the algorithm is able, using the reward system, to converge
towards the best channel selection given a (possibly varying) spectrum occupation. This algorithm is
described in SPEED-5G Deliverable D4.2 and has been validated through system-level simulation
(SLS).
The goal of this hardware-in-the-loop simulations is the evaluation of the behaviour of the algorithm
using realistic stimuli and real RF devices to perform the sensing. The validation is performed on both
the asymptotical convergence of the algorithm and on its ability to adapt to varying spectrum
occupancy conditions and fast convergence towards a suitable channel selection.
2.2 Linkage to the PoCs

This work will mainly feed PoC 1, where FBMC-MAC is used in the unlicensed 5 GHz spectrum. In
order to provide a sustainable operation in this shared spectrum, a channel selection algorithm is
required, to enable the selection of the best possible channel among a list of available channels.
Depending on its capability, RRM (either cRRM or dRRM, depending on the architecture) can send a
list of possible channels to the MAC layer or not. In any case, the small cell has to determine the
channel of operation on the unlicensed spectrum band and therefore a channel selection algorithm
has to be implemented to provide this feature. Testing the channel selection algorithm based on
reinforcement learning is the objective of this work, so to validate the performance on real-hardware
assumptions.
Indirectly, this work will also feed PoC 6 (SPEED-5G integrated PoC) as PoC 6 is partly based on PoC 1.
2.3 Algorithm and hardware in the loop descriptions

2.3.1 Algorithm description
An overview of the algorithm is represented in Figure 6, focusing on the FBMC-MAC operation carrier
on 𝑁 channels available in the unlicensed band. As the transceiver relies on FBMC-MAC, data is
transmitted in superframes during timeslots where the scheduler allocates ressource blocks to UE.
Because FBMC-MAC relies on a listen-before-talk approach, the superframe emission is made after a
clear channel assessment (CCA) procedure to make sure that the channel is not occupied by another
coexisting system. Figure 5 shows the superframe structure of FBMC-MAC; the interested reader can
refer to D5.2 deliverable for getting an in-depth description of FBMC-MAC [1].

Figure 5: FBMC-MAC superframe structure
The first timeslot of the FBMC-MAC superframe is a beacon used by the small cell to send common
and dedicated control channels. In particular, beacons contain the frame description (length, number
of slots UL/DL scheme), UE resource allocation in the superframe, possible command frames like
paging frames and channel switching information allowing SC to indicate that it will switch to another
operating channel in a number of superframes.
The channel selection algorithm uses the features of listen-before-talk and channel switching of
FBMC-MAC to operate. Per available channel, the algorithm maintains a 2 of indicators which drives
the computation of the reward function of each channel:
𝐶𝐶𝐴𝑂𝐾 (𝑖)
{ , 𝑖 ∈ [1, 𝑁]
𝐶𝐶𝐴𝑁𝑂𝐾 (𝑖)
knowing that when operating in channel i, CCAOK(i) is incremented each time a superframe is actually
sent (CCA is positive) and the mean PER of carried packets is below a certain threshold. CCANOK(i) is
incremented otherwise (CCA negative or mean PER is above the threshold). The reward ri associated
to channel i is a function of these 2 indicators meaning that 𝑟𝑖 = 𝑓(𝐶𝐶𝐴𝑂𝐾 (𝑖), 𝐶𝐶𝐴𝑁𝑂𝐾 (𝑖)). A simple
function for the reward is given by:
𝐶𝐶𝐴𝑜𝑘 (𝑖)
𝑟𝑖 = (Eq. 1)
𝐶𝐶𝐴𝑜𝑘 (𝑖) + 𝐶𝐶𝐴𝑛𝑜𝑘 (𝑖)
After each CCA, the reward of channel i is updated and a per-channel policy index is computed. The
latter relies on a Bayesian Upper Confidence Bounds approach, as described in [1]. Because this
algorithm will be implemented in embedded software in PoC1, the policy index is based on a simple
function as follows (given for channel 𝑗):
2 ln 𝑛
𝑃𝐼𝑗 (𝑛) = 𝑟̅𝑗 + √ (Eq. 2)
𝑛𝑗
where 𝑟̅𝑗 is the current average reward (as given previously), 𝑛 is the total number of iterations, and
𝑛𝑗 is the number of times channel j has been selected to be sensed. This policy index can be seen as
the quality of the estimation of the channel reward as the second term in the sum is a kind of bias
representing how far the current reward is from the true reward. After a convergence phase, the
policy index tends toward 1. The algorithm selects a channel, based on the following decision:
For iteration 𝑛, the selected channel defined by argmax∀ 𝑖 ∈ [1,N]{𝑃𝐼𝑗 (𝑛)}
Without loss of generality, Figure 6 shows an on-going transmission on channel 1 where the process
of indicators and policy indexes update is shown. The figure shows that after a number of
superframes (SF) sent on channel 1, the policy index of channel 𝑁 is getting the first in the list of per-

channel policy indexes values, triggering a switch to the channel 𝑁, which is performed thanks to a
channel switch procedure, supported by the FBMC-MAC ([1], [3]).
Freq (Hz) CCA CCA

« Switch to channel N » in beacon
free busy
Channel 1
Superframe #(i-1) SF #i SF #i+1 SF #k
No frame No frame
If PER < Th, If PER < Th, CCANOK(1) ++

CCAOK(1)++ CCANOK(1) ++ CCAOK (1)++
else else
CCANOK(1) ++ CCANOK(1) ++
Resume in Channel N
Channel N
SF
#k+1
If PER < Th,
CCAOK(n)++
else
CCANOK(n) ++
time
Figure 6: Channel selection algorithm overview
At initialisation stage, the channel selection decision is not applied until there is a transmission on
every single channel, for instance using a circular selection. Once this is done, the algorithm can be
run as exposed above.
2.3.2 Hardware-in-the-Loop platform
To test the algorithm performance with data from the “real-world”, we have bridged the Matlab-
based SLS where the algorithm is developed with hardware boards designed by CEA and used in WP5
for implementing the FBMC MAC. In this task there is no peculiar need for the FBMC MAC
implementation as only RF sensing capabilities are expected, to provide input to the channel
selection algorithm. As shown in Figure 7, the HIL platform is composed of 3 parts:
- A set of P interference sources which are programmed to transmit at a given power and
a given rate on one of the N possible channels. Modelling the rate of emission with e.g.
Poisson distribution with different and varying parameters can realistically represent the
activity of neighbouring systems, which a given small cell has to live alongside.
- The sensing board: this platform is able to sense a set of RF channels using an HW energy
detection block. The HW energy detection block is the same as the one used in the
FBMC-MAC implementation, as documented in [1].
- A Matlab-based SLS which simulates the deployment of SCs and UEs, coping with
heterogeneous downlink traffic. The SLS performs the decision of offloading traffic onto
the 5GHz band and runs the channel selection algorithm in order to maximise the
network throughput.
The HIL platform operates in the following way: interfering stations are programmed to mimic a
given spectrum occupancy ratio (both in the band and in time), using either a static interference

situation or more realistic situations where the occupancies of the different channels change over
time. In an offline manner, the sensing board periodically senses the different channels and provides
a file containing per-channel binary CCA decisions corresponding to the periodic sensing. This file is
given to the SLS as input to the channel selection algorithm during the simulation.
Chan1,
HIL platform Pout1, l1
SLS Matlab
Interfering stations
RX
HW-based clear
channel assessment reports
Chan2,
Pout2, l2
Chan3,
Pout3, l3
Figure 7: Hardware in the loop set-up for channel selection algorithm validation
2.4 Validation procedure

Essentially, the raw data coming from RX hardware board to the SLS software is the CCA result for a
given channel using an RF energy detection process. As configuration parameters, the HW board
takes i) the RF channel bandwidth and central frequency, ii) the sensing duration and iii) the ED
detection threshold.
Measurements performed by the HW board are given as inputs to Matlab code. The CCA file
corresponds to measurements done by the hardware sensing board, the interfering stations being
configured in a predefined way. The measurements are done in a periodic manner on the different
channels. For each channel, the data format is a vector composed of binary elements corresponding
to actual measurements (0: channel busy; 1: free channel). The matrix composed of all the vectors is
taken as such by the algorithm which will proceeds to a walk through the measurement results on all
the channels each time a channel is selected and the reward is computed for it. The channel selection
algorithm then converges towards the optimal solution (i.e. the least occupied channel).
The validation is done in 3 phases.
- Phase 1: the validation of the behavior of the algorithm is expected considering a static
configuration of the interfering stations in terms of activity factor. This step will show the
convergence and asymptotic behavior of the algorithm
- Phase 2: The ability of the algorithm to adapt to a dynamic and predefined modification
of the interference conditions is evaluated.
- Phase 3: The ability of the algorithm to adapt to non-deterministic modifications of the
interference conditions is evaluated.
The main performance metric used to validate the algorithm is the mean probability of success to
access on a free channel. As well, the evolution of the per-channel policy index and of the per-
channel reward (i.e. the count of per-channel successful attempts to access the channel) over time
has been considered.

2.5 HIL simulation results

2.5.1 Interference scenarios
As previously mentioned, 3 scenarios will be considered in this validation work. The baseline scenario
(scenario 1) is shown in Figure 8 and consists of having a list of 3 channels where interfering stations
have activity factors of 70%, 45% and 5% in channel 1, channel 2 and channel 3, respectively. This
mimics a situation where one channel is almost free (channel 3) whereas the 2 other ones have a
gradual and significant occupancy rate. This scenario is referred as to the “static case”.
Figure 8 Scenario 1 – static interference conditions for channel selection validation
The second considered scenario captures the case where the channel occupancy ratios are suddenly
modified, the goal is to assess how the algorithm is able to react to this change. The scenario is
captured in Figure 9; it is obtained the switch of the measured CCA from channel 1 to channel 3.
Referred as to the “semi-static case”, this scenario will demonstrate how the algorithm can converge
again towards a suitable solution.
Figure 9: Scenario 2 – semi-static interference conditions for channel selection validation
The third scenario is called the “dynamic case” and captures a situation where the 3 channels
experience various loads according to a random pattern (which could be caused by sporadic traffic
requests on 3 access points operating in the 2 channels). This scenario is depicted in Figure 10. Note
that Figure 10 is an illustration and will not be used as such in the simulations, where other
parameters for the simulation duration and switch probabilities will be used.

Figure 10: Scenario 3 – dynamic interference conditions for channel selection validation
The interfering stations are assumed to transmit at a transmit power such as the margin of the
strength of the signals coming from interfering sources received at the sensing station is stronger
than 10 dB above the sensitivity. Here, the sensitivity is assumed to be the mean level of energy
observed when the RX antenna port is connected to a load of 50 Ohms.
2.5.1.1 Simulation scenarios
A simple deployment layout of a 3-sector macro-cell with 1 or 2 SCs per sector and 50 UE per sector
is considered. The UEs have heterogeneous traffic patterns composed of near real-time video traffic.
Depending on the traffic, the algorithm implemented on the SLS decides whether some traffic should
be steered on unlicensed spectrum where 3 possible channels can be accessed; in the case some
traffic has to be steered onto the unlicensed spectrum, the channel selection algorithm has to
identify the most suitable channel, according to the algorithm depicted in section 2.3.
Based on periodic CCAs made on the 3 different channels with the HIL platform, the measurement
files containing the matrix of (binary) CCA results are used by SLS when the developed algorithm has
to select one of them. Each time a channel is selected, the SLS uses the column of CCA results on the
corresponding channel and retrieve from it the indication of the CCA result (walking through the
different rows). Depending on the binary value, the reward is computed accordingly. The goal of this
test is to validate the behaviour of the algorithm considering only 1 small cell.
2.5.2 Simulation results
The first set of results (Figure 11 to Figure 13) is related to the variation of the per-channel policy
indexes as a function of the learning iterations (i.e., time). For the 3 considered scenarios (see
Section 2.5.1), it shows that the per-channel policy index tends towards 1, i.e., when the algorithm
has learnt the reward of each channel. By zooming on the curves, Figure 11 shows also how this
algorithm actually works and progressively discards the loaded channels. For instance, on the
zoomed area, one can see that channel 1 (loaded at 70% of the time) is sometimes selected (because
its policy index becomes the largest, due to the bias term evolution over time, see (Eq. 2). Given that
this channel is loaded, the CCA may return a busy situation, which triggers an incremental increase of
the 𝐶𝐶𝐴𝑁𝑂𝐾 value and therefore inducing a significant decrease of the average channel reward value
(see (Eq. 1)). This explains the pseudo-periodic sharp decrease and progressive increase of the blue
and red curves. The very slow variation of the bias term when n is getting large (dominated by the
√ln 𝑛 term) explains how a loaded channel is checked with an interval which gets bigger and bigger,
leading to a progressive discard of this channel.

Figure 11: Per-channel policy indexes for the static case scenario
In Figure 12 and Figure 13, we show the effect of an environment with dynamic interference
conditions on the per-channel policy index. In particular, Figure 12 highlights the semi-static case
where a switching of the loads of channel 1 and channel 3 occur. One can see that the switch sparks
a quasi-instantaneous rise of the channel 1 policy index value, leading to a new stable situation
where channel 1 is selected (until the end of the simulation). In the same way, Figure 13 shows the
algorithm adaptation when the load on the 3 channels evolves following non deterministic events.
Figure 12: Per-channel policy indexes for the semi-static case scenario

Figure 13: Per-channel policy indexes for the dynamic case scenario
In Figure 14, the per-channel aggregated count of successful channel access is plotted for the 3
simulation scenarios (dashed lines for the static case, dash-dotted lines for the semi-static case and
dashed lines for the dynamic case). This figure further indicates how the algorithm works by showing
how often a channel has been accessed successfully. In other words, it shows the aggregated number
of 𝐶𝐶𝐴𝑂𝐾 (see Section 2.3.1) which have been experienced through the simulations. This figure
corroborates the previous ones on the policy indexes values variations by showing, for instance in
the dynamic case, that the count of channel accesses follows the indications appearing in Figure 13.
An interesting aspect that the count of successful channel accesses is irrespective to the number of
times the channel has been selected. For instance in the static case3, at the beginning of the
simulation, although the channel is selected few times (see in Figure 11 for instance), channel 1 can
be accessed successfully only once, when the number of iterations reaches 700; interestingly, this
successful channel access triggers a significant increase in the policy index of the channel, which
appears in Figure 11 to Figure 13.
Figure 14: Aggregated count of successful accesses in the 3 channels (aggregated reward)
Finally, Figure 15 captures the most important performance metric of the algorithm since it depicts
the probability of transmitting on a clear channel. This probability is obtained by averaging the per-
channel rewards, which are given in (Eq. 2). The best result is obtained for the static case, since the
3
Although the 3 cases give identical results as soon as there is no modification in the interference conditions

algorithm constantly improves the estimate of the per-channel reward as the number of iterations
increases. The very promising result is the actual average reward obtained for the dynamic case. In
Figure 15, one can see that every single modifications of the interference conditions translates into a
slight degradation of the probability of accessing a free channel but it doesn’t prevent the algorithm
to reach a performance close to the static case (less than 5% performance degradation).
Figure 15: Average free channel access probability (normalized reward)
Thanks to the HIL simulations results, the channel selection algorithm which will be part of PoC1 and
Poc6 can be considered as validated and ready to be implemented on top of the FBMC-MAC. It is
able to cope with real CCA measurements and it achieves promising performance, even in time
varying interference levels in the available radio channels.

3 Appropriate tuning of video QoS/QoE test bench
3.1 Motivation and Scope

Video applications are the main driver in developing future mobile networks like 5G and they will set
the performance requirements for data transmission in these networks. In this connection between
video transmission requirements and available network performance, the quality of service /
experience (QoS/QoE) parameters are a means to evaluate the improvement achieved based on the
architecture proposed by SPEED-5G.
3.2 Linkage to the PoCs

With respect to PoC 5 “Video quality measurements in 5G networks,” this task determines the
influence of different network impairments on a video transmission. Such impairments could be that
IP packets are lost because of interference on the radio channel, badly reordered at the receiver side,
because the traffic route has changed to a faster (shorter) transmission channel, or jitter occurs
because of additional traffic. All these impairments are considered since they may be side effects of
channel aggregation, offloading and/or dynamic channel switching which are main objectives of the
SPEED-5G technique. This is a pre-evaluation task designed to estimate the video quality that could
be achieved in a complete test setup as it is described in the integrated demos of PoC 6 (in
combination with the developed eDSA hardware) as shown in Figure 16.
Figure 16: Test setup of PoC 5 and PoC 6 in the focus of video QoS/QoE measurements
3.3 Scenario/use-case/feature description

Any IP network, especially the ones including a mobile communications network like LTE or 5G, does
not act as an ideal connection between client and server like a short high-speed cable. In real IP
networks, data transmissions are accompanied by influence from over-the-air conditions, network
regulation and side effects from other network traffic. Sometimes this may cause excessive demands
on channels or equipment with a resulting degradation of the service quality.
The integrated demos of PoC 6 will illustrate how the developed eDSA system improves the
behaviour of mobile communication networks in terms of throughput (bit rate), latency and
reliability. This is with the help of dynamic channel selection, offloading and channel aggregation
described in the use cases of PoC 6. These demos will produce some effects on the data channel,
which may be minor but detectable in the IP video transmission.

As per the given the application's video on demand, live video streaming, video gaming and video
conferencing, the focus of this project is set to live video streaming with 2 usable transmission
protocols. Mainly it is the point-to-point unicast transmission with TCP (transport control protocol),
as it is used already for video on demand services. The second transmission protocol is a point-to-
multipoint multicast UDP/RTP (User datagram protocol / Real-time protocol) streaming to enable the
video distribution to a very large number of viewers simultaneously, although this is currently not
possible over the internet because it needs a special setup in the network as it is already defined in
3GPP and known as “evolved Multimedia Broadcast Multicast Service” (eMBMS, [4]). For 5G, no
standard is set, but research and standardisation proposals are currently in progress, like for example
the German 5G-Media-Initiative ([5]) or the 5GPPP phase 2 project 5G-Xcast [6].
Currently used in these streaming applications is the streaming protocol “TS over IP”. The transport
stream (TS) is transmitted with about 7 transport stream packets (188 bytes) in a continuous data
flow wrapped in RTP for UDP multicast delivery. But for the internet, where multicast is unsuitable,
transport packets are stored in files representing some seconds of video. These files are located on a
web server for the client request. This streaming protocol is named HLS.
The newly invented streaming protocol CMAF (Common Media Application Format, [7]) - based on
DASH (Dynamic Adapted Streaming over HTTP, [8] [9]) - uses the ISOBMFF (ISO-Base Media File
Format, [10]) – also known as an mp4 container- to avoid the transport stream overhead and to
support the equal segmentation of all video and audio representations. This allows seamless
switching of quality levels (bit rates) due to network conditions as the main feature of DASH.
Inside the mp4 segments, the video is compressed according to the standards AVC/H.264 ([11]) or
with higher resolution request like 4k with HEVC/H.265 ([12]). Both standards are similar in the
structure, but with the newer HEVC higher compression rates respectively higher resolutions and
picture (frame) rates are possible.
For this new streaming protocol including the CMAF and HEVC, special server and client modules
were developed and will be integrated in the products R&S AVHE 100 Headend and R&S PRISMON.
3.4 Evaluation Criteria and Performance measures (KPIs)

The effects of network performance degradation are measurable in IP network transmission
parameters, which are:
KPI Description
Latency Delay of transmission between sender and receiver
Packet Jitter Variation of the transmission delay
Packet loss Errored IP packets per transmitted IP packets
Packet corruption Errored IP packet bytes per transmitted IP packet bytes
Packet duplication Copies of IP packets per transmitted IP packets
Packet re-ordering Difference in numbering of packets
Table 2: IP transmission related KPIs
These parameters will influence the video transmission. The intention of this task is to match these IP
network transmission parameters to the video QoS parameters measured by the video monitoring
system.
Parameters of QoS are:

KPI Description
Picture freeze (stall) Repetition of one picture over 4 or more picture periods
Peak Signal to Noise Ratio For the three components of a video picture Y, Cb and Cr, the
(PSNR) PSNR separately indicates the logarithmic ratio of the maximum
possible total deviation of all pixels from the nominal value of the
reference (MAX)
Mean opinion score video Based on ITU-T BT.500 quality in 5 steps (Bad, Poor, Fair, Good,
(MOS-V) Excellent) this measurement uses the Structural Similarity
Method (SSIM) algorithm.
Table 3: Video QoS related KPIs
In this task, the QoS values as a function of IP network transmission parameters are evaluated. These
simulated measurements will produce an estimation map for further trials that will be performed in
task T6.3. It shall show if optimisations or improvements of the system are required to guarantee the
usage of video services with no or at least minor restrictions anywhere and anytime in a 5G network.
Quality of Experience (QoE) includes these QoS parameters, but includes in addition system inherent
parameters like start-up-delay, media throughput, frequency of quality switches and human
expectations and weighting as well. These additional parameters are not measured in this task.
3.5 Capabilities of radio technologies considered in the SPEED-5G PoCs

The simulations performed in this chapter take into account the capabilities of the HW boards and
the developed software stack involved in the PoCs 1, 2 and 6, in order to anticipate the available
video data throughput. The considered set of radio technologies considered in the PoCs are the
following:
 LTE, implemented on USRPs
 FBMC-MAC, implemented on the CEA’s Flex board
 DCS MAC, implemented on URSPs
 A combination of the technologies embracing the previous ones and WiFi
Table 4 shows the raw throughput of an FBMC physical layer, assuming a 20 MHz channel and the
whole set of modulation and coding schemes (MCS) as defined in [1].

Modulation FEC rate Throughput (Mbps)

(bandwidth 20MHz)
QPSK 1/3 6.04
QPSK 1/2 9.06
QPSK 2/3 12.08
QPSK 3/4 13.59
QPSK 5/6 15.1
16-QAM 1/3 12.08
16-QAM 1/2 18.12
16-QAM 2/3 24.16
16-QAM 3/4 27.18
16-QAM 5/6 30.2
64-QAM 1/3 18.12
64-QAM 1/2 27.18
64-QAM 2/3 36.24
64-QAM 3/4 40.77
64-QAM 5/6 45.3
Table 4: Estimated MAC throughput on FBMC (for each modulation and coding scheme)
On this basis, FBMC-MAC has been implemented on the CEA FleX board, as reported in [3], using a
reduced set of MCS and a narrower bandwidth of 7.32 MHz. Table 5 shows the MAC throughput as a
function of the available MCS, obtained through measurements.
Modulation FEC rate MAC throughput (Mbps)
(bandwidth 7.32 MHz)
QPSK 1/2 3.8
QPSK 3/4 4.6
16QAM 1/2 6.5
16QAM 2/3 9.0
16QAM 3/4 10.2
64QAM 1/2 10.3
64QAM 2/3 11.84
Table 5: MAC throughput on FBMC (for each modulation and coding scheme)
An additional model of the FBMC-MAC used as an input of this work is the block error rate as a
function of SNR, which is shown in Figure 17. This has been measured on the FleX board, with
additive white Gaussian noise.

Figure 17: Block error rates on FBMC (for each modulation and coding scheme)
Finally, measurements characterising LTE implementation on USRPs (OpenAirInterface LTE

distribution) are given in Table 6.
KPI DownLink UpLink
MAC throughput (Mbps) 16.098 9.249
MAC throughput (Mbps) 34-35 17
MAC throughput (Mbps) 60 24
Jitter (ms) 0.473 0.853
Latency 14 14
Table 6: MAC KPIs on LTE
These simulation and measurement results, especially in the throughput data rate of about 15Mbps
are limiting the video transmission to actual formats and quality. Actual video formats are 1280x720
pixels per picture with 50 pictures per second (720p50) or 1920x1080 pixels times 25 pictures per
second (1080p25). These formats are called HDTV, whereas UHDTV comes with 4 times the size and
2 times the picture rate as 1080p25. The claim of 5G is the transmission of UHDTV services to mobile
users and therefore has to provide a downlink data rate of at least 15Mbps.

3.6 Configuration/Test Setup
Figure 18: Video Server and Monitoring System with Network Impairments Simulator
The video server and monitoring system run on a network emulator in the loop, instead of the real
transmission channel which will be provided in task 6.3 by PoC 1, PoC 2 and PoC 6. Because the
impairments of a real RF transmission (fading, multipath reception, …) and effects coming from the
control and operational processes on a SCs network are not available at this stage, the functions of
the Network emulator are used to inject impairments on the IP transmission. By setting the
impairments at IP level, real network conditions can be simulated.
The network emulator, called WANem Wide Area Network Emulator (version 2.3) software, comes
with a bootable CD to run on an i386 PC. This CD includes a Linux Knoppix operating system. After
booting the operating system on the PC, a network (IPv4) address is configured (usually by DHCP).
Main commands are available on the console. The PC is now able to simulate a real network.
With the configured IPv4 address, WANem settings can be controlled by Web-GUI on any browser
(<ip-address>/WANem).
Figure 19: Web-GUI of WANem Network Impairments Emulator
For network emulation, it is necessary to redirect the packet flow through the emulating PC.
Therefore on each outgoing equipment (here it is the video server system and the video quality
measurement system), a route command must be set.
3.7 Measurements on a TCP video transmission

For all simulations picture freeze and peak signal to noise ratio are evaluated. It is expected, that due
to TCP (transport control protocol) the measure results in a total failure of the video transmission
(picture freeze) when the impairments exceed a tolerance limit. In this task, the tolerance limits are

measured. This allows to specify a minimum channel quality for video transmissions or to adapt the
video test system to the achievable channel in the PoC1, PoC2, and PoC 6 setups.
This kind of internet protocol is used in so-called “over the top” OTT video streaming services.
Therefore MPEG DASH system is used. It is one characteristic of this system that segments of some
seconds of video are stored in a file and the files are provided by a server application (e.g. Apache
HTTP server). Clients request theses video segment files via HTTP, which includes a TCP transmission.
3.7.1 QoS parameters as a function of end-to-end latency
The results show a relation between the latency of a transmission and the achievable QoS. It shows
that up to a limit, the video service will be with no quality reduction. This is because a certain
buffering is already implemented due to video compression constraints.
Parameters:
 Video format UHD (3840x2160 = 4k)
 Video frame rate 50 Hz
 Video bitrate 10Mbps
 Video protocol: DASH (Segment duration 4.9s)
 IP Transmission protocol: TCP
 Error Correlation 0%
 Available throughput data rate 25Mbps
3.7.1.1 Symmetrical Latency
In this measurement, a latency buffer is inserted in both directions, which in network terms means
downlink and uplink.
Results:
Latency 2-way Latency Picture freeze
the setting measured in ms in %
in ms
0 0,35
10 21 0
100 212 0
120 251 0
150 312 0
170 352 0
200 410 0
220 452 50
250 511 100
300 612 100
Table 7: Picture freeze ratio vs. symmetrical latency

3.7.1.2 Latency in Uplink only
Because actual networks are designed to provide more capacity from the server to the client
(downlink) than from the client to the server, there may be a different behaviour in latency as well.
This measurement represents a fast downlink and a delayed uplink.
Results:
Latency 2-way Latency Picture freeze
setting measured in ms in %
in ms
0 0,27 0
10 13 0
100 104 0
200 205 0
300 304 0
350 355 0
400 405 0
450 455 0
475 480 0
500 504 100
Table 8: Picture freeze ratio with uplink latency
3.7.1.3 Interpretation of latency measurements results
The results show that the video transmissions is not affected by latency below 500ms from the time
sending the packet to the time receiving the acknowledgement of a successful delivery. This
independent of the unsymmetrical behaviour of networks.
3.7.2 QoS parameters as a function of packet jitter
The results show a relationship between the packet jitter of a transmission and the achievable QoS. It
is expected that the influence of jitter cannot be seen in the video service because it usually should
be absorbed by the buffering necessary for the video compression.
Results:
Jitter in ms Picture Freeze Picture Freeze Picture Freeze
@25MBps @15MBps @13MBps
in % in % in %
1 0 0 0
10 0 0 90
50 0 90 100
100 100 100 100
Table 9: QoS with jitter

3.7.2.1 Interpretation of jitter measurements results
This jitter parameter shows a dependency to the available bit rate, which can be explained by the
fact that the randomly delayed packets are delaying following packets also because they are not sent
immediately due to lack of capacity. In this constellation, the network simulator can’t be trusted in
the values set, because highly variable delays were measured by the “ping” command with this kind
of network simulation.
In general, the acceptable amount of jitter is high enough, so that there should be no problem in the
considered PoCs. With jitter caused by channel switching, offloading or other network management
procedures, however, there may be a problem when synchronization is lost and a new video session
has to be established.
3.7.3 QoS parameters as a function of packet loss ratio
The results show a relation between the packet loss of a transmission and the achievable QoS. The
video transmission over a TCP connection with a DASH protocol is tested.
3.7.3.1 Unlimited throughput data rate
Parameters:
 Latency 0.061ms

Results:
BLER Packet loss Picture freeze PSNR in dB
in % in %
0 0 0 100
0,000001 0,0001 0 100
0,000010 0,0010 0 100
0,000100 0,0100 0 100
0,001000 0,1000 0 100
0,010000 1,0000 0 100
0,020000 2,0000 0 100
0,030000 3,0000 0 100
0,035000 3,5000 0 100
0,037000 3,7000 20 0
0,038000 3,8000 50 0
0,039000 3,9000 60 0
0,040000 4,0000 70 0
0,050000 5,0000 100 0
0,060000 6,0000 100 0
Table 10: Video QoS variation with packet loss ratio (TCP, throughput headroom 100%)
3.7.3.2 Limited throughput data rate (25% Headroom)
Parameters:
 Latency 0.061ms
 Parameters:

Results:
in % in %
0 0 0 100
0,000001 0,0001 0 100
0,000010 0,0010 0 100
0,000100 0,0100 0 100
0,001000 0,1000 0 100
0,010000 1,0000 0 100
0,015000 1,5000 0 100
0,025000 2,0000 50 0
0,030000 3,0000 70 0
0,035000 3,5000 100 0
0,037000 3,7000 100 0
0,038000 3,8000 100 0
0,039000 3,9000 100 0
0,040000 4,0000 100 0
0,050000 5,0000 100 0
0,060000 6,0000 100 0
3.7.3.3 Limited throughput data rate (8% Headroom)
Parameters:
 Latency 0.061ms

Results:
in % in %
0 0 0 100
0,000001 0,0001 0 100
0,000010 0,0010 0 100
0,000100 0,0100 0 100
0,001000 0,1000 0 100
0,010000 1,0000 0 100
0,015000 1,3000 5 100
0,025000 1,4000 10 0
0,030000 1,5000 15 0
0,035000 1,6000 20 0
0,037000 1,7000 30 0
0,038000 2,0000 60 0
0,039000 2,2000 100 0
0,040000 4,0000 100 0
0,050000 5,0000 100 0
0,060000 6,0000 100 0
3.7.3.4 Interpretation of packet loss measurements results
The tests show the expected behaviour of a TCP connection. Because the compressed video is
packetized in segment files – each representing 4.9 seconds of video - which are stored on the local
content server, the video transmission is technically a file transfer where the client requests the files
via HTTP. To fit in IP-packets the files are segmented in small transport units.
This error correction mechanism works without any effect on the video quality as long as the repair
mechanism does not take too long. This deadline condition is affected if the repair packet is
corrupted also or there is not enough data rate available margin to send the repair packet in the
same channel. This leads to different picture failure points (picture freeze) depending on the
available throughput data rate.

Figure 20: Picture failure points with different throughput data rate conditions as function of packet loss
Below the limit of 13 Mbps data rate, no transmission of a 10 Mbps video signal is possible because
the measurement shows that the packaging of the video segments introduces an additional data
overhead of approx. 20%.
Figure 21: Throughput data rate requirement of a 10Mbps video
The measurement of picture freeze versus IP packet loss can now be combined with the BLER versus
SNR measurements performed on the FBMC setup with the assumption, that one block error causes
one IP packet error.
The following figure shows a minimum required signal to noise ratio (SNR) for an FBMC 64QAM 2/3
of 20,45dB.

Figure 22: Picture failure points with different video throughputs as a function of SNR
This modulation scheme was selected because of the throughput data rate (FBMC PHY) of 14.8
Mbps. Other modulation schemes are not sufficient to transmit UHD video. As result of this finding
the test system should be used with smaller picture sizes, e.g. HD instead of UHD to fit in the
available channel capacities.
MCS PHY throughput (Mbps)

QPSK ½ 4.86
QPSK ¾ 5.77
16QAM ½ 8.22
16QAM 2/3 11.3
16QAM 3/4 12.76
64QAM1/2 12.9
64 QAM 2/3 14.8
Table 13: FBMC PHY throughput data rate
The following figure shows the block error rate (BLER) versus signal to noise ratio (SNR) with the
different modulation and coding schemes of the FBMC implementation. With the assumption that
one block error causes one packet loss, it shows the test range that can be used for video quality
measurements given 6% is the most upper limit.
Figure 23: Relevant test range below 6% BLER with FBMC

With the results of the packet loss measurement, it can be assumed, that for an error-free video
transmission a packet error rate below 1% should be guaranteed. This leads to the following
minimum SNR values that should be reached.
Figure 24: Range of error-free video transmission below 1% BLER with FBMC
This graphical evaluation of SNR limits is also shown as a numerical result in Table 14.
MCS SNR for error free video (dB)
QPSK ½ < 5,8
QPSK ¾ < 7,8
16QAM ½ < 11,5
16QAM 2/3 < 12,7
16QAM 3/4 < 15,1
64QAM1/2 < 19,8
64 QAM 2/3 < 20,5
Table 14: FBMC SNR for error-free video
3.7.4 QoS parameters as a function of packet corruption
The measurement (Figure 25) shows the relationship between the packet corruption of a
transmission and the achievable QoS.
Figure 25: Picture failure points with different data rate conditions as a function of packet corruption

3.7.4.1 Interpretation of packet corruption measurements results
As expected, the measurement shows similar results to packet loss, but with a lightly better
resilience. This can be explained with the possibility that a corrupted packet may not be detected
because of the weak 16-bit checksum used in the TCP header and the fact that there is no further
checksum control in the HTTP layer (HTTP WebDAV). In this case, the corrupted IP packet is still used
to build the video data container on the receiver. This data container may then include a wrong byte,
but most data is valid and the impact of a wrong byte for the decompression may not be visible.
3.7.5 QoS parameters as a function of packet duplication
The results of the performed measurement are shown in Figure 26.
Figure 26: Picture failure points with different throughput data rate conditions as a function of packet
duplication
In general, the IP packet duplication can be eliminated by the operating system of the client. But with
throughput data rate limitations, an influence on the video QoS can be measured above 4% packet
duplication.
3.7.6 QoS parameters as a function of packet re-ordering
Measurements showed no influence of packet duplication up to an amount of 10%. The operating

system is able to handle this effect because of TCP. In the SPEED-5G setups packet reordering will be
no issue and in internet traffic re-ordering occurs less than 3% of the time.
3.8 Measurements on a UDP video transmission

For all simulations, picture freeze and peak signal to noise ratio is evaluated. It is expected, that due
to UDP (user datagram protocol) the measure results in a sporadic to continuous failure of the video
transmission (disordered pictures up to picture freeze), when the impairments exceed a tolerance
limit. In this task, the tolerance limits are measured. This allows us to specify a minimum channel
quality for video transmissions or adapt the video test system to the achievable channel rate in the
PoC 1, PoC 2, and PoC 6 setups.
This kind of internet protocol is used in broadcast video services like IPTV. Therefore MPEG transport
stream is used. One characteristic of this transport stream is, that the video stream is packetized in

small data units of 184 bytes. Clients receive 7 data units in one IP packet without any handshaking
by joining a multicast group. For this measurement, unicast addresses have to be used to ensure the
routing via the network simulator.
Because this transmission doesn’t provide any error handling and is not able to guarantee
consistency of the data, a forward error correction (FEC) system is in use to improve the transmission
quality. ProMPEG proposed to use an FEC system according to SMPTE-2022 ([13], [14]). The used FEC
settings cause a data overhead (additional throughput) of approximately 17%.
To evaluate with used systems measurements are performed with and without FEC.
3.8.1 QoS parameters as a function of end-to-end latency
3.8.1.1 Unidirectional Latency
Results:
Latency in PSNR PSNR

ms UDP ProMPEG
0 100 100
200 100 100
300 100 100
350 100 10
400 100 0
500 10 0
600 0 0
Table 15: QoS reduction due to latency (UDP/RTP)
3.8.1.2 Interpretation of latency measurements results
The measurement shows a difference between FEC and no error management. The effect goes equal
with a throughput limitation and looks like a limitation of the network simulator, because the latency
buffers may be not big enough for longer times. Anyway, the measured range should be sufficient
according to achievable latencies in HW platforms for PoC 1, Poc2, and PoC6.
3.8.2 QoS parameters as a function of packet jitter
The picture failure point as a function of the IP packet jitter was measured. No special jitter
correlation was given, but an additional delay of 50ms. No FEC was used.
Results:
Jitter in ms Picture Freeze PSNR in %
in %
1 0 0
5 0 0
7 0 90
10 100 100
Table 16: QoS with jitter

3.8.2.1 Interpretation of jitter measurements results
Actually, the jitter emulation shows a critical behaviour with permanent jitter on the UDP
transmission. The video test system loses the ability to decode the video with 10 ms jitter and needs
a period with less jitter to re-synchronize. This is a result of the system time clock recovery used with
this transport stream transmission.
3.8.3 QoS parameters as a function of packet loss
The measurement is performed with and without Forward Error Correction.
3.8.3.1 Without forward Error Correction (FEC)
Parameters:
 Video protocol: Transport Stream
 Transport stream throughput data rate 22Mbps
 IP Transmission protocol: UDP / RTP
Results:
in % in %
0 0 0 100
0,000001 0,0001 0 100
0,000010 0,0010 0 90
0,000100 0,0100 0 40
0,001000 0,1000 0 30
0,010000 1,0000 0 12
0,020000 2,0000 0 8
0,030000 3,0000 0 6
0,035000 3,5000 95 5
0,037000 3,7000 95 5
0,038000 3,8000 95 0
0,039000 3,9000 95 0
0,040000 4,0000 95 0
0,050000 5,0000 100 0
0,060000 6,0000 100 0
Table 17: QoS with packet loss (UDP no FEC)

3.8.3.2 ProMPEG Forward Error Correction (SMPTE 2022)
Parameters:
 Video protocol: Transport Stream
 Transport stream throughput data rate 22Mbps
 IP Transmission protocol: UDP / RTP
Results:
in % in %
0 0 0 100
0,000001 0,0001 0 100
0,000010 0,0010 0 100
0,000100 0,0100 0 100
0,001000 0,1000 0 100
0,010000 1,0000 0 100
0,020000 2,0000 0 80
0,030000 3,0000 0 45
0,035000 3,5000 0 41
0,037000 3,7000 0 39
0,038000 3,8000 0 38
0,039000 3,9000 0 37
0,040000 4,0000 0 37
0,050000 5,0000 0 28
0,060000 6,0000 0 20
Table 18: QoS with packet loss (UDP, proMPEG FEC)

Figure 27: QoS with packet loss (UDP, proMPEG FEC)
3.8.3.3 Interpretation of packet loss measurements results
The following figure shows one specific measurement, where the UDP transmission is interfered by
packet loss. Because one packet loss doesn’t cause a complete picture loss, only some areas of a
picture are affected. In comparison to an unaffected video (TCP in this setup) differences are marked
in a comparison window and the sum of differences are calculated to a reduced picture signal to
noise ratio (PSNR) down to approximately 40dB.
Figure 28: Picture quality reduction @ 0.01% packet loss (no FEC)
The same measurement with forward error correction shows a more robust behaviour, so that 3%
packet loss causes similar quality reduction as the 0.01% packet loss rate in a plain UDP/RTP
transmission.

Figure 29: Picture quality reduction @ 3% packet loss (proMPEG FEC)
The following graph compares the PSNR values in case of packet loss. UDP transmission without FEC
has a strong decline in quality and an abrupt picture freeze point at 3.1% whereas the transmission
with FEC (ProMPEG) has a more smoothly degradation without picture freeze up to 6% packet loss.
It shows the measurement results in function of SNR of a 64QAM 2/3 FBMC modulation.
Figure 30: Picture failure points in a UDP transmission (with and without FEC) as a function of SNR
3.8.4 QoS parameters as a function of packet corruption
The influence of bit errors in IP packets to the video quality was measured. The evaluated PSNR and
picture freeze points are displayed in the following graph.

Figure 31: Picture failure points in a UDP transmission (with and without FEC) as a function of packet corruption
Due to the transmission without return channel we see an early influence at low error rates (0.001%)
to the PSNR values. The picture is disordered although it does not freeze immediately. The FEC
system is able to shift this behaviour to higher tolerable error rates (1.5%).
3.8.5 QoS parameters as a function of packet duplication
Measurements showed no influence of packet duplication up to an amount of 9%.
3.8.6 QoS parameters as a function of packet re-ordering
Measurements showed no influence of packet duplication up to an amount of 10% because of RTP.

In the SPEED-5G setups packet reordering will be no issue and in internet traffic re-ordering occurs
less than 3% of the time.
3.8.7 Findings and summary
In performing the measurements, different behaviour in error handling have been observed:
 The measurements on the video application layer are quite slow acting and various in the
result because of the long buffering periods of the streaming systems and the random
coincidence of important data affected by the network interference.
 With the DASH/TCP system, no degradation of PSNR and MOS-V can be observed, meaning
that no deformed pictures are displayed. If IP transmission interference can’t be repaired by
retransmission of lost or corrupted packets the video freezes.
 With the TSoverIP/UDP system, IP transmission interference causes deformed pictures
measurable by reduced PSNR and MOS-V values. This effect can be healed up to some point
(1% packet loss) with the usage of Forward Error Correction (FEC-ProMPEG) for the price of
approximately 17% additional throughput capacity. Of course, too much interference causes
picture freeze also.
 The main bottleneck for IP video transmission through mobile networks is the throughput

capacity because all error handling and error correction systems need additional data rate to
do their job. If the capacity headroom is not sufficient, meaning that there is no standby data
rate available, the video quality reduces immediately and the QoE is down.
In the result of the impact of the PoC demonstrations, these measurements show that there are no
concerns but the available video data rates, which will have to be selected appropriately.
3.8.8 Tuning
The setup for PSNR/MOS-V measurement includes a picture comparison module with picture buffers
of 10 seconds of video. These buffers are necessary to synchronize the inputs for picture comparison.
The first adaption has to be made in the size of theses buffers because the UHD format contains 8
times the amount of data than HDTV. The second adaption results from the fact, that
UDP/RTP/TsOverIP has much lower latency by the system than the TCP/DASH streaming service
which has more processing and transmitting steps. For this reason an additional latency buffer of
approx. 25 seconds had to be inserted in the UDP/RTP/TsOverIP path.
A good choice of video sequences for testing had to be selected for the trials. Improper video
sequences include black, dark or unchanging pictures series, although the analyser can be adapted to
these conditions. But, in this case, picture freeze detection would be not sensitive enough to detect
transmission errors.
At least, the averaging of the measured values had to be increased, because the transmission errors
affect the video not in a constant way, but in critical or in less critical data partitions. Additional the
effect on the picture may be big or less, depending on the actual scene (e.g., if the colour of a picture
part changes forms white to light grey, the PSNR is higher than a change from black to light grey). In
the result, there is a very variating quality of the picture (with UDP transmission) depending on the
video content also.

4 RRM communication protocol evaluation
4.1 Motivation and Scope

This particular HIL test is intended to be the first trial of the proposed cRRM software architecture,
and in particular it’s messaging protocol. A pure software test has already been performed and is
reported in SPEED-5G Deliverable D4.3. The present test is intended to exercise actual
communication with hardware, in order to verify protocols and message sequences. This work is
exploitable for future development of the protocols, and useful in the perspective of PoC4 mainly as
a debugging tool.
4.2 Hardware-in-the-Loop platform

The hardware platform uses Wifi. The cRRM runs on a remote VM, and will receive messages from
the hardware and respond to them. The protocol is XML-RPC, a remote-procedure-call protocol. The
server and clients have been coded in python. The messages to be passed will be as defined on the
messaging procedures and primitives defined within the definition of the MAC and RRM interface
[15]. A separate, and physically remote VM, will run the cRRM.
The platform will use single-board PCs with Wifi dongles. These are not true SDRs, but some
operating parameters can be configured via standard Linux utilities, and these will be driven by
scripts to simulate SDR behaviour.
The basic test is to set up a radio link (e.g., video streaming); to cause channel impairment by
introducing interference; to have the radio notice the impairment via KPI monitoring; to request a
better channel by a request to the cRRM, and then actually switch to the new channel. A
fundamental constraint of the tests is that all KPI monitoring and hardware reconfiguration is done at
the application layer; in other words, these tests will run on standard PCs without any low-level
coding.
The main KPI of interest is the throughput on the wireless interface. This is monitored by a python
script which directly reads from /proc/net/dev, which is the low-level interface for such statistics
provided by the linux kernel (there are many other command-line utilities which could be used, such
as ifstat, iftop, tcpdump, bmon, slurm, tcptrack, …). This python script is integrated with the code
which communicates with the cRRM.
4.3 Parameters and statistics derived from hardware

The HIL test will be considered a success if reliable communication is established, and channel
switching is correctly triggered. But further to this, a test of latency of the entire client-server system
will be made. A block diagram and a photo of the Wifi test set-up are depicted below. All control is by
python scripts running purely at the application layer. Reading the current Wifi bitrate works by a
python script which reads from /proc/net/dev, which is a special device file maintained by the
linux kernel for reporting network statistics.

Figure 32: Block diagram of the setup for Wifi tests. The red rectangle indicates the removable absorbing
material used to impair the channel. The 60 GHz link is used to provide an unimpaired duplex channel for
reference purposes only. It is not strictly part of the current test.
Figure 33: The Wifi testbed. The channel is impaired by covering the tin shielding boxes with conducting foam.
The access point machine is at the lower left, with 2.5, 5, and 60 GHz dongles at the top left. On the
right are two client machines, one for 2.5 GHz and one for 5 GHz, with the dongles in shielding boxes.
The channel is impaired by covering these boxes with conducting foam. The 60 GHz link is at top
centre and in the middle an HDMI multiplexer for viewing all performed on a single machine.
4.4 Validation procedure and steps

1. Start cRRM server
2. Start client radios and video streaming service
3. Client radio starts sending channel quality reports (bits/s downlink speed). These reports
are sent once per second.
4. Impair the radio channel by blocking the signal path with conductive foam.
5. Data rate is reduced and the reduction and the reduction is detected by cRRM (the threshold

used was a reduction below 50% of the long-term average bit-rate)

6. cRRM performs computation to get suitable new band
7. New band data is sent to client radio
8. Client radio switches Wifi channel (i.e. 1 to 6, 6 to 11, or 11 to 1)
9. Client radio sends acknowledgement of channel change to cRRM
10. Log acknowledgement at cRRM
4.5 Test results

A fully successful test as described in the previous section has been conducted. As the screenshot
below demonstrates, upon impairing the 5 GHz channel by screening the AP, throughput drops from
around 1 MB/s to a few tens of kb/s, and even to zero. This was successfully reported to cRRM,
which sent a channel change request back to the client radio, which then performed the channel
change. Of course, this is not very useful in practice with Wifi as the radio technology, as the client
radios are dropped and will not reconnect until they rescan, and this means that the video stream
fails. But this is still considered a successful test, as the aim was purely to test the protocols for cRRM
communications, and these did perform as intended.
We emphasize the limitations of this set-up, as far as emulating a real system is concerned. The
shielding box does not model typical pathloss increases due to mobility very well, and channel
reselection, and device rescanning in Wifi is not at all user-friendly. But we nevertheless highlight
that the purpose of this test is not on the radio system performance, but simply on validation of the
cRRM communication protocol. In this respect the test was successful.
Figure 34: Screenshot of terminal of the demo during channel impairment. Note that the TX bitrate drops to
zero near the top of the screen (red arrow)..
4.6 Future work

The next steps will be to use the same cRRM and XML-RPC protocol in the several of the SPEED-5G
PoCs. This work will be covered in T6.3.

5 Conclusion
This deliverable provides the results of hardware-in-the-loop simulations performed as a preparation
for the project demonstration phase and as a validation of the eDSA concept by applying PHY
parameters and models coming from measurements made on the FBMC-MAC platforms.
In particular, this deliverable shows the gains one can expect from the hRRM concept applied on the
FBMC waveform, when dealing with heterogeneous traffic in dense small cell networks. This report
provides the necessary outlook on the project workflow by extrapolating the performance provided
by the project approach, showcased in PoC 1, considering large-scale networks which can’t be
captured in demonstration due to a limited number of hardware components. We have shown that
the combination of FBMC and hRRM can provide significant gains on the network performance, for a
traffic mixing eMBB, mMTC and URLLC.
Moreover, this deliverable also provides the necessary validation of key components of the project
testbed, before their integration in different PoCs. This is a very important step to maximise the
chance of a successful integration of these building blocks, and it has been reached by using different
flavours of the HIL approach. For instance, the FBMC-MAC operation on the 5 GHz band, which will
be part of PoC1 and PoC 6, has been anticipated by validating the channel selection algorithm which
will run on top of it. This has been done by using offline sensing measurement samples done on real
signals, which feed a system-level simulator, simulating static to dynamically changing interference
conditions on a number of channels where FBMC-MAC will have to operate. Also, the video QoS/QoE
test bench which will be used in PoC6 has been extensively tested, simulating the effect of the
communication impairments on IP packets. The network emulation is able to simulate various effects
like packet loss rates, jitter, or latency. This allows showing that except the video throughput which
needs to be appropriately tuned in PoC6, the video testbed is resilient to a very large span of
impairments. Finally, the communication protocol between cRRM and the cell has been validated by
means of a simplistic use case which has been run on off-the-shelf WiFi devices securing the
integration of cRRM in PoC1, PoC2, and PoC6.

References
[1] SPEED-5G, deliverable D5.2 “D5.2: MAC approaches with FBMC (final)”, June 2017
[2] Auer, P., Cesa-Bianchi, N., and Fischer, P. “Finite-time Analysis of the Multiarmed
Bandit Problem”, Machine Learning (2002), Volume 47, Issue 2–3, pp 235–256, May
2002
[3] SPEED-5G, deliverable D5.3 “Real-time protocol implementation”, September 2017
[4] 3GPP TS 26.114 V15.0.0 (2017-09) 3rd Generation Partnership Project; Technical
Specification Group Services and System Aspects; IP Multimedia Subsystem (IMS);
Multimedia Telephony; Media handling and interaction (Release 15)
[5] https://www.irt.de/no_cache/aktuell/news/view/datum/2017/05/18/5g-media-
initiative-gestartet/
[6] http://5g-xcast.eu/
[7] RFC 8216 HTTP Live Streaming (August 2017) https://tools.ietf.org/html/rfc8216
[8] ISO/IEC 23009-1 (May 2014) Information technology — Dynamic adaptive streaming
over HTTP (DASH) — Part 1: Media presentation description and segment formats.
[9] DASH Industry Forum http://dasif.org
[10] International Organization for Standardization ISO. Information technology, Coding of
audio-visual objects, Part 12: ISO base media ﬁle format, ISO/IEC 14496-12, 4th
Edition, 09/15/2012. International Organization for Standardization (ISO), Geneva,
2012.
[11] International Organization for Standardization ISO. Information technology, Coding of
audio-visual objects, Part 10: Advanced Video Coding, ISO/IEC 14496-10, 7th Edition,
05/01/2012. International Organization for Standard-ization (ISO), Geneva, 2012.
[12] International Telecommunication Union ITU-T H.265 (April 2015) Series H:
Audiovisual and Multimedia systems, Infrastructure of audiovisual services - Coding
of moving video - High efficiency video coding.
[13] RFC 6105 RTP Payload Format for Interleaved FEC https://tools.ietf.org/html/rfc6015
[14] Society of Motion Picture and Television Engineers SMPTE ST 2022-5 (2013) Forward
Error Correction for Transport of High Bit Rate Media Signals over Networks (HBRMT)
[15] SPEED-5G MAC-RRM messaging and primitive specifications: https://bscw.5g-
ppp.eu/sec/bscw.cgi/165155

Speed5g-D6.2 Hybrid Simulations With Hardware in The Loop v1.0

Diunggah oleh

Informasi Dokumen

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Speed5g-D6.2 Hybrid Simulations With Hardware in The Loop v1.0

Diunggah oleh

Hak Cipta:

Format Tersedia

Grant Agreement No.

quality of Service Provision and capacity Expansion through

D6.2: Hybrid simulations with hardware in the loop

Deliverable type Report

Document revision history

© 2015 - 2017 SPEED-5G Consortium Parties Page 2 of 53

© 2015 - 2017 SPEED-5G Consortium Parties Page 3 of 53

© 2015 - 2017 SPEED-5G Consortium Parties Page 4 of 53

3.7.5 QoS parameters as a function of packet duplication ............................................................ 41

© 2015 - 2017 SPEED-5G Consortium Parties Page 5 of 53

© 2015 - 2017 SPEED-5G Consortium Parties Page 6 of 53

© 2015 - 2017 SPEED-5G Consortium Parties Page 7 of 53

© 2015 - 2017 SPEED-5G Consortium Parties Page 8 of 53

This report is composed of 4 main parts.

© 2015 - 2017 SPEED-5G Consortium Parties Page 9 of 53

© 2015 - 2017 SPEED-5G Consortium Parties Page 10 of 53

1 Capacity expansions through FBMC MAC and hRRM

1.1 hRRM and FBMC MAC overall approach

1.2 Linkage to the SPEED-5G PoCs

© 2015 - 2017 SPEED-5G Consortium Parties Page 11 of 53

Figure 1: hRRM and FBMC MAC overall approach

1.3 Platform description

© 2015 - 2017 SPEED-5G Consortium Parties Page 12 of 53

LTE-A USRPs FLEX Boards

Figure 2: Hardware-in-the-Loop (HIL) platform

1.3.1 Hardware-in-the-Loop statistics

1.4 Evaluation results

1.4.1 Simulation scenarios

© 2015 - 2017 SPEED-5G Consortium Parties Page 13 of 53

Table 1: Summary of simulation parameters

© 2015 - 2017 SPEED-5G Consortium Parties Page 14 of 53

1.4.2 Simulation results

User Throughput (URLLC) User Throughput (eMBB)

User Throughput (mMTC)

Figure 3: User throughput

© 2015 - 2017 SPEED-5G Consortium Parties Page 15 of 53

Latency (Report download Time) (URLLC)

Figure 4: File/report download time

© 2015 - 2017 SPEED-5G Consortium Parties Page 16 of 53

2 HIL validation of the channel selection algorithm based on

2.1 Motivation and scope

2.2 Linkage to the PoCs

2.3 Algorithm and hardware in the loop descriptions

© 2015 - 2017 SPEED-5G Consortium Parties Page 17 of 53

Figure 5: FBMC-MAC superframe structure

© 2015 - 2017 SPEED-5G Consortium Parties Page 18 of 53

Freq (Hz) CCA CCA

If PER < Th, If PER < Th, CCANOK(1) ++

Figure 6: Channel selection algorithm overview

2.3.2 Hardware-in-the-Loop platform

© 2015 - 2017 SPEED-5G Consortium Parties Page 19 of 53

2.4 Validation procedure

© 2015 - 2017 SPEED-5G Consortium Parties Page 20 of 53

2.5 HIL simulation results

Figure 8 Scenario 1 – static interference conditions for channel selection validation

Figure 9: Scenario 2 – semi-static interference conditions for channel selection validation

© 2015 - 2017 SPEED-5G Consortium Parties Page 21 of 53

2.5.1.1 Simulation scenarios

2.5.2 Simulation results

© 2015 - 2017 SPEED-5G Consortium Parties Page 22 of 53

© 2015 - 2017 SPEED-5G Consortium Parties Page 23 of 53

© 2015 - 2017 SPEED-5G Consortium Parties Page 24 of 53

Figure 15: Average free channel access probability (normalized reward)