ThingPark Architecture Overview PDF

Actility ThingPark
Architecture Overview for Orange
AGENDA
Introduction
General overview
ThingPark Solution - Functional architecture

Components
Roles
ThingPark Wireless Solution - System architecture

Overview
IP Flows
ThingPark Wireless Solution - High Availability

VPN
LRC
Databases
Proxy & Application Servers
Monitoring
Introduction
Functional architecture
System architecture
High availability
Monitoring
Introduction - General Overview

4
Full ThingPark IoT Framework
Connectivity for the Internet of

Things with LPWA Networks
Accelerating IoT application

development on open APIs
LoRaWAN core network

infrastructure including network
controllers (LRC servers) and multiple
hardware options for base stations
Open data cloud services with

standard based REST API. LoRaWAN
to ETSI M2M/OneM2M interworking
Online marketplace engine
The marketplace engine to accelerate

the distribution and provisioning of
your Internet of Things bundles
Actility helps service providers all the way from connectivity to online distribution of their IoT offers
Introduction - General Overview

End Device :
Mobile M2M terminal
Gateway of LRR :
Radio basestation for LoRa devices with a dynamic bitrate adaptation saves batteries and airtime whenever
possible.
Symmetrical bidirectional communications
LoRa Radio Controller :
This is the endpoint of the End Device session.

It does traffic routing, End Device management and automatic data rate control.
OSS or Backoffice :
Its the network partner portal, applications to manage the End Devices including Billing, Logging and
Monitoring are here implemented.
ThingPark Store :
Outlet store for B2B End Devices, Applications and services.
ThingPark Cloud :
It sets an excellent platform standard, that focus on data analysis and representation (e.g. alarming, charging,
monitoring applications)
Business Application Clients :
Directly connected to the LRC for the Tunnel Mode applications.
ThingPark High Level Architecture

Operator OSS & BSS
Management
On-Line Shops
ThingPark Marketplace &

Store
Vendor
Manager
Supplier
Manager
Marketplace
SNMP Traps,
REST APIs
Ordering APIs
Store APIs
User Portal
Application Servers
REST APIs
ThingPark Wireless Core NW & OSS
Operator
Manager
Connectivity
Manager
Network
Manager
Wireless
Logger
Spectrum
Analysis
Tool
LoRaWAN
Network
Servers LRCs
ETSI M2M REST APIs
ThingPark Cloud
Device Usage Record Supervision,

Manager Manager (UDR) Monitoring,
Alarms
ETSI M2M
Net GSCL
ETSI M2M
NSCL
Store (EShop)
Billing &
Charging
Network
Survey
Security
Server
System
Management
Platform (SMP)
LoRaWAN LRR (Gateway)

Radio Network LoRa Air Interface
Planning Tool Dimensioning Tool
Local GW (GSCL)
Any Device
LoRa Device
Optional
Introduction
System architecture
High availability
Monitoring
DMP Mapping
ThingPark Wireless Solution - Functional Architecture

Application(s)
End user
PORTAL
Charging (ogone)
Operator
Vendor
Suppliers
End user
Configuration
Supply chain (supplier)

End user
STORE
Orders
(sef-activation)
SMP
invoices
BILLING
Operat or/vendor/
suppl ier/subscribers DB
eShop DB
Invoi ce DB
Charging
(UDR)
LoRA device/base
stations DB
End user
Application(s)
End user
TWA
WLOGGER
End user
Statistic DB (monoDB)
Log DB DB
Provisioning / command / SSH
LRC
Application(s)
Table A/B/L/S
(file syst em)
SUPPORT
Reverse SSH
SLRC
IEC link
8
LoRa
device
LRR
PKI
configuration
Operator
RCA
PKI DB DB

LRR
An IoT base station, enables bidirectional communication between a device (ex : smoke
detector, temperature sensor) and a controller (LRC).
It incorporates the LoRa technology in order to relay a LoRa supported device and a
Backhaul application protocol with redundant parallel connection to ThingPark Network
servers.
It works with a dynamic bitrate adaptation in order to saves batteries and airtime
whenever possible.
Actilitys LRR includes third parties LoRa transceiver hardware, as well as Actility firmware
software controlling the transceiver and ensuring connectivity with the ThingPark LRC
network backend manager.
Actility ThingPark software is not limited to any specific hardware base stations and its
software is transparent to the hardware, thus enabling operators to deploy base-stations
from different vendors, promoting competition in the hardware base station market.

Application(s)
End user
PORTAL
Charging (ogone)
Operator
Vendor
Suppliers
End user
Configuration

End user
STORE
Orders
(sef-activation)
SMP
invoices
BILLING
Operat or/vendor/
eShop DB
Invoi ce DB
Charging
(UDR)
LoRA device/base
stations DB
End user
Application(s)
End user
TWA
WLOGGER
End user
Log DB DB
LRC
Application(s)
Table A/B/L/S
(file syst em)
SUPPORT
Reverse SSH
SLRC
IEC link
10
LoRa
device
LRR
PKI
configuration
Operator
RCA
PKI DB DB

LRC
The LRC is the backend core server controller :
It acts as a mediation function towards application servers.
LRR Management, End Device Management, Traffic routing between Application servers and end
Devices, select the best LRR for an End Device.
It implements download packet routings, intelligent dynamic base station selection (for optimized traffic
routing), device authentication, duplicate packets removal.
It provides redundant and scalable centralized management for all LRR base stations and wireless end
devices.
Possible IEC104 connections :

Primary (and secondary) connection are Ethernet
Back-up via 3G/4G modem

Application(s)
End user
PORTAL
Charging (ogone)
Operator
Vendor
Suppliers
End user
Configuration

End user
STORE
Orders
(sef-activation)
SMP
invoices
BILLING
Operat or/vendor/
eShop DB
Invoi ce DB
Charging
(UDR)
LoRA device/base
stations DB
End user
Application(s)
End user
TWA
WLOGGER
End user
Log DB DB
LRC
Application(s)
Table A/B/L/S
(file syst em)
SUPPORT
Reverse SSH
SLRC
IEC link
12
LoRa
device
LRR
PKI
configuration
Operator
RCA
PKI DB DB
SLRC VPNC
It ends IPSec VPN connections by implementing a StrongSwan server.
Strongswan was choosen by Actility for its capability to terminate more than 1000 VPN tunnels
It provide the following functionality :

DHCP server
Firewall
Router
RCA
RCA is a PKI (Public Key Infrastructure) providing a set of functionalities to trust to a
certificate signed by a certificate authority :
generate, manage and revoke X.509 certificates allocated to base stations and VPNC
publish certificates
generate and publish revocation lists
13

Application(s)
End user
PORTAL
Charging (ogone)
Operator
Vendor
Suppliers
End user
Configuration

End user
STORE
Orders
(sef-activation)
SMP
invoices
BILLING
Operat or/vendor/
eShop DB
Invoi ce DB
Charging
(UDR)
LoRA device/base
stations DB
End user
Application(s)
End user
TWA
WLOGGER
End user
Log DB DB
LRC
Application(s)
Table A/B/L/S
(file syst em)
SUPPORT
Reverse SSH
SLRC
IEC link
14
LoRa
device
LRR
PKI
configuration
Operator
RCA
PKI DB DB

SUPPORT
The Support server ends reverse SSH connections initiated by base stations (LRR)
To enable advanced troubleshooting of a LRR an access to its Linux console might be
required
There is 3 steps to mount the reverse SSH connection :
From the LRC, a request is sent to the LRR via a downlink with a chosen port
The LRR mounts the reverse SSH towards the SUPPORT server
From the SUPPORT server it is possible to access to the LRR using a SSH connection on the chosen port
A GUI functionality will be available on the platform and all these steps will be done
automatically on the background
15

Application(s)
End user
PORTAL
Charging (ogone)
Operator
Vendor
Suppliers
End user
Configuration

End user
STORE
Orders
(sef-activation)
SMP
invoices
BILLING
Operat or/vendor/
eShop DB
Invoi ce DB
Charging
(UDR)
LoRA device/base
stations DB
End user
Application(s)
End user
TWA
WLOGGER
End user
Log DB DB
LRC
Application(s)
Table A/B/L/S
(file syst em)
SUPPORT
Reverse SSH
SLRC
IEC link
16
LoRa
device
LRR
PKI
configuration
Operator
RCA
PKI DB DB

TWA
It implements the following functions :
Device manager : This module enables ThingPark Wireless subscribers (end users) to manage their
LoRa devices.
Network manager : This module enables network partners who deploy and operate base stations, to
manage their base stations.
Connectivity manager : This function is used by connectivity suppliers to define and manage its
connectivity plans.
Operator manager : This function is used by operators to manage its LPWAN network: Define device
profiles, define base station profiles, manage connectivity suppliers, and manage network partners.
UDR generation : billing file generated every month in order to charge the customer using an offer.
17

Application(s)
End user
PORTAL
Charging (ogone)
Operator
Vendor
Suppliers
End user
Configuration

End user
STORE
Orders
(sef-activation)
SMP
invoices
BILLING
Operat or/vendor/
eShop DB
Invoi ce DB
Charging
(UDR)
LoRA device/base
stations DB
End user
Application(s)
End user
TWA
WLOGGER
End user
Log DB DB
LRC
Application(s)
Table A/B/L/S
(file syst em)
SUPPORT
Reverse SSH
SLRC
IEC link
18
LoRa
device
LRR
PKI
configuration
Operator
RCA
PKI DB DB

WLOGGER
LoRa uplink and downlink frames logger.
This application enables subscribers to :

View and analyze end devices traffic
Visualize base stations with best link to each sensor
Monitor SNR levels, LRR and LRC connections and more.
It also enables the decoding of payload data and other application layer messages and
frames for some partner devices.
19

Application(s)
End user
PORTAL
Charging (ogone)
Operator
Vendor
Suppliers
End user
Configuration

End user
STORE
Orders
(sef-activation)
SMP
invoices
BILLING
Operat or/vendor/
eShop DB
Invoi ce DB
Charging
(UDR)
LoRA device/base
stations DB
End user
Application(s)
End user
TWA
WLOGGER
End user
Log DB DB
LRC
Application(s)
Table A/B/L/S
(file syst em)
SUPPORT
Reverse SSH
SLRC
IEC link
20
LoRa
device
LRR
PKI
configuration
Operator
RCA
PKI DB DB

SMP
The SMP stands for System Management Platform.
It includes three main capabilities :
Account managements: All accounts defined in the system are stored in the SMP. This
applies for administrator accounts (operators, suppliers, vendors) and end user
accounts (subscribers).
SSO procedures with third party applications.

Marketplace management: The Operator marketplace is defined in the SMP. The
definition of products and services is handled through a supplier account. The
definition of offers (self-activation and managed-activation offers) is handled through a
vendor account.
21
ThingPark Wireless Solution - Proxy

PROXY
For the LP-WAN 2 different proxies are used for different reasons. Both are based in the
open source NGINX software.
Load balancing Proxy (Internal proxy):
This Proxy is mainly needed as a load balancer between the LRC and the applications servers.
HTTPS Reverse Proxy (External Proxy):

The main function is to make the bridge to internet.
The reverse HTTPS proxy ends the HTTPS connections (coming from Internet or Extranet) initiated from
administrators, end users and third party applications.
22

HSM
HSM HW Security Module which enables the security cryptographic key material for the
ThingPark applications.
Validates integrity code then

sends encrypted JOIN response
HSM
23
Sends JOIN_REQUEST
with encrypted AppKey
Sends JOIN_ACCEPT message
LRC
Sends JOIN_REQUEST request

with DevEUI AppEUI
Introduction
System architecture
High availability
Monitoring
DMP Mapping
24
ThingPark Wireless Solution VMs per Sites
25
26
ThingPark Wireless Solution Security Zoning

Different zones with different trust levels are configured, with firewall and routing rules between them
The upper-RED zone: the untrusted Internet (i.e. the interface of the RED zone in front of untrusted).
The lower-RED zone: first protected zone, separated from upper-RED by application level HTTP proxy, and
from the RAN by SLRC function. Lower-RED is mapped to a VLAN and a specific IP subnet.
The radio access network (RAN) is an independent IPSec VPN within the upper-RED zone. The RAN IPSEc VPN
is terminated by the SLRC function, which interfaces to the Lower-RED zone.
The ORANGE zone: second protected zone, hosting the application servers. Queried by RED and query GREEN.
The GREEN zone: Third protected zone, hosting the databases and PKI. Queried only by ORANGE.
The MANAGEMENT zones: used for administration and supervision.
UPPER-RED
LOWER-RED
ORANGE
MANAGEMENT
27
GREEN
ThingPark Wireless Solution - Overview
28
ThingPark Wireless Solution - IP Flows

i1: Enrollment GUI
Browser (Actility
engineers)
i2: PKI configuration GUI
Site A / B
AS_RCA
i18: IEC 104 link sync

i12: CRL upload
i19: Device Ctx sync

i20: Rep ository resync
Site A / B
i6: IPsec IK E / MOBI KE
i14: IEC 104 link (LRR)
i7: ESP (protocol 50)
i15: LRR download (LRR)
Site A / B
Site A / B
i23: Downlink req
VPN (ipsec)
I8a: Ping (SLRC)
i17: NTP request (LRR)
SLRC
i17b: Ping (LRR)
i4: Downlink req

Third party AS
i21: Uplink frames report
LRC
i9: Reverse LRR admin

Fro m LRR (through th e VPN)
i11: DHCP request
LRR
PROXY_HTTP
i22: Uplink/dwlink report
Site A / B
i24: Provisioning req
OPTIONAL To be evaluated
According to the access network.
i25: LRR cmd req
PROXY_AS
i9a: Reverse LRR admin
i26: LRR admin
Site A / B
SUPPORT
i10: DNS req uest

DNS
i3: LRR admin
OPTIONAL
To be evaluated
According to the
access network.
29
SSH term. (Actility
engineers)
AS_WLOGGER
AS_TWA
AS_TWA

WebApp (Actility
ThingPark Marketplace)
Site A / B
a1: Marketplace sync
PROXY_HTTP
a5: ThingPark API
Site A / B
a15: Marketplace req
WebApp (Actility
ThingPark Marketplace)
AS_SMP
WebApp (Payment
Service)
a16: Email
a1b: Charging s ync
SMTP
a15b: Charging req
a17: HTTP API
WebApp (Payment
Service)
a18: Rep ository resyn c

a15c: Application / Supplier req
a1c: Application / Supplier sync

WebApp (Partners)
WebApp (Partners)
Site A / B
a6: ThingPark API
a19: Email
AS_BILL
SMTP
a20: HTTP API

Browser (Actility
engineers)
a2: GUI
a21: Rep osito ry resyn c
Site A / B
Browser (Partners)
a7: ThingPark API
a3: GUI
a22: Email
AS_STORE
a32: UDR download

a4: GUI
Browser (Customers)
SMTP
a23: HTTP API
Billing
a8: ThingPark API

Site A / B
LRC
Site A / B
PROXY_AS
a9: ThingPark API
a24: Email
SMTP
a25: SNMP Notification
SNMP
a10: Admin API
AS_TWA
a26: Provisioning req

a27: LRR cmd req
a28: LRR admin
a11: Shellinabox API
Site A / B
PROXY_AS
LRC
SUPPORT
a29: HTTP API
a12: ThingPark API

a13: ThingPark API
Site A / B
a30: HTTP API
AS_WLOGGER
Site A / B
30
a14: ThingPark API
Site A / B
AS_PORTAL
a31: HTTP API
AS_SMP
AS_BILL
AS_STORE
AS_TWA
AS_WLOGGER

Actility engineers
Site A / B
AS_RCA
d9: Cluster rep lication

d1: MySql connection
d10: Incremental State Trans fer

d11: Total state transfer
PROXY_HTTP
Site A / B
AS_SMP
PROXY_HTTP
Site A / B
AS_BILL
PROXY_HTTP
Site A / B
AS_STORE
Site A / B / C
SQL

d16: Metadata sharding management
PROXY_HTTP
PROXY_AS
PROXY_HTTP
Site A / B
AS_WLOGGER
Site A / B
AS_PORTAL

d13: Mongo connection (MG_NODE only)
PROXY_HTTP
PROXY_AS
31
Site A / B
AS_TWA

d15: Mongo connection (MG_NODE only)
Site A / B / C
MG_CONF
d17/d18: Data sharding management
Site A / B (MG_NODE)
Site C (MG_ARB)
MG_NODE /
MG_ARB
d20/d21: Mongo primary election
d19: Mongo data replication (MG_NODE only)

m3: Nagios NRPE
Site A / B
Browser (Actility
engineers)
m1: Nagios GUI
m4: Email
SMTP
AS_NAGIOS
m5: SNMP Notification
SNMP
m2: Nagios NRPE
Site A / B / C
All ThingPark
servers
AS_NAGIOS (Site A) toward all ThingPark servers (Site A / C)
AS_NAGIOS (Site B) toward all ThingPark servers (Site B / C)
c1: SSH/SCP
SSH term (Actility
engineers)
Site A / B / C
c2: DNS request

DNS
All ThingPark
servers
c1a: ICMP
NTP
c3: NTP request
c4: Yum u pdate
c5: SSH/SCP
All ThingPark servers

(Site A / C) toward
REPO (Site A).
All ThingPark servers
(Site B / C) toward
REPO (Site B)
Site A / B
REPO
32
c6: SSH/SCP
33
Introduction
System architecture
High availability
Monitoring
34

LRC 1/2
Primary site
LRR
LRC01.ORANGE.COM
Sync link
LRC01.ORANGE.COM
Secondary site
LRC02.ORANGE.COM
LRC02.ORANGE.COM
35

LRC 2/2
Primary site
LRR
LRC01.ORANGE.COM
Sync link
LRC01.ORANGE.COM
Secondary site
LRC02.ORANGE.COM
LRC02.ORANGE.COM
36

DATABASES
It contains all the administratives data : accounts, profiles
It contains all uplinks and downlinks.

Actilitity have estimated there will be more than 1500 writes/sec.
37

MONGODB
Replica set : It is a group of mongoDB node (mongod processes) that maintain the same data set. ThingPark replica
sets are composed of a primary node, a secondary node and an arbiter node.
Primary : The primary accepts all write operations from application servers. A replica set can have only one
primary. To support replication, the primary records all changes to its data sets in its oplog.
Secondary : The secondary replicates the primarys oplog and applies the operations to their data sets such that the
secondary data sets reflect the primarys data sets. If the primary is unavailable, the replica set will elect the
secondary to be primary.
Arbiter : The arbiter is also a mongod instance that does not maintain data sets. The purpose of an arbiter is to
maintain a quorum in a replica set by responding to heartbeat and election requests by other replica set members.
38

MARIADB 1/2
39

MARIADB 2/2
Maxscale : MaxScale is a layer 7 load balancer (read/write splitting). ThingPark using a single master / multi slave
configuration. Where read transactions can be load balanced on all nodes, and where write transactions are only
processed on the master node.
Galera replication : MariaDB Galera Cluster implements the following replications mechanism:
o Synchronous cluster replication on the SQL transaction commit. Galera replication happens at transaction
commit time by broadcasting the transaction write set to the cluster for applying.
o Incremental State Transfer (IST) resynchronization. Galera IST applies when a node rejoin the cluster. In IST, the
cluster provisions a node by identifying the missing transactions on the joiner and sends them only, instead of
the entire state.
o State Snapshot Transfer (SST) resynchronization. Galera SST applies when a node join or rejoin the cluster. SST
processes a full data copy from one cluster node (donor) to the joining node (joiner).
40
Introduction
System architecture
High availability
Monitoring
41
ThingPark Wireless Solution Monitoring

Two kind of monitoring exist on ThingPark Wireless:
Network alarms
Base stations and devices
LRC send network information from network to TWA nodes
Information is computed and alarms generated by TWA
System alarms
Active monitoring on the platform is done by Nagios
Scripts (plugins) are launched on VMs to check the health of system, databases,
applications.
42
ThingPark Wireless OSS/Monitoring

The Wireless OSS (TWA) is responsible to compute
LoRa devices alarms
Base stations alarms
Alarm triggering is computed based on BS reports and device reports.

These alarms can be
Shown in GUI (Network Manager)
Sent by e-mail
Sent to Orange Supervisor ? as SNMP traps
43
44
45
ThingPark Solution System Monitoring

The system monitoring is responsible to monitor and generate alarms for all ThingPark servers and
databases.
A Nagios instance exists per site and launch scripts on each VM instances to perform specific health
checks
Health checks depends on server type
One site monitors all VM of the site, and Nagios VM of the other site.
Third site is monitored by primarys Nagios
Alarms are accessible through:

The system monitoring interface (including history)
SNMP or email notifications (SNMP to Customer supervisor, including heartbeat traps every 5 minutes)
The NAGIOS standard MIB available
NRPE agent is installed on each VM

Triggered periodically (5 min) by Nagios
Launch specific check scripts and configuration
46
ThingPark Solution System Monitoring
47
Thank You

ThingPark Architecture Overview PDF

Diunggah oleh

Informasi Dokumen

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

ThingPark Architecture Overview PDF

Diunggah oleh

Hak Cipta:

Format Tersedia

Actility ThingPark

Architecture Overview for Orange

ThingPark Solution - Functional architecture

ThingPark Wireless Solution - System architecture

ThingPark Wireless Solution - High Availability

Introduction - General Overview

Full ThingPark IoT Framework

Connectivity for the Internet of

Accelerating IoT application

LoRaWAN core network

Open data cloud services with

Online marketplace engine

The marketplace engine to accelerate

Introduction - General Overview

Mobile M2M terminal

LoRa Radio Controller :

This is the endpoint of the End Device session.

Outlet store for B2B End Devices, Applications and services.

Business Application Clients :

Directly connected to the LRC for the Tunnel Mode applications.

ThingPark High Level Architecture

ThingPark Marketplace &

ThingPark Wireless Core NW & OSS

ETSI M2M REST APIs

Device Usage Record Supervision,

LoRaWAN LRR (Gateway)

ThingPark Wireless Solution - Functional Architecture

Supply chain (supplier)

Provisioning / command / SSH

ThingPark Wireless Solution - Functional Architecture

ThingPark Wireless Solution - Functional Architecture

Supply chain (supplier)

Provisioning / command / SSH

ThingPark Wireless Solution - Functional Architecture

Possible IEC104 connections :

ThingPark Wireless Solution - Functional Architecture

Supply chain (supplier)

Provisioning / command / SSH

ThingPark Wireless Solution - Functional Architecture

It provide the following functionality :

ThingPark Wireless Solution - Functional Architecture

Supply chain (supplier)

Provisioning / command / SSH

ThingPark Wireless Solution - Functional Architecture

ThingPark Wireless Solution - Functional Architecture

Supply chain (supplier)

Provisioning / command / SSH

ThingPark Wireless Solution - Functional Architecture

ThingPark Wireless Solution - Functional Architecture

Supply chain (supplier)

Provisioning / command / SSH

ThingPark Wireless Solution - Functional Architecture

This application enables subscribers to :

ThingPark Wireless Solution - Functional Architecture

Supply chain (supplier)

Provisioning / command / SSH

ThingPark Wireless Solution - Functional Architecture

SSO procedures with third party applications.

ThingPark Wireless Solution - Proxy

HTTPS Reverse Proxy (External Proxy):

ThingPark Wireless Solution - Functional Architecture

Validates integrity code then

Sends JOIN_ACCEPT message