NETWORKING MIDDLEWARE FOR VIRTUAL REALITY

BASED SRT SIMULATOR

--------------------------------------------------------------------------------------------------------------------------------------------------------
NETWORKED VIRTUAL ENVIRONMENTS

1. Several computers running software's that communicate with each other.

2. The distributed interactive virtual environment could either be a game, virtual social space or a 3D simulation.
3. Best known networked simulator is a flight simulator which helps train pilots at a much lower cost.

A Boeing Flight Simulator designed as a cockpit

SCOPE OF INTERACTION

Immersive virtual reality applications require high-end hardware, a range of interaction devices and software
developed using different programming languages, running on multiple computers and operating systems. Some
software nodes in the distributed system could model logical aspects of the virtual environment such as pedestrian
or traffic systems and run a simulation while others could model the geometric representations and material
properties of the environment.
1. The system should render graphics and audio in real-time.
2. It should also provide any haptic feedbacks to the users in real time for an immersive experience.
3. Accept input from interaction devices such as joysticks, data gloves, audio input, haptic and infrared sensor based
input devices such as Leap motion.
4. Manage object interactions and physics collisions.
5. Create a consistent and continuous representation of the virtual environment.
ROLE OF NETWORKING MIDDLEWARE

1. Provide cross-platform abstractions for networking, provide client libraries in multiple programming languages.
2. Data-coupling between multiple clients/simulations.
3. The middleware should handle the low level details such opening the sockets and maintaining persistent
connection between multiple clients (client/server) or using multicast to transfer data.
4. Send large files are sent in multiple data packets using UDP etc., verify and retry the transfer if required.
5. Handle details such as publishing the relevant messages only to interested clients.
6. Security features such as using SSL or password authentication.
7. Discovery of broker/server IP address.
ISSUES IN NETWORKING FOR VIRTUAL REALITY BASED SRT
SIMULATOR

1. Latency and bandwidth

2. Synchronized or unsynchronized
3. Critical data
PUBLISH/SUBSCRIBE MIDDLEWARE

1. Publish/Subscribe design enables loosely coupled interactions with is a good fit for asynchronous simulation
scenarios where each individual simulation is synchronized by the data received.
2. Helps offload responsibilities of networking from simulation modules to the broker.
3. Publishers push their updates to the broker and the subscribers receive the updates for which they registered.
4. Provides design for filtering of data such that only relevant updates are received by a particular module.
5. Better opportunity for scalability than client-server model as new modules can be added that publish/subscribe
to a new or existing topic.
ECLIPSE MOSQUITTO – MQTT BROKER

1. Designed to be lightweight for IoT devices, it is an open source broker implementation of MQTT connectivity
protocol.
2. Ideal for low bandwidth, high-latency environments for Machine-to-machine(M2M) communications.
3. Topics can follow REST architectural style (pedestrian/2/action_update, pedestrian/2/position_update) so the
messages published can be targeted to a particular model in the simulation.
4. Multi-threaded design can allow each component to publish and subscribe to its own topic independently.
ECLIPSE PAHO + M2MQTT4UNITY – MQTT CLIENT

1. Eclipse Paho is an open source client implementation of MQTT connectivity protocol.

2. Client libraries are available for multiple platforms including C/C++(Posix and Windows) and Java.
3. M2MQTT4Unity is an open source client implementation of MQTT for Unity.
SCENEJS FOR INTEROPERABILITY

1. SceneJS defines a 3D scene graph representation in JSON format.

2. Created for WebGL applications and hence focuses on compactness.
3. Extensible, therefore it can be extended to define pedestrian and traffic simulations.
4. Other options are Babylon.js, X3D (XML based but the same structure could be followed).
5. Unity 5 provides a JSON utility class to convert objects to and from JSON format.
nodes: [ { type: "translate", id: "firstBoxPos", x: -3.0, y: 1.0, z: 5.0, nodes: [ { type: "geometry/box" } ] }, { type: "translate", y: 1.0, z: 5.0, nodes: [ { type: "geometry/box" } ] } ]

SCENEJS – JSON EXAMPLE

nodes: [
{
type: "translate",
id: "firstBoxPos",
x: -3.0,
y: 1.0,
z: 5.0,
nodes: [
{
type: "geometry/box"
}
]
}
]
COMPONENT DIAGRAM
cmp

Pedestrian Simulator

Simulator : Java

Operating System : Linux (Ubuntu)

Pedestrian (SceneJS)
PAHO : MQTT CLIENT

Graphics System
Virtualization : VIrtual Box
Message Broker
Visualization : Unity3d

MOSQUITTO : MQTT BROKER

Operating System : Windows

M2MQTT : MQTT CLIENT Operating System : Linux (Ubuntu)

SceneJS

Renderer : VR Glasses
Virtualization : VIrtual Box

Renderer : Deep Space Traffic Simulator

Simulator : C++

Operating System : Linux (Ubuntu)

Traffic (SceneJS)
PAHO : MQTT CLIENT

Virtualization : VIrtual Box

EXPERIMENTAL RESULTS

• Experiments were conducted using Unity, MQTT broker and a client

• MQTT broker and the client received the position updates in real time from unity