Network cards Pagers PDAs Photocopiers Portable Video games Pont of Sales terminals Printers Scanners Satellite phones Teleconferencing systems Televisions Set-top boxes VCRs Video phones Washers and dryers
camera
camcorder
TV
radio
game
PDA
makeup device?
army knife?
Data Streams
Distributed multimedia communication systems
data of discrete and continuous media are broken into individual units (packets) and transmitted.
Data Stream
sequence of individual packets that are transmitted in a time-dependant fashion. transmission of information carrying different media leads to data streams with varying features
Asynchronous Synchronous Isochronous
5
IP Ethernet
IP Ethernet
IPTV
IPTV applications
Homeowner entertainment Digital television On-demand video Business TV on desktop Distance learning Corporate communications Mobile phone television Video chat
M hnh IPTV
Software
Microsofts Windows Media Player Apple Computers QuickTime Real Networks RealPlayer Macromedias Flash Player VLC Media Player
Traditional components
Endpoints Infrastructure products
Gateways Gatekeepers Border controllers MCUs
What is endpoint?
Endpoint of the video architecture = the hardware components through which user send and receive video. Requirements:
Should share a common user interface Work on a wide variety of networks and protocols : IP,ISDN,V.35,SIP,SCCP and 3G. Quality, reliability and the ability to integrate with other communication systems (for example: instant message, web conferencing , existing video and audio feeds, IP telephony etc)
Endpoint category
Endpoint can be grouped into:
Personal systems. Group systems: Industry-or application-specific systems:
Examples of endpoint
Infrastructure products
Infrastructure products = components through which video are routed, merged and/or converted to other protocols in order for each endpoint to reach its destination. Category:
Gatekeeper Border controller Gateway MCU
Gatekeeper
Gatekeeper provides critical functionality to enable IP video communication. Include:
Alias Zone management Call management Authentication
TANDBERG Gatekeeper
Border controller
Allows H.323 traffic to traverse the firewall Create a specific rule for H.323 video Requirement:
Should be non-intrusive in regards to existing network design, allowing for easy deployment.
TANDBERG Border Controller
Enables video conferences between three or more persons. Like a bridge that interconnects calls from several sources Key feature:
Include non-blocking design, transcoding, rate matching, continuous presence and highest level encryption technologies
Development trend
Complicated functions are implemented in software to increase flexibility Highly distributed protocols, particularly SIP and its variants, tend to replace high-complexity and less scalable H.323 and the like P2P streaming emerges as a promising protocol architecture
Highly scalable since less centralization Mobilizing better resources of participants Disadvantages in: session discontinuity, dynamic connectivity, end-toend latency (due to e.g. swarming), security
IMS
Service Layer
Online Games IPTV Voice
IMS Core
Fixed Broadband Access Mobile Access network
Residential Network
CSCF=Call Session Control Function
Voice
QoS Control
IMS Core
Fixed Broadband Access Mobile Access network
Residential Network
NGN Backbone
UMTS
PSTN
DECT
Formed by nodes that: are heterogeneous mobile devices communicate solely over wireless links join and leave the network frequently Suitable for field applications no communication infrastructure needed possibly deploy real-time applications such as multimedia
Field applications Rescuing Activities Fire, accidental crashes Natural disasters: storms, earthquakes, flooding Collaborative Activities Working groups over the fields Environment protection Public events (festivals, sports, etc) Building blocks for other potential applications Inter-vehicle multimedia communications
Challenges Limitation in communication/computation resource (CPU, memory, bandwidth, etc) energy: nodes rely on battery Network connections are unstable erroneous Multimedia traffic heavy and bursty time-sensitive
Research themes
Redundant transmission significant overhead
Node 1
pa th
Node 0
Video source
Path diversity supported by routing protocols Path diversity supported by routing protocols
Pri ma ry
Node 3
ath Ex ten de dp
Node 2
Proactive intermediate node architecture video-cooperative forwarder Optimal scheduling (e.g. CoDiO) high complexity
Receiver Node 5
Node 4
MULTIPATH MULTIMEDIA TRANSPORT ARCHITECTURE OVERVIEW At the sender, raw video is first compressed by a video encoder into M streams. We call the coder a multistream coder. Then the streams are partitioned and assigned to K paths by a traffic allocator. These paths are maintained by a multipath routing protocol. When the flows arrive at the receiver, they are first put into a resequencing buffer to restore the original order. Finally, the video data is extracted from the resequencing buffer to be decode and display decoded displayed.
70
The general architecture for the multipath transport of real-time multimedia applications:
71
The general architecture for the multipath transport of realtime multimedia applications:
concurrent streaming
72
LC with ARQ
the two-path layered video transmission model with selective ARQ for base layer packets
73
Throughput
Congestion-distortion optimized scheduling Congestion-distortion optimized routing Capacity assignment
Traffic flows
Network layer
Link capacities
MAC layer
Link layer
Loss rate
0.5 dB
PSNR [dB]
2 dB
conventional forwarding...
ending host
relaying node
relaying node
ending host
relaying nodes simply forward packets as they arrive packets are treated equally regardless of their semantics only the sender can retransmit lost packets
proactive forwarding
Each relaying node transiently and proactively caches video packets retransmission can be made from the intermediate node instead of the sender each relaying node smartly selects packets to forward if the channel cannot accommodate all useless packets can be detected and removed to save energy and bandwidth
RtME
Rx
RtME
NACK Handler
cache-hit response queue
Warehouse
Upstream side NACK message cache-miss fresh packets
Tx
fresh queue downstream side cache-hit packets fresh packets NACK message
cross-layer communication
Lower layers
discarding packets
...
Node 0 Node h-1 Node h
...
Node n
Drop an I or P frame
time
NACK NACK
Locate and drop all dep. packets Id. of discarded frame
NACK
Tx
Rx
distortion
Adhoc Testbed
Introduction
Mobile WiMAX (802.16e) operation
Wireless mobile TV WiMAX defines only MAC/PHY of wireless link
Current
MBS to BS (base stations)
transport protocol - RTP / UDP / IP
83
Introduction (2)
Areas for improvement
Smoothen quality
during MSS movement during handoff in Multi-BS environment
Channel switching time Synchronization
Capacity improvements
Spectrum efficiency (number of TV channels)
85
87
88
Baseline System
H.264 / AVC
RTP/UDP/IP transport
RTP/UDP/IP transport
89
90
Issue #1 Synchronization
Difficult to achieve
OFDMA Frame synchronization problems because;
Each BS makes its own scheduling decision Each BS independently constructs its own OFDMA frame
OFDMA frames need to be the same across multiBSs in same geographic zone
Macro-diversity Reduced interference Smooth hand-off
91
95
Open issues
Application architectures
multimedia communication applications: video conference, IPTV, IMS, network gaming, 3D, etc middleware for software development (e.g. javabased)
Protocol design
cross-layer communication network resource utilization channel adaptive transmission error protection
96
Open issues
Wireless multimedia communications
QoS provisioning and architectures resource optimization joint source-channel coding
97
Open issues
Packet scheduling
resource-constraint/exploitation scalable multimedia streaming quality-resource optimization
Receiver enhancement
buffer management, feedback reporting error concealment techniques
98