ECE 442 Lecture 1

Logistics

Lectures
Instructor: Dr. Vicky Zhao (vzhao@ece.ualberta.ca, 780-492-9484)
MWF11-11:50, MEC 4-1

Introduction to Multimedia Signal

Processing Introduction

Office hours: W 1-3PM ECERF W2-106, or by appointment

Labs
Lab instructor: Ms. Xiaohui Gong (gxiaohui@ualberta.ca)
Friday 2-4:50PM, ETL E5-012

Vicky Zhao
Electrical & Computer Engineering
University of Alberta, Edmonton

First lab is on Jan. 16, please bring a headphone/earphone

Course webpage: https://eclass.srv.ualberta.ca

All lecture notes, sample code and other handouts will be posted online

Acknowledgement: Prof. Min Wu and Prof. K. J. Ray Liu at UMD, Prof. Wade Trappe at Rutgers,
Prof. Y. Wang at PolyTech and Prof. Xin Li at WVU for sharing the slides. Many of the slides
here are borrowed from theirs.

Textbook:
Fundamentals of Multimedia, Z. Li and M. Drew, Prentice Hall, 2004

ECE442 Introduction to Multimedia Signal Processing

ECE 442 Introduction to Multimedia Signal Processing

Scope of ECE 442

Topics to be Covered

An introductory course on multimedia signal processing

Image, video and audio processing

Basics:
Acquisition and display, color systems, quantization, etc.

Emphasis

Image representation:
Color images, grayscale images and binary images

Provide fundamental background on multimedia signal processing

Coupled with assignments and labs for hands-on experience and
reinforcement of the concepts

Compression:
Basic building blocks
Image, video and audio compression standards

Prerequisites:

ECE 340 (former EE338, Discrete-time Signals and Systems)

Basics of probability and statistics
MATLAB (C/C++) programming

ECE 442 Introduction to Multimedia Signal Processing

Multimedia communications and networking

Quality of service (QoS) management

Multimedia security and digital rights management

Digital watermarking

ECE 442 Introduction to Multimedia Signal Processing

Grading

Assignments

Course Policy
15%

Labs

15%

Midterm Exam

20%

NO LATE SUBMISSION WILL BE ACCEPTED

Start early and plan wisely!
Inform instructor of special circumstances

Final Exam

Computer codes
You should write your own codes unless otherwise stated
DO NOT COPY others codes
Clearly state the code modules obtained elsewhere and consult the
instructor for permission to use beforehand

50%

April 21 (Tuesday) 9-11AM

ECE 442 Introduction to Multimedia Signal Processing

Independent work vs. discussions

Write up your solutions INDIVIDUALLY
Discussions with classmates on assignments and labs are encouraged

Feb. 11 (Wed.) in-class exam

Late submission

Academic integrity: cheating, plagiarism, fabrication of results,

ECE 442 Introduction to Multimedia Signal Processing

What is Multimedia?

When talking about multimedia, different people have

different viewpoints:
A PC vendor: a PC that has the ability to do audio/video playback (sound
capability, a DVD-ROM drive, audio/video cards, and multimediaenabled microprocessors)

Introduction to Multimedia
A consumer entertainment vendor: interactive cable TV with hundreds of
digital channels available, or a cable TV-like service delivered over a
high-speed Internet connection.
A Computer Science (CS) student: applications that use multiple
modalities, including text, images, drawings (graphics), animation, video,
sound including speech, and interactivity.

ECE442 Introduction to Multimedia Signal Processing

ECE 442 Introduction to Multimedia Signal Processing

How are Multimedia Data used?

Multimedia Research Topics

Digital audio/video editing and production systems

Electronic newspapers/magazines

On-line reference works: e.g. encyclopaedia, online games, etc

IPTV and video-on-demand

Video conferencing, teleconferencing (skype, Windows Meeting

Space by Microsoft, Apple iChat,)

Video surveillance and homeland security

Smart home: e.g., voice recognition

Online audio/video databases and sharing (P2P)

Distributed lectures, telemedicine, and much more

ECE 442 Introduction to Multimedia Signal Processing

Multimedia processing and coding: multimedia content analysis,

content-based multimedia retrieval, multimedia security,
audio/image/video processing, compression, etc.

Multimedia system support and networking: network protocols,

Internet, operating systems, servers and clients, quality of service
(QoS), and databases.
Multimedia tools, end-systems and applications: hypermedia
systems, user interfaces, authoring systems.

Multi-modal interaction and integration: ubiquity webeverywhere devices, multimedia education including Computer
Supported Collaborative Learning, and design and applications of
virtual environments.

ECE 442 Introduction to Multimedia Signal Processing

10

Light

Light is an electromagnetic wave

with wavelength of 350m to 780m stimulating human visual response

Introduction to Image Processing

A picture is worth a thousand words.

Acknowledgement: Prof. Min Wu and Prof. K. J. Ray Liu at UMD, Prof. Wade Trappe at Rutgers,
Prof. Y. Wang at PolyTech and Prof. Xin Li at WVU for sharing the slides. Many of the slides
here are borrowed from theirs.

from [Gonzalez]
ECE442 Introduction to Multimedia Signal Processing

ECE 442 Introduction to Multimedia Signal Processing

12

UMCP ENEE631 Slides (created by M.Wu 2007)

Sampling and Quantization

What we perceive as a grayscale image is a pattern of light intensity over a 2-D

plane (aka image plane)
Described by a nonnegative real-valued function I(x,y) of two spatial
coordinates on an image plane.
I(x,y) is the intensity of the image at the point (x,y).
An image is usually defined on a bounded rectangle for processing
I: [0, a] [0, b] [0, inf )

Color image
Can be represented by three functions:
R(x,y) for red, G(x,y) for green, B(x,y) for blue.

UMCP ENEE631 Slides (created by M.Wu 2001)

What is an Image?

Computer handles digital data

Sampling
Sample the value of the image at the nodes of a regular grid
on the image plane.
A pixel (picture element) at (i, j) is the image intensity
value at grid point indexed by the integer coordinate (i, j).

Is a process of transforming a real valued sampled image to

one taking only a finite number of distinct values.
Each sampled value in a 256-level grayscale image is
represented by 8 bits.
=> Stay tuned for the theories on these in future weeks.

0 (black)
13

ECE 442 Introduction to Multimedia Signal Processing

255 (white)

Quantization

14

ECE 442 Introduction to Multimedia Signal Processing

Examples of Sampling
UMCP ENEE631 Slides (created by M.Wu 2001)

UMCP ENEE631 Slides (created by M.Wu 2001)

Examples of Quantization

256x256

8 bits / pixel

64x64

4 bits / pixel
16x16

ECE 442 Introduction to Multimedia Signal Processing

2 bits / pixel

15

ECE 442 Introduction to Multimedia Signal Processing

16

Why Digital?

What is So Special about Multimedia?

Exactness
Perfect reproduction without degradation
Perfect duplication of processing result

Several MB for a raw 600*800 color image

Hundreds of MB/second for raw video sequences

Convenient & powerful computer-aided processing

Can perform rather sophisticated processing through hardware or
software
Even kindergartners can do some!
Easy storage and transmission
1 CD can store hundreds of family photos!
Paperless transmission of high quality photos through networks
within seconds

17

Multimedia data have a lot of redundancies

Spatial redundancy: neighboring pixels have similar values
Temporal redundancy in video: adjacent frames have similar content

Humans are robust to minor changes of multimedia data.

Small perturbations will not introduce perceptually noticeable difference

ECE 442 Introduction to Multimedia Signal Processing

The size of multimedia data is huge

In multimedia data, not all bits are equally important.

ECE 442 Introduction to Multimedia Signal Processing

Why Do We Need Image/Video Processing?

18

Image Representation

Enhancement and restoration

Remove artifacts and scratches from an old photo/movie
Improve contrast and correct blurred images

Transmission and storage

Images and Video can be more effectively transmitted and stored

Information analysis and automated recognition

Content analysis, multimedia database indexing
Surveillance

Evidence
Careful image manipulation can reveal information not present
Detect image tampering

8 bits/pixel

Security and rights protection

Encryption and watermarking preventing illegal content manipulation
Authentication and verification of multimedia content and the source
ECE 442 Introduction to Multimedia Signal Processing

24 bits/pixel
19

ECE 442 Introduction to Multimedia Signal Processing

1 bit/pixel20

Multimedia Compression

Multimedia Communications
Heterogeneous users/networks
Packet loss and bit errors
Time-varying available bandwidth
Time-varying delays

Color image of 600x800 pixels

Without compression
600*800 * 24 bits/pixel
= 11.52K bits = 1.44M bytes
After JPEG compression (popularly used
on web)
only 89K bytes
compression ratio ~ 16:1

256 kb/s

Receiver 2

Access
SW

Domain B

Domain A
Domain C

Access
SW

Movie
720x480 per frame, 30 frames/sec, 24
bits/pixel
Raw video ~ 243M bits/sec
DVD ~ about 5M bits/sec
Compression ratio ~ 48:1

Internet

Gateway

Ethernet
Source

1 Mb/s

Telephone
networks

Receiver 1

64 kb/s

Receiver 3

Library of Congress by M.Wu (600x800)

From Min Wu ENEE631
21 UMD

ECE 442 Introduction to Multimedia Signal Processing

ECE 442 Introduction to Multimedia Signal Processing

Summary

Multimedia Security
Embed patterns and content features
using a lookup-table

High embedding capacity/security via

shuffling

Introduction to multimedia and image processing

differentiate content vs. non-content

change (compression)

ECE 442 Introduction to Multimedia Signal Processing

content changed

24
from M. Wu UMD ENEE631

Today
Course overview

locate alteration

unchanged

from O. Wu EEL 6935 UFL

22

Next time
Color systems

ECE 442 Introduction to Multimedia Signal Processing

25