Introduction
Video Coding
The objective of video coding is to compress moving images.
The MPEG (Moving Picture Experts Group) and H.26X are the major standards for video coding.
Basic Concept
Use interframe correlation for attaining better rate-distorion performance.
ITU-T
VCEG
H.261 (1990)
ISO/IEC
MPEG
H.263 H.263++ (1995/96) H.263+ (2000) (1997/98) H.264 MPEG-2 ( MPEG-4 (H.262) Part 10 ) (1994/95) MPEG-4 v1 (2002) (1998/99) MPEG-4 v2 (1999/00) MPEG-1 MPEG-4 v3 (1993) (2001)
1990
1992
1994
1996
1998
2000
2002 200
Moving images contain significant temporal redundancy successive frames are very similar
Intraframe Coding
Intraframe coding does not exploit the correlation among adjacent frames; Intraframe coding therefore is similar to the still image coding.
Interframe Coding
The interframe coding should include motion estimation/compensation process to remove temporal redundancy.
The amount of data to be coded can be reduced significantly if the previous frame is subtracted from the current frame.
Block-Matching
The MPEG and H.26X standards use block-matching technique for motion estimation /compensation. In the block-matching technique, each current frame is divide into equal-size blocks, called source blocks. Each source block is associated with a search region in the reference frame.
The objective of block-matching is to find a candidate block in the search region best matched to the source block. The relative distances between a source block and its candidate blocks are called motion vectors.
Video Sequence
X: Source block for block-matching Bx: Search area associated with X MV: Motion vector
The reconstructed reference frame
Results of blockmatching
Candidate block
Full-search algorithm
u
Candidate Block
Search Area
Three-step algorithm
-7 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 7
1 1
2 2 2 2
-6
-5
-4
-3
-2
-1
The first step involves block-matching based on 4-pel resolution at the nine location. The second step involves block-matching based on 2-pel resolution around the location determined by the first step. The third step repeats the process in the second step (but with resolution 1-pel).
Encoder side 1. Divide frame f into equal-size blocks. 2. For each source block obtained in step1, (2.1) Find its motion vector using the block-matching algorithm based on the reconstructed frame f-1. (2.2) Compute the DFD of the block.
3. Transmit the motion vector of each block to decoder. 4. Compress DFDs of each block. 5. Transmit the encoded DFDs to decoder.
Coder Control
Video in
DFD
Decoder
Transform/ Quantizer
Motion Estimator
Motion Vector
Decoder side 1.Receive motion vector of each block from encoder. 2.Based on the motion vector ,find the bestmatching block from the reference frame. That is, find the predicted current frame from the reference frame.
3.Receive the encoded DFD of each block from encoder. 4.Decode the DFD. 5.Each reconstructed block in the current frame = Its decompressed DFD + the best-matching block.
Encoded DFD
Encoded bitstream in
Motion vector
Bidirectional Prediction
The block-matching operations can be extended for bi-directional prediction, where a block in the current frame is estimated from a block in: (a) Previous frame (b) Future frame
Each block in a bi-directional frame is the average of a block in the previous frame and a block in the future frame.
A video sequences therefore may contain three different types of frames: (a) Intra frames, (I-frames) (b) Predicted frames (P-frames), (c) Bi-directional frames (B-frames)
The MPEG standards uses all the three types of frames as shown below.
Encoding order: I0, P3, B1, B2, P6, B4, B5, I9, B7, B8. Playback order: I0, B1, B2, P3, B4, B5, P6, B7, B8, I9.
Video Structure
Video standards such as MPEG and H.26X code video sequences in hierarchy of layers. There are usually 5 layers: 1. GOP (Group of pictures) 2. Picture 3. Slice 4. Marcoblock 5. Block
A GOP usually started with I frame, followed by a sequence of P and B frames. A Picture is indeed a frame in the video sequence.
A Slice is a portion in a picture. Some standards do not have slices. Some view a slice as a row. Each slice in H.264 is not necessary to be a row. It can be any shape containing integral number of macroblocks.
A Macroblock is a 1616 block. Many standards use Marcoblocks as the basic unit for block-matching operations. A Block is a 88 block. Many standards use the Blocks as the basic unit for DCT.
Temporal Scalability
We can use B frames for attaining temporal scalability.
B frames depend on other frames. No other frames depend on B frames. We can discard B frames without affecting other frames.
SNR Scalability
The base layer uses coaser quantizer for DFD coding. The residuals in the base layer is refined in the enhancement layer.