Research Proposal

PhD Research Project Proposal

By
Manoranjan Paul

1. Title
A New Approach to Very Low Bit-Rate Video
Coding Systems

2. Objective
To develop a very low bit-rate video coding system (MPEG4/H.26X
based architectures, algorithms, and applications) with scalable bit-
rate vs. video picture quality facilities for video-telephony,
videoconferencing, and video streaming to mobile phones and
handheld personal digital assistants (PDA), which have limited
processing and bandwidth capacity.

3. Statement of the Research Problem

Introduction
Video is incredibly stimulating in electronic commu-nications and
multimedia applications. Video can do a great deal to enhance a
presentation, illustrate a proper technique, or advertise a new product.
Video files are photographic images played at speeds that make it
appear to the human eye as if the images or frames are in full motion.
Such files can be extremely large because of the number of images
required to give the appearance of motion. Depending on the screen
size of the video file, a single second of uncompressed video running at
30 frames per second may require more than 30 MB of storage space.
In addition to storage, Bandwidth, the amount of data a communication
channel can carry, is also an obstacle to the delivery of video.

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (1 of 13) [4/29/2003 12:25:12 AM]

Research Proposal

Delivering video over the Internet is challenging because it needs huge

amount of storage capacity and a wide bandwidth.
Despite these challenges, more and more multimedia applications,
including digital videos, are disseminated everyday via the Internet
and mobile telephone networks. In order to be used effectively
however, video is often compressed for storage and transfer, and then
decompressed for use. Research into very low bit-rate digital video
coding face additional daunting challenges of meeting two inherently
conflicting requirements—reducing the transmission bit-rate as well as
increasing the image quality—that are sometimes inversely
proportional due to bandwidth of communication medium and
compu-tational complexity. So video compression with reasonable
image quality is a very active research topic currently. Compression
depends on two factors, (1) motion estimation— a process of
estimating the pixels of the current frame from the reference frame
and (2) motion compensation— a process of compensation for the
residual error after motion estimation. The following are three widely
used motion estimation processes:
1. Optical flow equation based methods,
2. Pel-recursive methods, and
3. Block matching methods.
The first two methods are computationally expensive. So block-
matching methods have been widely adopted by the international
digital video coding standards such as H.261/263/263+/263++
[6][7][8] and MPEG-1/2/4 [3][4][5]. H.26X series are widely used for
video-phoney and video-conferencing applications that require very
low bit-rate to acco-mmodate public switched telephone network
(PSTN).
An image is partitioned into a set of non-overlapped, equally spaced,
fixed size, small rectangular blocks (known as macroblocks); and the
translation motion within each block is assumed to be uniform.
Although this simple model considers translation motion only, other
types of motions, such as rotation and zooming of large objects, may
be closely approximated by the piecewise translation of these small
blocks provided these blocks are small enough. This observation
originally made by Jain and Jain, has been confirmed persistently since
then.
Displacement vectors for these blocks are estimated by finding their
best-matched same positioned block in the previous frame. The same
positioned block in the previous frame is not always the exact

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (2 of 13) [4/29/2003 12:25:12 AM]

 Research Proposal

representation of current block so pixels-by-pixels difference should be

calculated for error compensation to reconstruct the current block at
the decoder. In this manner, motion estimation and compensation are
significantly easier than that for arbitrarily shaped blocks. Since the
motion of each block is described by only one displacement vector, the
side information on motion vectors decreases. Furthermore, the
rectangular shape information is known to both the encoder and the
decoder, and hence does not need to be encoded, which saves both
computational load and side information. Most international video
coding standards use a block size of area 16×16 or 8×8.
Although very simple, straightforward and efficient, the block matching
motion compensation technique has its drawbacks. Firstly, it has an
unreliable motion vector field with respect to the true motion in 3-D
world space. In particular, it has unsatisfactory motion estimation and
compensation along moving boundaries. Secondly, it causes block
artifacts. Thirdly, it needs to handle side information, that is, it needs
to encode and transmit motion vectors as an overhead to the receiving
end, thus making it difficult to use smaller block sizes to achieve higher
accuracy in motion estimation.
All these drawbacks are primarily due to its simple model: each block is
assumed to experience a uniform translation and the motion vectors of
partitioned blocks are estimated independently of each other.
Unreliable motion estimation, particularly along moving boundaries,
causes more prediction error, hence reducing coding efficiency.
The block artifacts do not cause severe perceptual degradation to the
human visual system (HVS) when the available coding bit rate is high.
This is because, with a high bit rate, a sufficient amount of the motion-
compensated prediction error can be transmitted to the receiving end,
hence improving the subjective visual effect to such an extent that the
block artifacts are not perceptually annoying. However, when the
available bit rate is low, particularly and below 64 kbps, the artifacts
become visually unpleasant.
The assumption is that the motion within each block is uniform,
requires a small block size such as 16×16 and 8×8. A small size leads to
a large number of motion vectors, however, resulting in a large
overhead of side information. A study by [30] indicated that 8×8 block
matching performs much better than 16×16 in terms of decoded quality
due to better motion estimation and compensation. The bit rate
required for encoding motion vectors, however, increase significantly
(about four times), which may be prohibitive for very low bit rate

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (3 of 13) [4/29/2003 12:25:12 AM]

Research Proposal

coding since the total bit rate needed for both prediction error and
motion vectors may exceed the available bit rate. It is noted the when
the coding bit rate is quite low, say, for example of the order of 20
kbps, the side information becomes compatible with the main
information (prediction error) (see [31]).
In spite of its drawbacks, block matching is still by far the most
popular and efficient motion estimation and compensation technique
utilized for video coding, and it has been adopted for use by various
international coding standards. Some of the recent advancements in
block matching techniques are summarised below:
Hierarchical Block Matching—A set of different sizes for both the
original block and correlation window are used. The successive levels
with smaller window sizes and smaller displacement ranges are
capable of adaptively estimating motion vectors more locally.
Multigrid Block Matching—Uses a multigrid structure, another
computational powerful structure in image processing, to provide a
variable size block matching. With a split-and-merge strategy, the
thresholding multigrid block matching technique segments an image
into a set of variable size blocks, each of which experiences an
approximately uniform motion. It makes uniform motion within each
block more accurate than fixed-size block matching can do.
Predictive Motion Field Segmentation—To solve moving boundary
problems, predictive motion field segmentation technique make the
blocks involving moving boundaries have the moving field measured
with pixel resolution instead of block resolution. This method saves
shape overhead, and avoids side information with various
techniques.
Overlapped Block Matching—It enlarges blocks so as to make them
overlap. A window function makes motion estimation and
compensation so it achieves better performance than the
conventional block matching.

Motivation
After analysing the error surface between two successive frames in
video sequences, it has been observed that there are many
macroblocks, which have little or no motion. For this kind of
macroblocks, there is no need to consider all of the pixels in those
blocks. Considering only those intensity-changing pixels, further
compression may be possible with less computational time. It has also

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (4 of 13) [4/29/2003 12:25:12 AM]

 Research Proposal

been observed that motion vectors of a macroblock are sometimes

related to its adjacent macroblock because these macroblocks may be
the part of the same object. So motion vectors may be predicted from
the adjacent macroblocks. Furthermore it may be also concluded from
our error surface analysis, completed so far, that there exist
relationships between velocity of the objects in a video sequence and
motion vectors of the macroblock. Understanding such relationships
may help research into estimating the motion vectors and
compensation error more accurately than existing methods. During the
last decade, a number of signal processing applications have emerged
using wavelet theory. Among those applications, the most widespread
developments have occurred in the area of data compression. Wavelet
techniques have demonstrated the ability to provide not only high
coding efficiency, but also spatial and quality scalability features. Some
algorithms are developed by exploiting the features of the wavelet
transformation such as the Embedded Zero-tree Wavelet method
(EZW) (see [32]) and the Set Partitioning In Hierarchical Trees
(SPIHT) (see [33]) technique.

Review of Relevant Research and Theory

Some recent research partly addressed some of the issues stated in the
above motivation. [1] proposed four 128-pixel patterns to represent
the less moving macroblocks. For the very low motion video, the
number of intensity changing pixels is not as high as 128 pixels.
Otherwise for the very high motion video, pattern represent is not
fruitful to represent the macroblocks. [11] proposed eight 64-pixel
patterns for the same purpose. To do this, proposed method divides the
macroblocks into three categories according to their moving region.
The proposed patterns successfully represent the macroblock with low
motion. Using a fixed number of patterns does not fit well for all kinds
of video sequences because objects have different texture and
movements. [12] proposed a strategy for the prediction of the motion
vector from the adjacent macroblocks. Considering a moving region of
the macroblocks may lead the motion vector prediction a new
dimension. We know motion vectors of the macroblocks located on the
boundaries of the objects are not accurately estimated. [13]
successfully proposed a method for edge oriented block motion
estimation. [14] proposed a method, which overcome the local minima
problem of the error surface in tradition fast searching algorithm. [15] -
[19] and [21] proposed various kinds of fast search strategies. Fast
search strategy reduces the computational time for searching and also

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (5 of 13) [4/29/2003 12:25:12 AM]

 Research Proposal

may reduce image quality. [23][24] proposed feature based search

strategy. A new error criterion is proposed in [20]. An efficient
encoding method is described in [22]. Morphological operations
contribute a significant improvement in motion estimation. These kinds
of operations reduce the noise in frames of video sequence. See [9].
Wavelet transformation contributes a significant improvement in low
bit rate video coding and video image quality. [25] and [29] described
the processes how the video image quality can be improved using
wavelet transformation. [26]-[28] successfully applied the Embedded
Zero-tree Wavelet algorithm to improve the coding efficiency.

Research Hypothesis
Many of the existing video-coding standards, e.g., MPEG-1/2, tend to
employ block-based techniques because of their implementation
simplicity and also because they generally provide good results when
the bandwidth requirement is relaxed. This is, however, not the case
with low bit-rate block-based video coding such as in H.26X. The shape
of a moving object is generally arbitrary and may not necessarily be
aligned with the hypothetical grid structure created by the fixed-sized,
non-overlapping rectangular blocks, termed MacroBlock (MB) in the
coding standards. The typical size of a MB, being 16×16 pixels, leads to
a large number of blocks, some of which will contain only static
background, some will have moving objects and some a combination of
the two. [1] and [11] successfully introduced four 128-pixel and eight
64-pixels patterns respectively to represent the small moving region
macroblocks. In [11], MBs were classified according to the following
three mutually exclusive classes:

I ) Static MacroBlock (SMB)—Blocks that contain little or no motion;

II) Active MacroBlock (AMB)—Blocks that contain moving object(s) with
little static background;
III) Active-Region MacroBlock (RMB)—Blocks that contain both static
background and some part(s) of moving object(s).

By treating AMB and RMB alike, as is done in H.263/H.263+, leads to

coding inefficiencies [1]. In order to improve this efficiency, block size
may be reduced only to add additional information to be transmitted
due to the increase in the number of blocks [10].
Both [1] and [11] successfully addressed the above issue by

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (6 of 13) [4/29/2003 12:25:12 AM]

Research Proposal

segmenting each RMB into two regions using a fixed number of

predefined regular patterns. They respectively considered four, 128-
pixel and eight, 64-pixel predefined RMB patterns. Once the
segmentation process was complete, motion estimation/compensation
was then only performed on moving regions. Each SMB was skipped for
transmission (since they did not contain motion and could be copied
from the reference frame) and each AMB was treated exactly as
defined in H.263 standard, using motion estimation and compensation
techniques. The non-coding of SMB patterns and limiting the number of
RMBs to a prescribed set of patterns lead to an improved coding
efficiency.
Error surface analysis among frames may indicate a relation between
movements of the objects with respect to the background. Since there
are inter frames relations, error surface may be predicted from
adjacent error surfaces. In this way motion vector may be predicted
from the connected macroblocks.
Patterns representation clearly reduces the number of transmission
bits. By introducing more patterns, we can find more compression. But
the problem is that to search the suitable pattern from a large number
of pattern codebook, impose a large number of computational times. A
quick searching algorithm may reduce the time for pattern searching.
Another point is that 64/128 pixels fixed pattern cannot represent all
RMBs properly. So arbitrary shaped pattern may be effective. So the
following are my research goals:
Low bit-rate video coding algorithm focusing on moving region: Divide
the macroblocks of a frame into some classes of little or no motion, a
few motion and full motion blocks according to object movements;
we can reduce the motion vector calculating time and
compensational error. As the consequence we may get better
compression ratio as well as better image quality.
Apply error surface co-relation in video coding: Error surface analysis
explores a lot of relationships among objects’ motion, velocity,
texture, shape etc. After observing the error surface features we can
conclude the nature of the objects of that particular video sequence.
Apply artificial neural network in presenting arbitrary shaped moving
objects: If we know the shape of the objects in a macroblocks, we
approximate the macroblocks by using arbitrary shaped pattern. To
find out the shaped of the macroblocks, we may use artificial neural
networks to classify the macroblocks into some classes. By doing this
we can reduce the computational time of motion vector estimation

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (7 of 13) [4/29/2003 12:25:12 AM]

Research Proposal

and error compensation.

Apply wavelet transformation, EZW, and SPIHT algorithm: Now data
compression is using the wavelet transformation instead of the
discrete cosine transformation. It also improves the quality of the
images. There are some efficient algorithms for example, EZW,
SPIHT, etc., are used to encode the image data, and the potential for
exploiting this new technology will be another area that this research
project will examine.

4. General Research Time-Table

First Year Second Year Third Year

Literature
review
Error-surface analysis
Designing architectures & algorithms, and implementing a low bit rate video coding
system
Apply wavelet transformation, EZW, and
SPIHT
Explore ANN & alternative data mining
methods
Thesis writing

5. Research Already Undertaken

Literature review of error surface analysis, video and still image
compression, encoding and decoding techniques, general coding
technique (for example DPCM, Run length coding, Zero length coding,
Huffman Coding, Arithmetic coding, dictionary coding etc.), video and

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (8 of 13) [4/29/2003 12:25:12 AM]

Research Proposal

still image quantization techniques, all digital video coding standard

such as H.261 series, MPEG series, Wavelet and Fourier
transformation techniques, Embedded Zerotree Wavelet and Set
Partitioning In Hierarchical Tree algorithm, various video and still
image file format and some research works related of low bit rate
video coding
Error Surface analysis of frames difference for motion estimation,
nature of movements of the objects, objects involvements in a
particularly frame, change of pixels intensity vs. objects velocity etc.
Design a set of regular patterns for small moving region in a
macroblock: Depends on the observations, already found some
frequent patterns.
Develop a Variable Pattern Selection (VPS) algorithm and its
Extended (EVPS) version: An algorithm, which estimates the best
pattern for a particular video sequences.
Develop a Fast Pattern Searching (FPS) algorithm: It reduces the
time significantly compare to other algorithms for finding the best
patterns.

6. Publications
Journal Papers
A Variable Pattern Selection Algorithm for Low Bit-Rate Video
Coding (In preparation for Journal of Pattern Recognition)
A Fast Pattern Searching Algorithm with Controlled Bit-Rate. (to
be prepared for signal and image processing journal)

Conference papers
A Low Bit-Rate Video Coding Algorithm Focusing on Moving
Regions with Variable Regular Patterns (Accepted by the
International Conference on Signal Processing (ICSP-2002), Beijing,
China, 26–30 August, 2002) (see attachment)
A Variable Pattern Selection Algorithm with Improved Pattern
Selection Technique for Low Bit-Rate Video Coding Focusing on
Moving Objects (submitted to International Workshop on Intelligent
Knowledge Management Technique (I-KOMAT 2002, Podere
d’Ombriano, Crema, Italy, 16-18 September, 2002) (see attachment)

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (9 of 13) [4/29/2003 12:25:12 AM]

 Research Proposal

A Fast Pattern Searching Algorithm for Pattern Matching in Low

Bit-Rate Video Coding (to be submitted to The 6th World Multi
Conference on Systemics, Cybernetics and Informatics, SCI’02, July
14-18, 2002, Orlando, Florida, USA) (see attachment)

7. Applications of this research

Studio-based and desktop video conferencing
Surveillance and monitoring
Telemedicine
Computer based training
Video phone
Video over Internet

8. Conclusions
Low bit-rate in video coding has many applications, including video
telephony, Videoconference, live mobile phone telecasts, etc. Especially
when a device has no or little storage memory, very low bit-rate video
coding may change its usefulness tremendously. My objective is the
development of a very low bit-rate video coding system without
sacrificing the video image quality, which will add a new dimension in
the application of multimedia.

9. Ph.D. Supervision Team:

Joint-Supervisor: Prof. Laurence Dooley (50%)
Joint-Supervisor: Dr. Manzur Murshed (50%)

10. References
[1] Fukuhara, T., K. Asai, and T. Murakami, “Very low bit-rate
video coding with block partitioning and adaptive selection of two
time-differential frame memories,” IEEE Trans. Circuits Syst.

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (10 of 13) [4/29/2003 12:25:12 AM]

Research Proposal

Video Technol., 7, 212–220, 1997.

[2] Gonzalez, R.C. and R. E. Woods, Digital Image Processing,
Addition-Wesley, 1992.
[3] ISO/IEC 11172, International Standard, 1992.
[4] ISO/IEC 13818, MPEG-2 International Standard, Video
Recommendation ITU-T H.262, 1995.
[5] ISO/IEC N4030, MPEG-4 International Standard, 2001.
[6] ITU Telecom. Standardization Sector of ITU, “Video codec
for audio-visual services at p× 64kbits/s,” ITU-T Recommendation
H.261,1993.
[7] ITU Telecom. Standardization Sector of ITU, “Video coding
for low bitrate communication,” Draft ITU-T Recommendation
H.263, 1996.
[8] ITU Telecom. Standardization Sector of ITU, “Video coding
for low bitrate communication,” Draft ITU-T Recommendation
H.263, Version 2, 1998.
[9] Maragos, P., “Tutorial on advances in morphological image
processing and analysis,” Opt. Eng., 26(7), 623–632, 1987.
[10] Shi, Y.Q. and H. Sun, Image and Video Compression for
Multimedia Engineering Fundamentals, Algorithms, and Standards,
CRC Press, 1999.
[11] Wong, K.-W., K.-M. Lam, and W.-C. Siu, “An Efficient Low Bit-
Rate Video-Coding Algorithm Focusing on Moving regions,” IEEE
trans. circuits and systems for video technology, 11(10),
1128–1134, 2001.
[12] Gallant, M. and F. Kossentini, “An Efficient Computation-
Constrained Block-Based Motion Estimation Algorithm for Low Bit
Rate Video Coding,”
[13] Chain, Y.-L. and W.-C. Siu, “Edge orieted block motion
estimation for video coding,” IEEE Proc. –Vis. Image Signal
Process., Vol 144, No. 3, June 1997
[14] Chan, Y.-L and W.-C. Siu, “Reliable block motion estimation
through the confidence measure of error surface,” Signal
Processing 76(1999) 135-146, 1998
[15] Ganbari, M. “The Cross-Search Algorithm for Motion
Estimation,” IEEE Trans. On Communications, 38-7, 1990
[16] Po, L,-M. and W.-C. Ma, “A Novel Four-Step Search Algorithm

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (11 of 13) [4/29/2003 12:25:12 AM]

Research Proposal

for Fast Block Motion Estimation,”

[17] Liu, L.-K and E. Feig, “A Block-Based Gradient Descent
Search Algorithm for Block Motion Estimation in video Coding,”
1051-8215(96)05194-4.
[18] Lu, J. and M. L. Liou, “A Simple and Efficient Search
Algorithm for Block-Matching Motion Estimation,” IEEE Trans. On
Cir. And Sys. For Video Tech., Vol. 7, No. 2, 1997
[19] Chow, K.H.-K and M. L. Liou, “Genetic Motion Search
Algorithm for Video Compression,” IEEE Trans. On Cir. And Sys.
For Video Tech., Vol.3, No.62, 1993
[20] Fuldseth, A. and T. A. Ramstad, ”A new error criterion for
block based motion estimation,”
[21] Li R., B. Zeng, and M. L. Liou, “ A new Three-Step Search
Algorithm for Block Motion Estimation,” IEEE Trans. On Cir. And
Sys. For Video Tech., Vol.4, No.4, 1994
[22] Kossentini, F. and Y.-W. Lee, “Computation-Constrained Fast
MPEG-2 Encoding,” IEEE Signal Processing Letters, Vol. 4, No. 8,
1997
[23] Chan, Y.-L and W.-C. Siu, “A Feature-Assisted Search
Strategy for Block Motion Estimation,” 0-7803-5467-2/99 IEEE
[24] Chan, Y.-L and W.-C. Siu, “An Efficient Search Strategy for
Block Motion Estimation Using Image Features,” 1057-
7149(01)06035-3
[25] Fan, G., and W-K Cham, “Postprocessing of Low Bit-Rate
Wavelet-Based Image Coding Using Multiscale Edge
Characterization,” IEEE Trans. On Cir. And Sys. For Video Tech.,
Vol.11, No.12, 2001
[26] Ramaswamy, V.N., K.R. Namuduri, and N. Ranganathan,
“Context-based Lossless Image Coding Using EZW Framework,”
IEEE Trans. On Cir. And Sys. For Video Tech., Vol.11, No.4, 2001
[27] Ramaswamy, V.N., K.R. Namuduri, and N. Ranganathan,
“Performance Analysis of Wavelets in Embedded Zerotree-based
Lossless Image Coding Schemes,” IEEE Trans. On Sig. Proc.,
Vol.XX, No.Y, ZZZZ
[28] Jianhua, W.U., C. Olivier, and C. Chateller, “Embedded
Zerotree Runlength Wavelet Image Coding,” 0-7803-6725-1/01
2001 IEEE
[29] Shipeng, L. and W. Li, “Shape-Adaptive Discrete Wavelet

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (12 of 13) [4/29/2003 12:25:12 AM]

Research Proposal

Transforms for Arbitrarily Shaped Visual Object Coding,” IEEE

Trans. On Cir. And Sys. For Video Tech., Vol.10, No.5, 2000
[30] Chan, M. H., Y.B. Yu, and A. G. Constantinides, “Variable size
block matching motion compensation with applications to video
coding,” IEEE Proc., 137(4), 205-212, 1990
[31] Lin, S., Y. Q. Shi, and Y.-Q. Zang, “An optical flow-based
motion compensation algorithm for very low bit-rate video
coding,” Proc. 1997 IEEE Int. Conf. Acoustics, Speech Signal
Process., 2869-2872, Munich, Germany, 1997; Int. Imaging Syst.
Technol., 9(4), 230-237, 1998
[32] Shapiro, J., “Embedded image coding using zerotrees of
wavelet coefficient,” IEEE Trans. Signal Process., 3445-3462,
1993
[33] Said, A. and W.A. Pearlman, “A new fast and efficient image
codec based on set partitioning in hierarchical trees,” IEEE Trans.
Circuits Syst. Video Technol., 243-250, 1996

http://www.gippsland.monash.edu.au/~ranjan/PhD-Research-Proposal-of-Manoranjan-Paul.htm (13 of 13) [4/29/2003 12:25:12 AM]