International Conference on Intelligent Information Hiding and Multimedia Signal Processing

A H.264 Based Joint Source Channel Coding Scheme over Wireless Channels
Xuejuan Gao, Li Zhuo, Suyu Wang, Lansun Shen Signal & Information Processing Lab, Beijing University of Technology, Beijing, 100022, China snow03129@emails.bjut.edu.cn Abstract
Joint source channel coding (JSCC) is an effective method which trade off the efficiency and the quality of the video transmission. In this paper, a new ratequality (R-Q) model is first proposed to represent the coding characteristics of H.264 instead of ratedistortion (R-D) model. Then, the end-to-end video transmission distortion is analyzed and an adaptive JSCC over wireless channel based on the R-Q model of the H.264 and the error protection characteristics of the Turbo code is proposed, which can optimize the rate allocation of the available network bandwidth between source coding and channel coding according to the current status of the wireless channels, so as to improve the robustness. The experiment results indicate that, compared with the scheme using the fixed channel coding rate, under the same channel conditions, our proposed JSCC scheme can greatly improve the transmission robustness and achieve better reconstructed video quality at the receiver. channel coding method, they all optimize the source and channel rate allocation based on the source R-D model and channel transmission distortion model in essence. In this paper, instead of using R-D model to perform rate control as many existing methods do, we propose a new rate-quality (R-Q) model. If use PSNR (Peak Signal to Noise Ratio) as the measurement metric, we find that there is a conic relationship between video quality Q and log(R), QP and log(R), no matter it is an I frame or a P frame. So we first set up a novel R-Q model of H.264. Then the end-to-end video transmission distortion is analyzed and an adaptive JSCC over wireless channel based on the R-Q model and the error protection characteristics of the Turbo code is proposed, which can optimize the rate allocation of the available network bandwidth between source coding and channel coding adaptively according to the actual condition of wireless channel, so as to improve the robustness of the video transmission. The experiment results indicate that our proposed JSCC scheme can greatly improve the transmission robustness and achieve better reconstructed video quality compared with the JSCC scheme using the fixed channel coding rate.

1. Introduction
Joint source channel coding (JSCC), an effective coding method, can minimize the end-to-end transmission distortion by optimizing the bit rate allocation between source encoder and the channel encoder or between different parts of source encoder. JSCC has been studied for many years. Therefore, there are a lot of literatures [1-3]: ZhiHai He et al proposed a joint source channel rate-distortion (R-D) analysis method [4], where an analytic solution for adaptive intra mode selection and joint source-channel rate control based on the R-D model has been developed. Zhang et al developed a MPEG-4 PFGS coding method based power-optimized joint sourcechannel coding method over wireless channel [5], which reduces total power consumption based on the proposed end-to-end power-optimized architecture. Although those schemes adopted different source or

2. Proposed R-Q model of H.264 encoder

H.264 coding standard supports two kinds of coding modes: Intra and Inter [6], which render quite different R-Q behaviors. For the Inter coding mode, the prediction of current coding MB will refer to one or several previous frames, while the prediction for the Intra coding mode only refers to the adjacent MBs in the current frame. To analyze the R-Q performance of the H.264 encoder, we performed a lot of experiments on various video sequences with different motion characteristics and different format. Each test video sequence includes 300 frames. While testing the R-Q behavior of I frame, we set all frames as I frame. For the P frame, we adopt GOP as the coding structure. That is, the first frame of

978-0-7695-3278-3/08 $25.00 2008 IEEE DOI 10.1109/IIH-MSP.2008.30

683

each GOP is I frame, and the others are all P frames. If we use PSNR as the reconstructed video quality measurement metric, and calculate the logarithm value of the rate R, we can find that, there is a linear relationship between QP and Q while log(R)-Q and log(R)-QP curves approximately render a conic relationship. The relationship between QP and Q, log(R) and Q, as well as log(R) and Q are shown in Figure 1. From Figure 1, we can see that we can use one simple conic model to describe the coding characteristics of H.264. Therefore, three mathematical models are proposed as follows to describe the coding performance of H.264: QP = p1 [log( R )] 2 + p 2 log( R ) + p3 (1) Q = p 41 [log( R )] 2 + p5 log( R ) + p 6
Q = p7 QP + p8 QP + p9
2

(a) QP-Q curve

(b) log(R)-Q curve

(c) log(R)-QP curve

Where QP represents quantization step, R coding rate and Q the reconstructed video quality (i.e.PSNR value) and pi, i=1,2,,9 are model parameters. The three models are named together as R-Q model in this paper. To prove the accuracy of our proposed R-Q model, we performed experiments with different video sequences. Figure 2 shows the comparison of model data and actual tested data of Akiyo sequence (CIF format). It can be seen clearly from Figure 2 that the proposed R-Q model can exactly describe the relationship among Q, R and QP. We have performed the test over a wide range of video sequences also, and the proposed R-Q model has been found to hold.

Figure 1. R-Q models of P frame of foreman sequence (CIF)

(a) QP-Q curve

(b) log(R)-Q curve

3. R-Q model based JSCC scheme

The basic idea of JSCC can be formulated as the following optimal model [4]: * * RS , RC = arg min[DSC ] s .t . R s + R c R (2) Where DS+C represents the end-to-end distortion, R is bandwidth, RS is bit rate of source encoder and RC is bit rate of channel encoder. DS+C is caused by two reasons: source distortion and channel distortion. Due to the randomness of channel bit errors, DS+C is actually a random variable. If there is no channel bit error, DS+C is only caused by source distortion. Figure 3 shows our adopted JSCC framework, which uses H.264 standard as source coding method and Turbo code as channel coding method. Firstly, the optimal rate allocation module will find the optimal bit rate RS* and RC* which can minimize the DS+C according to the current conditions of the wireless channels. Then, H.264 encoder and Turbo encoder perform coding separately with the optimal bit rate RS*and RC*. At the receiver side, the coded video
(c) log(R)-QP Curve

Figure 2. Comparison of R-Q curves

Original picture H.264 encoder Turbo encoder

Rate allocation module Channel condition Reconstruction picture H.264 decoder Turbo decoder

wireless channel

Figure 3. The architecture of our JSCC stream transmitted over wireless channel is decoded to achieve the reconstructed video.

3.1. The end-to-end distortion analysis

In this paper, we use DS+C(n) to describe the end-toend distortion of frame n. Because H.264 uses

684

prediction coding mode, channel bit error will cause error propagation. Therefore, error concealment methods are usually exploited to restrict error propagation. Let Pe be the channel bit error rate (BER), the expected average of DS+C(n) is defined as: (3) DS +C ( n ) = ( 1 Pe )DS ( n ) + Pe Der ( n ) where DS(n) represents the distortion between the reconstructed picture and the original picture without bit error, Der(n) represents the distortion between the reconstructed picture after error concealment and the original picture if the channel errors occur. In terms of the characteristic of Turbo code, Pe can be formulated as follows [7]:
Pe 1 = 2 r Eb e rN 0
rE b N0

~ ~ as F ( n ) , where j is the position n ,iF ( n 1, j )e of pixel i in the n-1 th frame according to the motion ( n ) is the quantization value of vector (MV) and e prediction error. Otherwise, if there are bit errors, we use the same error concealment method as I frame. Then the expected average of DS+C(n) of P frame is given by:
2 ~ ( n 1, j ) F D S + C ( n ) = D S ( n ) + ( 1 Pe ) E F ( n 1, j )

( n 1, j )} represents the reference frame Note that {F 2 ~ ( n 1, j ) F of motion compensation and E F ( n 1, j )

( n ,i ) F ( n 1,i ) + Pe E F

}+ P E {F ( n ,i ) F ( n 1,i ) }
2 e

(9)

(4)

Note that Eb is the power of every signal, N0 is yawp power of channel and r is bit rate of channel coding. If we assume F(n,i) to be the original value of pixel n, ithe construction value of i in the nth frame, F ~ pixel i at the encoder side and Fn,i the reconstructed pixel i at the decoder side, then DS(n) and Der(n) can be given respectively:
( n,i ) DS ( n ) = E F ( n ,i ) F

the mean square error of reference frames of motion compensation between the encoder end and the decoder end, which can be obtained in reference [4].

3.2. The optimal rate allocation of JSCC

If we assume the frame rate of H.264 encoder is F (fps) and use second as the basic rate allocation unit, the DS+C in (2) can be given by:
DS +C = 1 F

2 ~ Der ( n ) = E F ( n ,i ) F ( n ,i )

D
n =1

S +C

(n)

(10)

(5)

Due to the different coding mode of I frame and P frame, we need to analyze them separately. For I frame, if there are bit errors occurred, we use the MB at the same position in the n-1 th frame to replace the corrupted one. Then we will have:
2 ~ Der ( n ) = F ( n ,i ) F ( n 1,i )

2 ~ ( n ,i ) + F ( n ,i ) F = E F ( n ,i ) F ( n 1,i )

(6)

Because of the uncorrelated characteristics of source distortion and channel distortion, (6) can be changed into:
~ ( n ,i ) |2 } + E{| F ( n ,i ) F Der ( n ) = E{| F ( n ,i ) F ( n 1,i ) |2 } ~ ( n ,i ) F (7) = DS ( n ) + E{| F ( n 1,i ) |2 }

We can obtain DS+C(n) from (8) or (9) according to different coding mode. Apparently, (2) is an optimization problem with the restriction conditions, whose optimal answers are RS*and RC*. Let R be the total bit rate of channel bandwidth. If consider the relationship among the coding rate r of Turbo code, RS and R : r=RS/R , then RS = r R RC = 1 r R . Therefore, (2) is transformed to find only one optimal r* to minimize the DS+C, which can use Lagrange optimization algorithm to search.

4. Experiment results and analysis

To illustrate the effectiveness of our proposed JSCC scheme, we performed experiments on a lot of different video sequences (both CIF and QCIF format) under various channel condition. The compared method is fixed channel coding rate (r=1/2) method. In this paper, the channel coding rates of Turbo code include r={2/3,1/2,1/3}. Each test video sequence includes 300 frames using GOP as the coding structure. Each GOP includes 15 frames with a structure of {I,P,P,P...}. Figure 4 shows the comparison results with different channel bandwidth and BER. Figure 5 shows the comparison results of the reconstructed quality of two sequences under different channel bandwidth and BER. It can be seen clearly from Figure 4 and Figure 5

( n ,i ) F ( n 1, i ) | 2 } D S + C ( n ) = D S ( n ) + Pe E {| F ~ ( n 1, i ) F + P E {| F ( n 1, i ) | 2 }
e

Then the expected average of DS+C(n) of I frame is:

(8)

Note that DS(n) can be obtained from the R-Q model of (1), E F ( n ,i ) F ( n1, i ) 2 represents the mean

square error of the construction picture between the nth frame and the n-1 th frame at the encoder side and 2 ~ which describes the source ( n 1,i ) F E F ( n1,i )

distortion of the n-1th frame, can be obtained in reference [4]. For P frame, if there are no bit errors, the construction value at the receiver can be described

685

that compared with the fixed channel coding rate scheme, our proposed one can achieve a higher reconstructed quality of 1~2dB at the receiver side. Figure 6 shows the reconstructed pictures of different sequences (CIF format), where BER is 10-3 and available channel bandwidth is 150kbps. Figure 6 illustrates that our proposed method can also achieve a better reconstructed video quality. Tests over other video sequences yielded similar results.

(a) BER 10-4, foreman QCIF format sequence

5. Conclusion
In this paper, we first propose a novel R-Q model of H.264. Then, the end-to-end video transmission distortion is analyzed and an adaptive JSCC over wireless channel based on the R-Q model and the error protection characteristics Turbo code is proposed, which can optimize the rate allocation of the available bandwidth between source coding and channel coding adaptively according to the actual condition of current wireless channel. The experiment results indicate that the proposed JSCC scheme can greatly improve the transmission robustness and achieve better reconstructed video quality.

(b) BER 10-5, news CIF format sequence

Figure 4. Comparison of reconstructed quality with different total bit rate

6. Acknowledgments
This work was supported by NSFC (60772069), the Beijing Novel Program (2005B08).

(a) BER 10-5, total bit rate 240kbps, akiyo sequence

7. Reference
[1] Robert E. Van Dyck, David J, Transport of Wireless Video Using Separate, Concatenated and Joint SourceChannel Coding, Proceedings of the IEEE, VOL. 87, NO.10, Oct. 1999, pp.1734-1750. [2] Fang Zhijun, Xu Shenghua, et al, Joint Source-Channel Coding for MPEG-4 Streams Transmission Over 3G Networks, Wireless Communications, Networking and Mobile Computing, 2005. Proceedings, Volume 2, 23-26 Sept. 2005 Volume: 2, pp.1261-1264. [3] Feng Wang, Guangxi Zhu, Zhenming Zhang, et al, Joint Source-Channel Rate Allocation and Unequal Error Protection for Dependent Video Transmission, Wireless Communications, Networking and Mobile Computing, 2005. Proceedings, Volume 2, 23-26 Sept. 2005, pp.1275 -1280. [4] Zhihai He, Jianfei Cai , Chang Wen Chen, Joint Source Channel Rate-Distortion Analysis for Adaptive Mode Selection and Rate Control in Wireless Video Coding, IEEE Transactions on circuits and systems for video technology,VOL.12,NO.6,June. 2002, pp. 511-523. [5] Qian Zhang, Wen Wu Zhu et al, A power-optimized joint source-channel coding for scalable video streaming over wireless channel, IEEE International Symposium on Circuits & Systems, Sydney, Australia, 2001, pp. 510-513. (b) BER 10-4, total bit rate 450kbps, mobile sequence

Figure 5. Comparison of reconstructed quality

(a) Adaptive rate allocation JSCC method

(b) Fix rate allocation JSCC method

Figure 6. The reconstructed pictures

[6] Thomaswiegand, Gary J. Sullivan, Gisle bjntegaard, et al, Overview of the H.264/AVC Video Coding Standard IEEE Transactions on circuits and systems for video technology, VOL. 13, NO.7, JULY .2003, pp. 560-576. [7] Liu Donghua, Theory and application of Turbo code, Publishing House of Electronics Industry, Beijing, Jan. 2004

686