Pattern Recognition 35 (2002) 2041 2049

www.elsevier.com/locate/patcog

2D human body tracking with Structural Kalman lter

Dae-Sik Jang, Seok-Woo Jang, Hyung-Il Choi
School of Media, Soongsil University, 1-1 Sangdo 5 Dong, Dong Jak-Ku, Seoul 156-743, South Korea Received 25 January 2000; accepted 16 October 2001

Abstract Tracking moving objects is one of the most important techniques in motion analysis and understanding, and it has many di cult problems to solve. Especially, estimating and identifying moving objects, when the background and moving objects vary dynamically, are very di cult. It is possible under such a complex environment that targets may disappear totally or partially due to occlusion by other objects. The Kalman lter has been used to estimate motion information and use the information in predicting the appearance of targets in the succeeding frames. In this paper, we propose another version of the Kalman lter, to be called Structural Kalman lter, which can successfully work its role of estimating motion information under such a deteriorating condition as occlusion. Experimental results show that the suggested approach is very e ective in estimating and tracking non-rigid moving objects reliably. ? 2002 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.
Keywords: Motion analysis; Structural Kalman lter; Moving object tracking; Occlusion

1. Introduction There has been much interest in motion understanding with the progress of computer vision and multimedia technologies. Especially, estimating and tracking moving objects have received great attention because they are essential for motion understanding [13]. However, such tasks become very di cult to solve when the background and moving objects vary dynamically and no signicant solutions for them have been reported. We can nd various approaches for tracking moving objects in the literature [4 6]. Most of them make some e orts to estimate motion information, and use the information in predicting the appearance of targets in the succeeding frames. The Kalman lter has been used for this purpose successfully. It presumes that the behavior of a moving object could be characterized by a predened model [5,6], and the model is usually represented in terms of its state

vector [7]. However, in a real world environment, we often face a situation where the predened behavior model falls apart. It is possible that a target may disappear totally or partially due to occlusion by other objects. In addition, the sudden deformation of a target itself, like articulated movements of human body, can cause the failure of the predened behavior model of the Kalman lter. We propose in this paper another version of the Kalman lter, to be called Structural Kalman lter, which can overcome the above mentioned problems and successfully work its role of estimating motion information under such a deteriorating condition as occlusion. The Structural Kalman lter utilizes the relational information among sub-regions of a moving object. The relational information is to supplement the unreliable measurements on a partially occluded sub-region, so that a priori estimate of the next state of the sub-region might be obtained based on the relational information as well as the actual measurements. 2. Organization of Structural Kalman lter The Kalman lter is one of the most popular estimation techniques in motion prediction because it provides an

Corresponding author. Tel.: +82-2-820-0679; fax: +822-822-3622. E-mail address: hic@computing.soongsil.ac.kr (H.-I. Choi).

0031-3203/02/$22.00 ? 2002 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved. PII: S 0 0 3 1 - 3 2 0 3 ( 0 1 ) 0 0 2 0 1 - 1

2042

D.-S. Jang et al. / Pattern Recognition 35 (2002) 2041 2049

Initial state

Correction step Prediction step Compute Kalman gain

Project current state

Estimate next state

Update system state

Update error covariance

Fig. 1. Feedback cycle of Kalman lter.

optimal estimation method for linear dynamic systems with white Gaussian noise. It also provides a generalized recursive algorithm that can be implemented easily with computers. In general, the Kalman lter describes a system with a system state model and a measurement model as in Eq. (1). (k) = s (k 1) (k 1) + w (k); s (1)

m (k) = H (k) (k) + v (k): s

The system state (k) at the kth time frame is linearly ass sociated with the state at k1th time frame, and there is also a linear relationship between the measurement m (k) and the system state (k). The random variables w (k) and v (k) reps resent the state and measurement noise, respectively. They are assumed to be independent of each other, and have white Gaussian distribution. In Eq. (1), (k) is called the state transition matrix that relates the state at time frame k to the state at frame k + 1, and H (k) is called the observation matrix that relates the state to the measurement. The Kalman lter estimates the system state by using a form of feedback control. Fig. 1 shows the feedback cycle. The lter estimates the system state at some time and then obtains feedback in the form of measurements. As such, the tasks of the Kalman lter fall into two phases: prediction step and correction step. The prediction step is responsible for projecting forward the current state and obtaining a priori estimates of the state and measurement for the next time frame. The correction step is responsible for the feedback. That is, it incorporates an actual measurement into the a priori estimate to obtain an improved a posteriori estimate. The improved estimate in turn fed into the prediction step and the predictioncorrection cycle is repeated.

It is frequently the case in our tracking problem that an actual measurement on a target is not reliable due, in large part, to occlusion. When an actual measurement is heavily corrupted, the measurement error accumulates through the feedback cycle and we cannot expect the reliable estimation. Our Structural Kalman lter is intended to solve the problem of partial occlusion. It does not treat a target as one entity to evaluate. It rather partitions a target into several meaningful sub-regions, and evaluates each sub-region independently together with their relationship. The overall evaluation is then used to estimate the motion information on possibly occluded sub-regions. Our Structural Kalman lter is a composite of two types of the Kalman lters: Cell Kalman lters and Relation Kalman lters. The Cell Kalman lter is allocated to each sub-region and the Relation Kalman lter is allocated to the connection between two adjacent sub-regions. Fig. 2 shows the framework of the Structural Kalman lter with four Cell Kalman lters (KF1 ; KF2 ; KF3 ; KF4 ) and four Relation Kalman lters (KF12 ; KF23 ; KF13 ; KF34 ). It is the adjacency of sub-regions that requires the assignment of the Relation Kalman lter. Fig. 2 thus implies that sub-region 3 is adjacent to all the other sub-regions. The Cell Kalman lter is to estimate motion information of each sub-region of a target, and the Relation Kalman lter is to estimate the relative relationship between two adjacent sub-regions. The nal estimate of a sub-region is obtained by combining the estimates of involved Kalman lters. For example, it is (KF1 ; KF2 ; KF12 ; KF13 ) that may a ect the nal estimate of sub-region 1 of Fig. 2. When the sub-region 1 is judged not to be occluded, KF1 is enough to estimate its motion information. However, when the sub-region 1 is judged to be occluded, we rely on the estimates of (KF2 ; KF3 ; KF12 ; KF13 ) to supplement the corrupted estimate of KF1 .

D.-S. Jang et al. / Pattern Recognition 35 (2002) 2041 2049

2043

KF1

3. Estimation of lter parameters Our Structural Kalman lter estimates the system state in a way of feedback control that involves the prediction step and correction step. This strategy is very similar to that of the typical Kalman lter, though the detailed operations in each step are quite di erent. Especially, the correction step combines the predicted information of Cell lters with that of Relation lters to provide an accurate estimate for the current state. Fig. 3 shows the estimation ow of the Structural Kalman lter. The prediction step determines a priori estimates for the next state based on the current state estimates. This step is applied to each Cell and Relation lter, treating each one independently. Under the linear model as in Eq. (1), we ignore the noise terms and get the recursive form of estimates like in Eq. (5).
i sc (k + 1) = c i (k) sc (k);

KF12

KF13 KF4

2 KF2

KF23 3 KF3

KF34

Fig. 2. A sample framework of the Structural Kalman lter.

Though our Structural Kalman lter is not limited to some specic form of a measurement vector and state vector, we dene in this paper a measurement vector and system state of the jth Cell Kalman lter as in Eq. (2). That is, the measurement vector contains the position of a sub-region, and the state vector includes the position and its rst derivative. j sc (k) = c mj (k) c mj (k) ; c mj (k) = xj yj : (2)

i mic (k + 1) = Hc (k) sc (k + 1); ij sr (k + 1) = r ij (k) sr (k);

ij mij (k + 1) = Hc (k) sc (k + 1); c

(5)

The Relation Kalman lter is to predict the relative relationship of adjacent sub-regions. We dene its measurement vector and state vector to represent the relative position between the ith sub-region and the jth sub-region as in Eq. (3). ij (k) = sr r mij (k) r mij (k) ; r mij (k) = xij yij ; (3)

where xij and yij depict the positional di erence of the jth sub-region from the ith sub-region with respect to x-axis and y-axis, respectively. We need to dene another measurement vector, called the inferred measurement vector, which is to indirectly express the measurement of a sub-region through the relative relationship with its pair associated by the connection. Eq. (4) tells about it. r mj (k) = mic (k) + mij (k); c r mi (k) = mj (k) mij (k);
j

(4)

where the superscript denotes a priori estimate for the next state and the superscript denotes a posteriori estimate. Eq. (5) shows how to compute the best estimate of the next measurement after the current measurement has been made. Once we get a posteriori estimate of a system state, we can obtain a priori estimate of a system state and then in turn a priori estimate of a measurement. It provides the answer to our problem of predicting the position of a sub-region. Now the remaining problem is how to get a posteriori estimate of the system state. This is the task of the correction step. The correction step of the Structural Kalman lter computes a posteriori estimate of the system state as a linear combination of a priori estimate and a weighted sum of di erences between actual measurements and predicted measurements. When taking the weighted sum of di erences, this step requires the participation of all the related Cell lters and Relation lters. For example, to obtain the 1 estimate sc (k); a posteriori estimate of KF1 in Fig. 2, we need to consider actual and predicted measurements of KF2; KF3 ; KF12 and KF13 . It is because our Structural Kalman lter is conceived to supplement the unreliable measurement of some sub-region with the measurements of its adjacent sub-regions. Eq. (6) shows how to compute a posteriori estimate of KF1 .
j j sc (k) = sc (k) + (1

where m (k) is computed by adding the measurement vec r tor of mij (k) to the measurement vector of mic (k). In a similar manner, we can compute mi (k) by subtracting the measurement vector of mij (k) from the measurement vec tor of mj (k). When no occlusion happens and the Kalman lters work perfectly, the inferred measurement mj (k) must be exactly equal to mj (k).

j (k)) {Kc (k)(mj (k) mj (k))} c c

(k)

1 Nj

(1
i

(k))Krij (k)(mj (k)

mj (k)) : c

(6)

2044

D.-S. Jang et al. / Pattern Recognition 35 (2002) 2041 2049

Initialization Initialization

Matched Matched measurement measurement

Prediction step

Correction step

State estimate State estimate Cell Kalman filter Cell Kalman filter

Calculate relational Calculate relational measurement vector mesurement vector Update Update Cell Kalman filter Cell Kalman filter measurements measurements

State estimate State estimate Relation Kalman Relation Kalman filter filter

Update Update Relation Kalman Relation Kalman filter measurements filter measurements

Fig. 3. Estimation ow of the Structural Kalman lter.

Eq. (6) computes a posteriori estimate of the jth Cell lter with three terms. The rst term denotes a priori estimate that is estimated at the time of k 1. It is adjusted by the second and third term. The second term contains two types j of weights, j (k) and Kc (k), together with the di erence between the actual and predicted measurements on the jth j sub-region. The Kalman gain of Kc (k) is the typical weight that emphasizes or de-emphasizes the di erence. It is computed by manipulating the error covariance [7,8]. Another weight of j (k) is a newly introduced term that re ects the degree of occlusion of the jth sub-region. As the region is judged occluded more and more, the weight becomes higher and higher so that the jth sub-region contributes less and less to the determination of the a posteriori estimate. The remaining problem is how to determine the measure of occlusion. For the time being, we leave it as it is. We will discuss it in detail later. The third term of Eq. (6) is to re ect the in uence of sub-regions adjacent to the jth sub-region. The amount of inuence is controlled by the weight of j (k). In other words, as the jth sub-region is judged occluded more and more, the computation of the a posteriori estimate relies on its adjacent sub-regions more and more. However, if some adjacent sub-region is also heavily occluded, the sub-region may not be of much help in the estimation. This consideration is handled by the use of 1 i (k) divided by Nj that denotes the number of sub-regions adjacent to the jth sub-region. The third term also contains the Kalman gain and the di erence between the inferred and predicted inferred measurements on the jth sub-region. The inference is made by Eq. (4).

That is, it involves the adjacent Cell lter and the associated Relation lter. The correction step also computes a posteriori estimate of the system state of the Relation lter. The estimation is achieved by adding to a priori estimate the weighted di erence between the actual and predicted measurements as in Eq. (7).
ij ij sr (k) = sr (k) + K ij (k)(mij (k) mij (k)): r r

(7)

In the Relation lter, the measurement depicts the relative di erence between the measurements of two associated sub-regions. In this paper, we consider only the positional di erence for the measurement. The predicted measurement is computed by Eq. (5). However, when we compute the actual measurement, we concern the possibility of occlusion and re ect the possibility in the form of Eq. (8). mij (k) = {(1 r
j

) (k)mj (k) (1 i ) (k)mic (k)} c

i

+ { j (k)mj (k) c

(k)mic (k)}

(8)

Eq. (8) is the weighted sum of two di erent types of di erences. One is the di erence between actual measurements, and the other is the di erence between the predicted measurement of associated Cell lters. Each measurement is properly multiplied by the weights representing the amount of occlusion or the amount of non-occlusion of the corresponding sub-region.

D.-S. Jang et al. / Pattern Recognition 35 (2002) 2041 2049

2045

So far we have postponed the discussion about the measure of j (k) representing the degree of occlusion. In order to judge whether a sub-region is occluded or not, we need a model about the sub-region. If a sub-region matches well with its model, there is a high possibility that the sub-region is preserved without occlusion. However, the problem of modeling a sub-region is not our main concern. One can nd various types of approaches for modeling a target in the literature [5,6]. Some examples are active models and deformable templates. We use a good model of a sub-region [6]. In fact, to dene a model of a sub-region in the current time frame, we use its corresponding sub-region of a previous time frame. We compare some predened features of the pair of corresponding sub-regions. The amount of dissimilarity between the features is computed, and it is then used to determine the measure.
j

(k) = 1 e|Features

(k1)Featuresj (k)|

(9)

4. Structural Kalman lter in action: predicting the motion information In the previous two sections, we presented the basic form of the Structural Kalman lter and its operational characteristics, especially with regard to estimating parameters. This section applies the Structural Kalman lter to the specic problem of predicting motion information of moving objects. Fig. 4 shows the sequence of images to be used for the experiment. A person is moving around the room. During the movement, its body is partially occluded from time to time. A camera was setup in the room with uorescent lights on its ceiling. We turned on some of the lights in the middle of the tracking process in order to evaluate the e ects of illumination changes. We set the frame interval to be 1=10 s, since the current implementation of the method requires such a processing time per frame for a video image (360 240 pixels with 32 bits per pixel) on IBM-PC. We set a certain area near the door in the room as an area for detecting a moving object, so that we can detect a newly entering moving object by searching this area. In Fig. 5, the white rectangle denotes this area. We obtained an initial model of a target by taking a di erence operation between two successive frames [5]. The di erence operation yields very na ve areas of moving objects. After some preprocessing like noise removing and region lling, we get a moving area as in Fig. 6(b) which represents a target. The extracted moving area is then partitioned into several sub-regions. We use a hierarchical top-down segmentation technique for this purpose [9]. At each level of the hierarchy, we apply K-means clustering algorithm to partition into sub-regions. We used the property of regional homogeneity as a criterion of clustering [6]. In this specic experiment, K was set to be 2. The process

of our hierarchical top-down segmentation is depicted in Fig. 7. We apply size-ltering to partitioned sub-regions, so that small-sized regions may be removed. The Cell Kalman lter is now assigned to each of the remaining region. The adjacency among sub-regions is examined, so that the Relational Kalman lter may be assigned between two adjacent sub-regions. Fig. 6(c) shows the resulting sub-regions and their adjacency. In this example, our Structural Kalman lter has 3 cell lters, KF1 , KF2 , and KF3 , and 2 relation lters, KF12 and KF23 as in Fig. 8. The center coordinates of sub-regions are used for measurements of the corresponding cell lters, while we use for the measurements of relation lters the disparities between measurements of associated cell lters. The system state involves the measurement and its rst-order derivative. Under this setting, the system transition matrix and measurement matrix have the simple form like in Eq. (10). 1 0 t 0 t2 0 0 1 0 t 0 t2 0 0 1 0 t 0 j = ; 0 0 0 1 0 t 0 0 0 0 1 0 0 0 0 0 0 1 Hj = 1 0 0 0 0 0 0 1 0 0 0 0 : (10)

We compare sub-regions of one frame, to each of them a cell lter is associated, against sub-regions of successive frame, so that the best matching pair could be found. The comparison is performed with regional features like color, texture and shape. Once the best matching pairs are found, we can obtain j (k), the measure representing the degree of occlusion, by using Eq. (9). Fig. 9 shows values of j (k) of 3rd sub-region in a sequence of frames of Fig. 4. We can observe that the measure gets a very high value at frame 126 due to severe occlusion. Fig. 10 quantitatively illustrates how the a posteriori esj timate of sc is computed by adjusting the a priori estij mate of sc with the second and third term of Eq. (6). In Fig. 10, the solid line depict the second term and the dotted line depict the third term of Eq. (6), respectively. We can notice that the contribution of the third term increases as that of the second term decreases. In other words, when a sub-region is judged occluded much, the role of relation lters becomes much important in determining the a posteriori estimate. Fig. 11 depicts the trajectories of predicted measurements of 3rd sub-region. The gure conrms how successfully our Structural Kalman lter works under the situation of partial occlusion. For example, the 3rd sub-region is heavily

2046

D.-S. Jang et al. / Pattern Recognition 35 (2002) 2041 2049

Fig. 4. Sequence of test images: (a) 28th frame; (b) 41st frame; (c) 57th frame; (d) 68th frame; (e) 84th frame; (f) 94th frame; (g) 126th frame; and (h) 137th frame.

D.-S. Jang et al. / Pattern Recognition 35 (2002) 2041 2049

2047

Fig. 5. Search area for detecting a moving object.

Fig. 8. Construction of Structural Kalman lter for a target.

Fig. 6. Forming an initial model of a target. (a) input image; (b) di erence image; (c) initial model. Fig. 9. Amount of occlusion.

Moving Area Moving Area

Generate a highlevel node

Clustering Clustering

Generate a lowGenerate a lowlevel node level node

N

Initial model Initial model

Stop condition Stop condition

Fig. 7. Hierarchical top-town segmentation.

Fig. 10. Variation of in uence of relation lters.

2048

D.-S. Jang et al. / Pattern Recognition 35 (2002) 2041 2049

Fig. 11. Trajectories of predicted measurements.

occluded at frame 126 as shown in Fig. 4. However, we can notice that the Structural Kalman lter does not lose the sub-region and successfully keeps tracking the subregion. 5. Conclusions and discussions In this paper, we proposed a new framework named Structural Kalman lter that can estimate information on occluded region with the help of relation information among adjacent regions. When no occlusion happens, our Structural Kalman lter operates exactly like the typical Kalman lter. It is because relation lters become inactive under such a situation. However, as an object gets occluded more and more, its dependency on relation information becomes increased and the role of relation lters becomes crucial in determining the a priori estimate of an object. The Structural Kalman lter was designed to estimate motion information of a moving object. It requires a sort of model of an object, so that the amount of occlusion could be computed through matching the model against corrupted input data. The computed information is to determine how much of the outcomes of relation lters are to be involved in the overall outcome. We did not evaluate at present the e ects of the matching metric on the performance of the Structural Kalman lter. The model of an object presented in this paper is very simple and coarse. Depending on applications, one may employ more elaborate models like snake models or deformable temples suggested in Refs. [10,11]. The Structural Kalman lter requires the partition of an object into several sub-cells. It is because the relation lters are dened among adjacent sub-cells. It is not clear at present how large the number of sub-cells should be. The compulsory partition of a homogeneous object may cause a severe problem when we perform the process of model

matching, unless the model can distinguish the partitioned sub-cells. However, the judicious design and implementation of the Structural Kalman lter may help solving this problem. For example, we may include some hypothetical objects as parts of a target object or several objects may be combined as one entity. The suggested Structural Kalman lter takes the so-called model-based approach. We form an initial model of a target at the beginning stage of tracking. The Structural Kalman lter then overcomes the problem of partial occlusion occurring in the course of tracking a moving target by activating the relation lters. The amount of the activation is computed by matching the initial model against the measurement of a current frame. Therefore, our Structural Kalman lter would fail to operate successfully if an initial model itself turns out to be occluded. To solve the problem, we might need to adopt a deformable model suggested in Refs. [10,11]. Also, we consider that our Structural Kalman lter can be expanded to keep tracking multiple targets by adding initial models as many as the number of targets. The experiments show very promising results in tracking the moving object that can be occluded partially. The relation information among adjacent sub-regions turns out to be well used in supplementing the unreliable measurement of a partially occluded sub-region. We may draw as conclusions that our Structural Kalman lter provides another framework that can overcome the problems of the Kalman lter caused by corrupted measurements. 6. Summary Tracking moving objects is one of the most important techniques in motion analysis and understanding, and it has many di cult problems to solve. Especially, estimating and identifying moving objects, when the background and moving objects vary dynamically, are very di cult. It is possible under such a complex environment that targets may disappear totally or partially due to occlusion by other objects. The Kalman lter has been used to estimate motion information and use the information in predicting the appearance of targets in succeeding frames. In this paper, we propose another version of the Kalman lter, to be called Structural Kalman lter, which can successfully work its role of estimating motion information under such a deteriorating condition as occlusion. The Structural Kalman lter utilizes the relational information among sub-regions of a moving object. The relational information is to supplement the unreliable measurements on a partially occluded sub-region, so that a priori estimate of the next state of the sub-region might be obtained based on the relational information as well as the actual measurements. Experimental results show that the suggested approach is very e ective in estimating and tracking non-rigid moving objects reliably. The relation information among adjacent

D.-S. Jang et al. / Pattern Recognition 35 (2002) 2041 2049

2049

sub-regions turns out to be well used in supplementing the unreliable measurement of a partially occluded sub-region. We may draw as conclusions that our Structural Kalman lter provides another framework that can overcome the problems of the Kalman lter caused by corrupted measurements. Acknowledgements This work was partially supported by the KOSEF through the AITrc and BK21 program (E-0075). References
[1] Ouseb Lee, Yao Wang, Motion-compensated prediction using nodal-based deformable block matching, J. Visual Commun. Image Representation 6 (1) (1995) 2634. [2] A.L. Gilbert, M.K. Giles, G.M. Flachs, R.B. Rogers, Yee Hsun U, A real-time video tracking system, IEEE Trans. Pattern Anal. Mach. Intell. 2 (1980) 4756.

[3] T. Uno, M. Ejiri, T. Tokunaga, A method of real-time recognition of moving objects and its application, Pattern Recognition 8 (1976) 201208. [4] D.P. Huttenlocher, J.J. Noh, W.J. Rucklidge, Tracking non-rigid objects in complex scenes, Fourth International Conference on Computer Vision, 1993, pp. 93101. [5] Dae-Sik Jang, Gye-Young Kim, Hyung-Il Choi, Model-based tracking of moving object, Pattern Recognition 30 (6) (1997) 9991008. [6] Dae-Sik Jang, Hyung-Il Choi, Active models for tracking moving objects, Pattern Recognition 33 (2000) 11351146. [7] S. Haykin, Adaptive Filter Theory, Prentice-Hall, Englewood Cli s, NJ, 1986. [8] G. Minkler, J. Minkler, Theory and Application of Kalman Filtering, Magellan Book Company, Baltimore, MD, 1993. [9] Yu-Ichi Ohta, Takeo Kanade, Toshiyuki Sakai, Color information for region segmentation, Comput. Vision, Graphics, Image Process. 13 (1980) 222241. [10] M. Kass, A. Witkin, D. Terzopoulos, Snakes: active contour models, Int. J. Comput. Vision (1988) 321331. [11] D.J. Williams, M. Shah, A fast algorithm for active contours, Third International Conference on Computer Vision, 1990, pp. 592595.

About the AuthorDAE-SIK JANG received the B.S., M.S., Ph.D. degrees in Computer Science from Soongsil University, Korea, in 1994, 1996, 1998, respectively. Dr. Jang is presently a principal researcher of Computer Vision Laboratory. Mr. Jangs research interests include motion understanding, fuzzy systems, neural network and pattern recognition. About the AuthorSEOK-WOO JANG received the B.S., M.S., Ph.D. degrees in Computer Science from Soongsil University, Korea, in 1995, 1997, 2000, respectively. Jang is presently a principal researcher of Computer Vision Laboratory. Mr. Jangs research interests include motion understanding, fuzzy systems, video coding and pattern recognition. About the AuthorHYUNG-IL CHOI received the BS degree in electronic engineering from Yonsei University, Korea, in 1979, the MS degree in Computer Engineering from University of Michigan, in 1982, the Ph.D. degree in Computer Engineering from University of Michigan, in 1987. Dr. Choi is presently a professor and director of the Computer Vision Laboratory. Dr. Chois areas of research interest include Computer Vision, fuzzy & neural network, pattern recognition, knowledge-based system. He is a member of the IEEE.