Face Detection Algorithms and their Optimization in Embedded Systems

Haisum Bhatti
Computer Science SZABIST Karachi, Pakistan
AbstractThis paper gives a brief description of problems faced in detecting faces while used in embedded low resource devices such as hand held cameras, mobile phones and other embedded systems. Paper also discusses some optimizations for typical face detection algorithms, which may be used to obtain maximum performance with minimum resources for face detection in low resource devices. Face detection; embedded systems; hybrid approach; Viola-Jones; algorithm optimization;

A. Processing Power Apart from latest smart phones, mobiles generally have low processing power. Further, as proved by experiments ARM9 platforms have lesser overall performance than Pentium or Core processors [6]. And it is not convenient to use processing speed hungry algorithms in mobile devices. B. Low Memory Memory of mobile platforms is also low. Typical memory available for S60 devices is 3.5MB while for smart phones it is up to 200MB [6]. So algorithm should avoid using extensive memory when implemented on mobile device. C. OS Inconsistency Operating systems used in mobile devices vary with manufacturers. For example Nokia uses Symbian OS and Windows 8 mobile, other manufacturers such as HTC, Samsung and Google use Android and Apple uses iOS. Due to inconsistency of operating systems algorithms cant be specific about hardware. They have to be generalized enough to be platform and hardware independent. III. VIOLA-JONES OBJECT DETECTION ALGORITHM Viola-Jones is a robust object detection framework for detecting objects in an image. In paper titled Robust Realtime Object Detection three new concepts have been introduced to make object detection rapid [5]. First is of integral image which transforms regular image in gray scale and calculates intensity of gray color at each pixel and adds its intensity to pixels on left and top; this helps in finding combined intensity of any sub-part of image in one linear addition operation. Integral images are also used in making calculations faster as face features are analyzed. Second is self-learning algorithm which uses Ada-Boost algorithm to find special areas of face through analysis of color intensities [7]. Third contribution from paper is converting these special facial features in a cascade so that only important facial features are processed and background regions are discarded. FACIAL FEATURES BASED APPROACH USING VIOLA-JONES This approach uses techniques proposed in Viola-Jones algorithm [5]. In order to detect an object a classifier, which is optimized by training it with thousands of faces with various angles, is used to separate face like regions from background. To achieve this separation, captured image is separated in various regions. These regions are compared with unique facial features determined by algorithm through training of algorithm [5]. Regions with IV.

I.

INTRODUCTION

Face detection has been studied in computer vision since a long time now; perhaps most famous early example of such studies is dated back to year 1989 [1]. Many algorithms have also been proposed for detecting faces with minimum fake or missed results; few of these algorithms are popular and widely used in various applications [2]. However most of algorithms defined are either too resource consuming or have bad detection rate when used on mobile platforms [3]. An analysis of popular open source computer vision libraries which provide face detection APIs, such as Open CV, Aforge.Net, Egmucv and Accord.Net reveals Viola-Jones is widely used and accepted algorithm for face detection [4]. This paper first starts with Section I discussing issues involved in detecting face on mobile devices. Later this paper explains Viola-jones face detection algorithm in Section II, because it is most vital and common algorithm for face detection and all optimized algorithms discussed in this paper rely on it [5]. Section III describes Skin color based approach for face detection and optimizations to make it work on mobile devices. Section IV describes Facial feature based approach of Viola-Jones. Section V describes Optimized Hybrid approach of face detection which is useful and reliable in detecting faces on mobile devices. Section VI concludes this paper summarizing all the content discussed in earlier sections. II. ISSUES IN FACE DETECTION ON MOBILE DEVICES Face detection requires intense processing of images. However most of algorithms and works done till now are used and tested on PC platforms. Mobile based face detection is much difficult due to lack of hardware richness. For example a normal mobile phone would have a processor equivalent to 200 MHz or 400 MHz and small memory of 32mb or 64mb. To run algorithms on low memory and processing power they must be optimized. Following are some limitations on mobile platforms.

features are separated from non-feature containing regions by discarding irrelevant parts of image. Various other optimizations steps and techniques are introduced in order to optimize popular and widely used Viola-Jones algorithm to be able to use it in mobile platforms [8]. It is shown that by reducing the data and search amount and performing calculations on a fixed area, real time face detection may be achieved without using additional hardware on mobile devices. However training with this approach requires considerable amount of processing power, so it is not feasible to use this algorithm without optimization for mobile platforms with low processing power. SKIN COLOR BASED APPROACH FOR FACE DETECTION In this approach the detection process starts by performing analysis of picture and extracting skin colors with 1s representing regions with skin color and 0s representing regions with non-skin colors [3]. This binary image is then passed to a processing scheme which uses face size, aspect ratio and color to determine probable face locations. Then if multiple areas with skin color are found, areas with larger amount of skin color are preferred and face are shown in a square box. This algorithm is optimized further by applying various techniques to make more accurate estimates of skin color based on aspect ratio and color thresholds [9] [10]. VI. OPTIMIZED HYBRID FACE DETECTION APPROACH Although skin color based model gives good results and consumes lesser memory and processing power but it requires good amount of training for face like features to work precisely. Furthermore change in lightening conditions will also affect accuracy of color based approach used in skin color based algorithm. Further if camera quality is changed, for example if a digital camera is used to train algorithm and later it is tried on mobile phone camera, algorithm will have to be re-trained. To address this issue an online training approach is used [3]. In this approach, first facial feature based approach is used to separate face regions from background. Then skin color data is collected from this area for training. In order to improve performance of algorithm and have minimum non skin pixels, we reduce 10% of area from each size of sub-image so that we can skip hair and remaining background parts. Training starts as soon as skin color is collected. Color based processing is started immediately after skin color model is trained. As algorithm starts to process frames, first few are skipped as warm up frames because most of mobile phone cameras use auto white balance or auto exposure fixes. Then feature based algorithm is run to process at least next 5 frames. Once face area is captured, skin color is captured and training starts. As soon as training finishes, skin color based algorithm starts. If face area is detected, further search is done in neighboring area of face to determine possible face area. If skin color based model fails to detect V.

face for ten consecutive times, then skin color based approach is re-invoked to calibrate the picture again, assuming that the lighting conditions might have been changed, or a new person may have entered the scene. VII. CONCLUSION This paper discussed importance of face detection and its importance and involvement in field of computer vision. After introduction issues involved in mobile devices are discussed to give reader a brief overview of challenges involved in implementing face detection on embedded devices. Later Viola-Jones object detection algorithm is introduced briefly. Paper further discussed facial feature based approach for face detection followed by skin color based approach. Hybrid approach discussed by this paper introduces algorithm that is optimized and useful for implementing face detection on mobile devices. Although there are other algorithms also available for face detection on embedded systems, but Hybrid approach discussed in this paper gives out promising results [2]. A frontal face was not needed when detection was performed on Nokia 3650 Symbian phone and a 90% detection rate was recorded [3].

ACKNOWLEDEGMENT This paper acknowledges authors of all articles, papers and online resources referred to in this article. Specially Estevez Leonardo, Jianfeng Re, Nasser Kehtarnavaz , Mohammad Rahman, Dzmitry Tsishkou and Liming Chen for their useful papers. Further paper also acknowledges Mr. Rahim Hasnani for his generous guidance on paper writing. REFERENCES
[1] [2] [3] [4] [5] T. Kohonen, "Self-Organization and Associative Memory," Springer series in information sciences, vol. 8, p. 312, 1989. M. Yang, D. Kriegman, and N. Ahuja, "Detecting faces in images," IEEE Trans. on PAMI, vol. 24, no. 1, pp. 34-58, Jan. 2002. M. Rahman, J. Ren, N. Kehtarnavaz, and E. Leonardo, "Hybrid realtime face detection for mobile devices," pp. 1-5, Apr. 2009. M. Sun, "OpenCV + face detection," Computer Vision, vol. 1, p. 12, 2008. Jones P. and Viola M., "Rapid object detection using a boosted," IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, vol. 1, pp. 1-25, Dec. 2001. Tsishkou D., L. Chen, and E. Bovbel, "Mobile face detection and tracking for media," Int. J. Wireless and Mobile Computing, vol. 2, no. 2/3, pp. 1-2, Mar. 2007. R. E. Schapire and Y. Freund, "A short introduction to boosting," Journal of Japanese Society for Artificial Intelligence, vol. 14, no. 5, pp. 1-14, Sep. 1999. J. Ren, N. Kehtarnavaz, and L. Estevez, "Real-Time optimization of Viola-Jones face detection for mobile platforms, Circuits and Systems Workshop: System-on-Chip - Design, Applications, Integration, and Software, vol. 12, no. 1, pp. 1-4, Oct. 2008. M. Rahman, M. Gamadia and N. Kehtarnavaz, "Real-time facebased auto-focus for digital still and cell-phone cameras," Image

[6]

[7]

[8]

[9]

Analysis and Interpretation, vol. 4, pp. 177-180, Mar. 2008. [10] M. Rahman, J. Ren, and N. Kehtarnavaz, "Real-time implementation," Acoustics, Speech and Signal Processing, vol. 8, pp. 1353-1356, Apr. 2009.