By
Anam Arif
Munazza Javed
Malik Ibrahim
M Shabaz Swati
i
Dedication
This thesis is dedicated to our
Dear Parents
And
Respectable Teachers
ii
CERTIFICATION
Entitled: ………………………………………………………………….
Computer Science
Supervisor:
Head of Department:
External examiner:
iii
ACKNOWLEDGMENT
We are thankful to Almighty ALLAH who is Merciful, Beneficent and Gracious, who has
provided the opportunity for leading the project on this path and gave the courage and strength to
complete this hard work. There has been imbuement of many people towards the completion of
this research. We forward the sincere apologies to all those whom we may not succeed to
mention.
We also extend our thanks to Dr. Noor-ul-Amin, Head of Department of IT, for his co-
operating, warm and helping attitude towards studies.
Our deepest appreciation goes to Mam Sumera Sattar for her additional support in each
and every step of our project.
We are deeply thankful to our Dear parents, their prayers became the stepping-stones
towards the completion of our project.
Anam Arif
Malik Ibrahim
Munazza Javed
M Shabaz Swati
iv
CONTENTS
1. INTRODUCTION 1
1.1 Historical development of hand gesture recognition 1
1.1.1 History of data glove 2
1.1.2 Vision based hand gesture recognition 6
1.2 Related work in gesture recognition 6
1.3 Objective of the project 7
1.3.1 Introduction of the project 7
1.3.2 Primary objective 7
1.3.3 Secondary objective 7
1.4 Limitations of the project 7
2. LITERATURE REVIEW 8
2.1 Introduction 8
2.2 Gesture definition 9
2.3 Hand gestures 9
2.3.1 Static gestures 9
2.3.2 Dynamic gestures 10
2.4 The basics of gesture recognition 10
2.5 Applications of hand gesture recognition 11
3. IMAGE PROCESSING AND NEURAL NETWORK 15
3.1 Image 15
3.2 Image creation 15
3.3 Image sensors 16
3.4 Image processing 16
3.5 Proposed combinational features for classification 20
3.5.1 structural features based on region properties 20
3.6 Artificial neural networks 21
3.7 Neural networks architectures 22
3.7.1 Single layer neural network 22
3.7.2 Multilayer neural network 22
3.8 Further classification on the basis of learning paradigms 23
3.8.1 learning paradigms 23
3.8.1.1 Supervised learning 23
3.8.1.2 Unsupervised learning 24
3.9 Advantages of neural computing 24
3.10 Summary 24
v
4. DOMAIN METHODOLOGIES 26
4.1 Digital image processing 26
4.1.1 Applications 26
4.2 Matlab 26
4.2.1 Variables 27
4.2.2 Functions and scripts 27
4.2.3 Matlab operators 27
4.2.4 Graphics 30
4.2.5 Programming constructs 30
4.2.6 Global variables 31
4.2.7 Debugging 31
4.2.8 Other features 31
4.3 The Matlab language 32
4.4 Publishing results 34
4.5 Deploying applications 34
5. SYSTEM OVERVIEW 35
5.1 The system flow 35
6. EXISTING SYSTEMS 48
6.1 Vision based solutions 48
6.2 Sensor based solutions 48
6.3 Comparison and analysis 49
7. GRAPHICAL USER INTERFACE 51
8. CONCLUSION AND FUTURE ASPECTS 57
9. REFERENCES 58
vi
LIST OF FIGURES
1.1 Artistic impression of a sensor glove that places sensors on finger joints 2
1.2 MIT Acceleglove with its multiple sensors 3
1.3 Cyber Glove III for motion capture 3
1.4 CyberGlove II 4
1.5 5DT Motion Capture Glove and Sensor Glove Ultra.(Left) Current version,
(right) early version 4
1.6 X-IST Data Glove 5
1.7 P5 glove 5
2.1 Vitarka Mudra (Rose, 1919) 8
2.2 Hand gestures 10
2.3 A generic gesture recognition system 11
2.4 A Gestural Interface to Virtual Environments (O'Hagan, Zelinsky, &
Rougeaux, 2002) 12
2.5 The ASL sign language 13
2.6 Menu Items are displayed in a pie shape; the thumb is extended to Switch
from Draw Mode to Menu Mode [37] 13
2.7 Human –Robot Interaction [40] 14
3.1 Single layer neural network 22
3.2 Multilayer neural network 23
5.1 Hand segmentation 36
5.2 K curvature method 38
5.3 The flow of k curvature algorithm 39
5.4 Flow chart of static hand gesture recognition system 41
5.5 Flow chart of dynamic hand gesture recognition system 43, 44
5.6 Flow chart of virtual mouse 46
7.1 The guide tool window 52
vii
LIST OF TABLES
viii
Abstract
Real-time, static and dynamic hand gesture recognition affords users the ability to interact
with computers in more natural and intuitive ways. The gesture can be used to communicate
much more information by itself compared to computer mice, joysticks, etc. allowing a greater
number of possibilities for computer interaction.
The purpose of this work was to design, develop and study a practical framework for real-time
gesture recognition that can be used in a variety of human-computer interaction applications.
The system include the processes of image acquisition, hand segmentation, feature of
hand extraction, gesture recognition etc.
ix