Republic of Iraq Ministry of Higher Education and Scientific Research University of Technology Al-Rasheed College of Engineering and Science

Electrical and Electronic Engineering Department

Design and Analysis of Virtual Human Arm Driven by EMG Signal

A thesis submitted to the AL-Rasheed College of Engineering and Science in partial fulfillment of the requirements for the Degree of Doctor of Philosophy in Electrical and Electronic Engineering ( Control Engineering )

By Yousif Ismail Mohammed AL Mashhadany (M.Sc. 1999)

Supervised by Asst. Prof. Dr. Mohammed Zeki AL Faiz
Ramadan 1430 September 2009

) (

To the memory of my father. To my mother with my gratitude. To my brothers with my thanks. To my wife and kids with my love.

Acknowledgment
First of all, praise be to ALLAH for the support in all my life.

Special mention has to be made to my supervisor Dr. M.Z. ALFaiz, I offer him my deepest appreciation for his continuous work the preparation of the thesis. He tried hard to ensure the high quality of the work.

Special thanks are due to Dr. Mohammed Abd AL-Waihed, Dr. Aimd AL-Salihy, Dr Firooq in the Medical City hospital and all who have helped me directly or indirectly in this work.

Special thanks are due to all staff of AL Rasheed College for helping me to complete my study.

Special thanks go to my mother and my brothers for their endless support and encouragement.

Last, but not least, special thanks go to my wife for supporting me all the time.

Abstract
The real human arm is a complex kinematics system. It works according to instructions calculated by a highly accurate method, and has a moving system working by deity manner. The design and simulation analysis of arm moving system model with 7-Degree of Freedom (DOF) are presented. The electromyography (EMG) signal is the activation signal for muscles in human arm and accordingly the arm will have a certain movement. The arm dynamic system in real environment simulation was implemented using the inverse kinematics problem (IKP) analytical solution. The computation time of this simulation is very fast and has the ability to include any real constrain. The finite recurrent back propagation neural network ( FRBP-NN ) is used with the simulated system for identification, classification, and human arm movements recognition with respect to a specific EMG signal. Virtual Reality Toolbox, which is interfaced with the Simulink\ MATLAB, is used as the environment for system simulation. This work consists of four main parts: The first part presents a proposed algorithm for analytical solution of IKP introduced with its simulation. The simulation results are then compared with another simulation results for the latest algorithms in this field. The second part describes the EMG signal, its biology generation, EMG sources, specification and characteristics. Moreover, it presents the EMG real measuring system. Analysis of the processed signal is done to select suitable features (amplitude and period) that can be used in the neuroidentifier. A proposed structure of neuro-classification and neurorecognition is used for human arm movements based on EMG signal. An i

EMG simulator is modified to generate an EMG data related to a clinical specification which is used to simulate an EMG signal by using EMGlab software. The third part deals with the Virtual Reality (VR) environment. The virtual reality is a simulation software for three dimension (3D) objects. It is used to present different models ( human body model and skeleton human arm ) for virtual human arm. Mixed reality human (MRH) is a new type of embodied for objects in VR, that is used to present the model of skeleton human arm. VR is used with the simulink to generate the commands for the above mentioned models, which are calculated either by IKP or by a recognized EMG signal. In the fourth part a design of a proposed package is presented, which introduces and solves all the above three parts. This package includes all simulation data transportation between different programs, and it contains many helps, which explain the individual parts for the working system and important notes joined by photos and movies. Satisfactory results are obtained, which give the solution of the forward and inverse kinematic human arm by the proposed method. Moreover, it shows the usefulness of using FRBP-NN to recognize the movements of human arm using EMG signal and the effective use of virtual environment to implement all movements of virtual human arm that are similar to natural human performance.

ii

List of Contents Subject Abstract List of Contents List of Symbols List of Abbreviations Chapter One : General Introduction 1.1 Introduction 1.2 Fundamental Concepts 1.3 Literature Survey 1.4 Aim of the Work 1.5 Work Contributions 1.6 Outline of Thesis Chapter Two : Modeling, Analysis of Human Arm 2.1 Introduction 2.2 Autonomy Human Arm 2.2.1 Skeletal Structure 2.2.2 Human Arm Muscles 2.3 Types of Human Arm Joints and Models 2.3.1 Joint Rotations 2.3.2 Joints Model 2.4 Human Arm Kinematic 2.4.1 Forward Kinematic 2.4.2 The Inverse Kinematic (IK) iii 10 10 10 12 14 14 16 19 20 23 1 1 3 8 8 8 Page i iii vi ix

2.4.2.1 Analytical Solution of I.K Human Arm 2.4.2.2 Analytical Solution of I.K Manipulator Chapter Three : EMG Signal and Virtual Environment 3.1 Introduction 3.2 Physiology of (EMG) Signal 3.3 EMG Signal Fundamentals 3.3.1 Electrical Activity Generation of EMG Signal 3.3.2 EMG Characteristics 3.4 The Virtual Reality Axiom 3.5 Virtual Reality Requirements 3.6 VRML Coordinate System 3.7 Virtual Human Body

24 32

35 35 38 38 40 43 43 45 46

Chapter Four : Proposed Algorithm and Structures for Virtual Human Arm Movements 4.1 Introduction 4.2 Proposed Algorithm of Human Arm Inverse Kinematics 4.3 Human Arm EMG Processing 4.4 Simulation Generated EMG signal 4.5 Simulation of Generated EMG and Measured EMG Signals 4.6 EMG Recognition System 4.7 Object Design in Virtual Reality: 4.7.1 Object Structure Design in VRML. 4.7.2 Human Body Project in VRML 49 49 57 60 63 65 68 68 70

iv

4.7.2.1 Skeleton of Human Body 4.7.2.2 complete Human Body Chapter Five : Simulation Results and Discussion 5.1 Introduction 5.2 Proposed Package of a Complete Virtual Human Arm Driven by EMG Signal 5.3 Measurement of Real EMG Signal 5.4 Neural Network Recognition System 5.5 Kinematics of Human Arm 5.6 Human Arm Movements Implementation in VR 5.6.1Virtual Human Body Model Demo 5.6.2 Virtual Kinematics Human Arm 5.6.3 Virtual Human Arm driven by EMG Signal Chapter Six : Conclusions and Suggestions for Future Work 6.1 Conclusions 6.2 Suggestions for Future Work References Appendix A Appendix B Appendix C

70 72

73 73 75 83 94 99 99 101 103

107 108

List of Symbols
Symbol AD-H Ae Ai As ai a(x,y,z) BD-H C ci di enew eold e() G Gf
B

Symbol Name D-H parameters for shoulder joint to elbow joint Transformation matrix for elbow to wrist joint Homogenous transformation for joint No. i Transformation matrix for shoulder to elbow joint Constant displacement of frame in axis xi Position vector for elbow at zeros swivel angle D-H parameters for elbow to wrist joint Center of circle for elbow movement Cosine of i Constant displacement of frame in axis zi New position of elbow position (cm) Old position of elbow position (cm) Position of elbow according swivel angle (cm) Homogenous matrix of End-Effector Homogenous transformation matrix for n joint Normal vector of the plane swivel Parameters of orientation matrix For end-effector Position in space with (x,y,z) coordinate (cm) Rotation matrix with dimension nn Rotation matrix for As & Ae respectively Radius of circle for elbow movement (cm) Sine of i Homogenous matrix for shoulder joint Homogenous matrix for wrist joint Transition vector for transformation between elbow and wrist joints Position vector for target in space (cm) Transition vector with dimension n1 Coordinate of target at x-axis (cm) Coordinate of target at y-axis (cm) vi

nx,ny,nz,ox,oy,oz mx,my,mz P Rnn Rs & Re r si T1 T2 te tg tn1

tg x
tg y

Symbol
tg z

Symbol Name Coordinate of target at z-axis (cm) Transition vector for transformation between shoulder and elbow joints. Homogenous matrix for elbow joint span the plane is the default elbow position New position of wrist position Old position of wrist position Eular coordinate axis magnitude vector for old position of elbow magnitude vector for new position of elbow Rotation of frame by bond angle around axis xi (rad) Rotation angle a round x-axis in shoulder joint (rad) Rotation angle a round y-axis in shoulder joint (rad) Rotation angle a round z-axis in shoulder joint (rad) Rotation angle a round y-axis in elbow joint (rad) Rotation angle a round x-axis in wrist joint (rad) Rotation angle a round y-axis in wrist joint (rad) Rotation angle a round z-axis in wrist joint (rad) Rotation of frame by dihedral angle around axis zi (rad) Eular angles (e around z-axis , e around y-axis and e around x-axis) (rad) Swivel angle for elbow (rad) Cosine of di in transformation matrix Sine of di in transformation matrix

ts Ty

x, y, z x goal , y goal , z goal

wnew wold XYZ

i 1 2 3 4 5 6 7 di e e e cdi sdi

vii

List of Abbreviation
Abbreviation AAR AL AR AT CNS CT D-H DOF DQEMG E-C EDL EDS EMD EMG FR GL GTO GUI iEMG IK IKP LG MAV MPF MR MRH MU MUAP MUAPT MVC NMJ OR Details Area to Amplitude Ratio Anthropomorphic Limb Augmented Reality Anaerobic Threshold Central Nervous System Contraction Time Denavit Hartenberg Degree of Freedom Decomposition based Quantitative EMG Excitation Contraction Extensor Digitorm Longus Extensor Digitorum Superficialis Electromechanical Delay Electromygraphic Firing Rate Glntens Maximums Golgi Tendon Organ Graphic User Interface integrated electromygraphic Inverse Kinematics Inverse Kinematics Problem Lateral Gastrocnemius Mean Absolute Value Mean Power Frequency Magnetic Resonance Mixed Reality Human Motor Uint Motor Uint Action Potential Motor Uint Action Potential Train Maximal Voluntary Contractions Neuromuscular Junction Objective Reality viii

Abbreviation PSD RE RMS SNR sEMG TA VE VH VR VRML 2D 3D

Details Power Spectral Density Real Environment Root Mean Square Signal to Noise Ratio surface electromygraphic Tibialis Anterior Virtual Environment Virtual Human Virtual Reality. Virtual Reality Modeling Language Two Dimensions Three Dimensions

ix

Chapter One General Introduction

Chapter One General Introduction

1.1 Introduction There are numerous ways to model the human arm, especially with respect to the shoulder area. This model allows accurate modeling of the interdependence between joint movements. The virtual arm (VA) is a mathematical and three-dimensional (3D) graphical representation of a human arm. It is displayed on a computer screen and it moves in response to an electromyographic activity recorded from the arm muscles.

Electromyographics (EMGs) are used to recognize the movements developed by individual muscles crossing the bones of the arm. This chapter introduces the fundamental concepts of some main terms, gives the literature survey of the related work, presents the contribution of this work in points of research, aim of the work and finally gives the outline of the thesis.

1.2 Fundamental Concepts Kinematics is concerned with the motion of articulated structures. The surrounding environment and other physical forces are not considered. An articulated structure is composed of links, i.e. rigid segments, and joints to connect them. If a structure has no branches, it is referred to as kinematic chain. The first element of such a chain is denoted as root, the last one is the end-effector [1]. Motion is a change in the position of an object with respect to a reference, and Mechanics is the science that studies the motion of objects. For practical purposes, its treatment is split into two fields:

Forward Kinematics, is concerned with the computation of the orientation and the position of the end-effector in the world frame for a given a kinematics chain and a set of joint angles for the joints of the chain. This is done by concatenating the frame transitions of the chain in the correct order. Inverse Kinematics (IK), is a widely used technique for controlling the arms of human-like characters in interactive applications, and is employed in several domains: ergonomics, virtual reality, computer games, animation, etc. For instance, nearly all professional animation packages have built-in IK solvers for manipulating skeleton hierarchies. The term IKP refers to the specific problem of finding analytic solutions for every joint angle and the degree of redundancy of the kinematics chain [3]. The EMG is sometimes referred to as a myoelectric activity. Muscle tissue conducts electrical potentials similar to the way nerves do and the name given to these electrical signals is the muscle action potential. Surface EMG (sEMG) is a method of recording the information presented in these muscle action potentials. EMG Electrodes The EMG signal is measured either non-invasively with surface electrodes, or invasively, with wire or needle electrodes [4]. The word virtual has been overused to describe just about anything that deals with a computervirtual banking or virtual teleconference. The term virtual reality (VR) is the effect created by generating an environment that does not exist in the real world. Usually, a stereoscopic display is a computer-generated three- dimensional environment giving the immersion effect. The environment is interactive, allowing the participant to look and navigate about the environment, enhancing the immersion effect [5].

Although this definition has elements that enhance our understanding of VR, it also limits our understanding of what virtual reality is by specifying hardware that is used to create the virtual effect. Other

attempts at defining VR is the combination of real-time 3-D computer graphics with shading and texture mapping, high resolution stereoscopic large screen or head mounted displays, along with novel user interfaces [5].

1.3 Literature Survey The system design in this work requires research in three main approaches, so the review is presented according to these approaches. This review will focus on the researches in the last nine years, due to the extensive research activity in this period. In EMG signal field, Gribble et.al., 2003 [6] introduced a study to test for a possible relationship between cocontraction and movement accuracy in multi-joint limb movements. The EMG activity of seven single- and doublejoint shoulder and elbow muscles was recorded using surface electrodes while subjects performed a pointing task in a horizontal plane to targets that varied randomly in size. Nan Bu et.al., 2003 [7] proposed a new EMG discrimination method based on a recurrent log-linearized Gaussian mixture network (R-LLGMN) for prostheticc control. Because of the recurrent connections between the third and the fourth layers in the R-LLGMN, the temporal information in the EMG signal can be used for the pattern discrimination. Bida, 2005 [8] focused on demonstrating that advances in EMG amplitude processors result in EMG-torque model performance

improvements. Advances in EMG amplitude estimation were applied to the EMG-torque problem for constant-posture, non-fatiguing, and force-varying 3

contractions about the elbow. The dynamic relationship between EMG amplitude and joint torque was formulated as a standard linear least squares problem. Hamilton and Stashuk, 2005 [9] developed a muscle model and EMG simulation algorithms which were to produce EMG signals consistent with those acquired from real muscle. The data from this simulation tool can be used to develop and verify EMG signal analysis tools. Specically, the simulated signals can be used within EMG signal decomposition and quantitative analysis algorithms to help develop and evaluate signal statistics which correlate with specic aspects of the structure and activation of a muscle. The most efcient EMG statistics can then be used for muscle characterization and clinical interpretation. Reaz et.al., 2006 [10] gave a brief information about EMG and revealed the various methodologies to analyze the signal. Techniques for EMG signal detection, decomposition, process and classification were discussed along with their advantages and disadvantages. Discovery of a problem or disadvantage in one method leads to other improved methods. This study clearly points up the various types of EMG signal analysis techniques so that right methods can be applied during any clinical diagnosis, biomedical research, hardware implementations and end user applications. Hou et.al., 2007 [11] developed a spinal force prediction model using an RFNN. The EMG feedback represents the muscular activation dynamics better. At the same time, it utilizes the advantages of recurrent properties. The model predicts forces directly from kinematics data, avoiding EMG measurements and the use of biomechanics model. It can help us understand

the relationships between kinematic variables and EMG signals and spinal forces. An adaptive learning algorithm is derived for the RFNN. Cipriani et.al., 2008 [12] found a trade off between good grasping capabilities of the device (strongly related to the complexity of the control interface) and low subject effort into completing grasping tasks, without addressing advanced algorithms for EMG signal processing. It determines whether a vibrotactile feedback system is subjectively or objectively useful, and how this system changes user performance. Shrirao et.al., 2009 [13] presented that Root Mean Square (RMS) of the surface EMG signals obtained from Dxtensor Digitorum Superficialis (EDS) muscle during flexion-extension rotation of the index finger at different speeds showed hysteresis. Six different neural network committees were developed to predict the joint angle from the RMS o f the SEMG signal . During testing, the neural network committees were able to predict the joint In the human arm field, Baerlocher, 2001 [14] determined a posture satisfying a set of prescribed tasks, where each task concern the interactive manipulation of complex articulated figures by means of geometric constraints usually expressed in the Cartesian space. Wu et.al., 2004 [15] proposed a new analytic IK solver suitable for multiple constrained 12-DOF human limbs. By decomposing human skeleton into five parts one head chain , two arm chains and two leg chains, a multi-constrained human skeleton can be solved analytically. Mihelj, 2006 [16] The computation of the inverse kinematic model of the human arm for robot based rehabilitation that uses measurements of the hand position and orientation and radial acceleration of the upper arm, analytical analysis and empirical validation of the method are presented. The

algorithm enables estimation of human arm angles, which can be used in trajectory planning for rehabilitation robots. Kallmann, 2007 [17] shows a new customizable whole-body IK system which is fast, robust, and simple to implement. A new analytical IK formulation based on the swing and-twist parameterization handles collision avoidance and joint coupling in a unied way (source code provided). The approach organizes key body postures as a function of the goal direction to reach, simplifying interpolation and allowing for an intuitive way of designing body behaviors. Drzevitzky, 2008 [3] introduced IK problems for anthropomorphic limbs and showed how to solve those analytically in order to obtain symbolic solutions. The symbolic solutions can be modified and recomputed to match ,for example, other input values that serve as constraints when solving the according Inverse Kinematics problem. Finally in VR field, Powell, 2007 [18] showed that the evolutionary pathway of virtual reality technology development will not be able to overcome all of the barriers and limitations inherent in the current generation of interfaces. He used a reverse tree methodology to explore alternate pathways to achieve strong VR. Antonio .et.al., 2008 [2] presented an initial developed on virtual agents. It describes how the agent was built searching an humanoid representation through a geometric model. The building of a solid platform gives total mobility and realism to both, the agent and the virtual environment where our humanoid works. This work allow us to keep on working on virtual agents area, to analyze and propose techniques to give the agent total autonomy to its motion.

Tsepkovskiy et.al., 2008 [19] introduced the study for a design of the 3D and VRML virtual hand models for different types of mechanical gripper. A 3D animation using Matlab/simulink is made. Whitworth, 2009 [20] presented the idea that the universe is a virtual reality created by information processing, and relates this strange idea to the findings of modern physics about the physical world. The virtual reality concept is familiar to us from online worlds, but our world as a virtual reality is usually a subject for science fiction rather than science. Yet the world could be an information simulation running on a multi-dimensional space-time screen. Kotranza .et.al., 2009 [21] presented mixed reality humans (MRHs), a new type of embodied agent enabling a touch-driven communication. Affording touch between human and agent allows MRHs to simulate interpersonal scenarios in which touch plays a crucial role. Two studies provide an initial evaluation of user behavior with an MRH patient and the usability and acceptability of an MRH patient for practice and evaluation of medical students clinical skills. From the above survey it can be seen that there is a need to direct research to further our knowledge in the modeling of the human body. The design of virtual human arm driven by EMG signal is a necessity. This requires analytical solution for the IKP of the human arm model and the implementation of human arm in VR environment. Also the identification, classification of EMG signal and the human arm movement recognition. The ultimate grouping of these subjects should lead to the real life simulation of the human arm.

1.4 Aim of the Work The main objectives of this work can be summarized as follows: 1. Analytical solution of IKP for solution with VR. 2. Design and analysis of EMG NN identification and classification system that can be used to recognize the human arm movements based EMG signal. 3. Designing different VR model for human arm to implement the recognition of its movement by virtual environments. human arm to implement such a

1.5 Work Contribution The contribution of this work falls in four points: The first, an algorithm for analytical solution for IKP of human arm with all real constrains in human arm is proposed. The second, a structure for neuro-classification and human arm movement recognition system based EMG signal is proposed. The third, applying MVR concept for human arm to present the skeleton human arm , human body and movement implementation. The fourth, is building a suitable package to simulate all the above systems with movies helps.

1.6 Outline of Thesis The thesis is organized as follows: Chapter One: Presents an introduction to the design of virtual human arm based on EMG and reviews the main research activities in the three specific fields. The aim of the current research is also set and summery for contribution of work. 8

Chapter two: presents the modeling of human arm, forward kinematics and analytical solution of IK for human arm manipulator and the analysis of the results. Chapter three: presents the background of EMG signal, biologist of signal, fundamental of signal, characteristic of signal, its analysis, the measurement system of EMG signal. This chapter presents VR axiom, its requirements and VRML builder for VR and explain virtually of human body . Chapter four: presents the details of proposed algorithm for IKP by analytical method and presents the human arm EMG signal: processing of this signal, generation by EMG simulator and its simulation, explain the design of classification EMG signal and recognition of movements of human arm by using NN. It presents the details of virtual design for skeleton human arm, virtual skeleton human body and human body form. Chapter five: presents simulation results for all parts of system design and its discussion. The GUI designed for demonstration and simulation purposes is also explained. Chapter six: presents the conclusion of this work and suggestions for future work. References: presents the references with my two papers published. Appendix A: presents the analytical solution for swivel angle and elbow joint angle by classical approach. Appendix B: presents program details of the simulation of the virtual

human body and its results. Appendix C: presents some program in MATLAB which are used in this work.

Chapter Two Modeling, Analysis of Human Arm

Chapter Two Modeling, Analysis of Human Arm

2.1 Introduction The human arm can be modeled as a 7-DOF mechanism, consisting of three joints with two links and the hand. The IKP of the human arm can be stated as follows: given the position and the orientation of the hand, find the seven joint angles. This chapter presents the autonomy human arm which include the muscles and the skeleton structure of the arm. Joints arm types, their model and joint limitation with the prescribed D-H parameters. The forward and inverse kinematics are presented together with the analytical solution of IKP for a real human arm. Two case studies are considered with full analytical solution, one for real human arm and the other for manipulator as a human arm with 5-DOF. 2.2 Autonomy Human Arm In order to develop a consistent topological data structure, a preliminary modeling of the musculoskeletal structure is necessary. Since biomechanical modeling is intended, it is natural to begin with the observation of the system components and their motion before modeling. In the following section, basic anatomical descriptions of the human arm are presented to outline various structures involved in its mechanics [22]. 2.2.1 Skeletal Structure The human arm is composed of three chained mechanisms, the shoulder girdle, the elbow and the wrist, whose association allows a wide range of combined motion, and confers to the human arm the highest 10

mobility in the human body. Due to the complexity of the hand mechanics, it was taken as another rigid segment in the extension of the forearm. Considering bones in pairs, seven joints may be distinguished: The sterno-clavicular joint, which articulates the clavicle by its proximal end onto the sternum. The acromio-clavicular joint, which articulates the scapula by its acromion onto the distal end of the clavicle. The scapulothoracic joint, which allows the scapula to glide on the thorax. The gleno-humeral joint, which allows the humeral head to rotate in the glenoid fossa of the scapula. The ulno-humeral and the humero-radial joints, which articulate both ulna and radius on the distal end of the humerus, and finally the ulno-radial joint where both distal ends of ulna and radius join together with the hand [23]. Assuming translations negligible compared to rotations, all, except the scapulothoracic joint, are usually assumed as ball and socket joint, having more or less 3-DOF. The scapulothoracic joint is a special case since it does not properly involve articular structures between scapula and thorax. However, due to its surrounding muscles, the scapula is usually considered as constrained to glide on the thorax. This reduces the number of DOF of the scapulothoracic joint to four. Considering all joints independently, the number of DOF of the upper limb would amount to (22). However, as they are organized in closed chains, the number of DOF of the upper arm reduces to 7-DOF[24]. Fig.2.1 illustrates the bones of human arm starting with the clavicle or collar bone, which is classified as along bone that makes up part of the shoulder girdle, it receives its name from the Latin claviculea (little key) because the bone rotates along its axis like a key when the shoulder is abducted. The second bone is the scapula or shoulder blade, which connects the humerus with clavicle, while the third bone is the humerus a longer bone 11

in arm runs from shoulder to elbow, in skeleton it fits between the scapula and the ulna. The forth part is the radius and ulna bones which connects the elbow to wrist by radius bone from the outside of limb to thumb side of the wrist and the ulna from the inner side of arm to form hinge joint with the trochlea of the humerus [25]. Clavicle Scapula Humerus

Radius Ulna

Hand

Fig. 2.1. Skeleton of Human Arm [25]. . 2.2.2 Human Arm Muscles The human arm has 22 main muscles, it is divided to three groups , the muscle of shoulder girdle, the muscle of upper arm and the muscle of forearm. These muscles are shown in Figs (2.2 and 2.3), [26,27].

12

Deltoid Teres major Triceps brachii Long head Triceps brachii Lateral head Triceps brachii Medial head

Trapezius

Deltoid Pectoralis major Coracobrachialis Biceps brachii Long head Biceps brachii short head brachialis

Latissimus dorsi

latissimus

Fig.2.2 Muscles of Shoulder and Upper Arm [27]

. Extensor carpi Radialis longus Extensor carpi Radialis brevis Flexor crapi ulnaris

brachii

Biceps brachii brachioradialis Pronator teres Palmaris longus Flexor crapi ulnaris Flexor digitorum superficialis

Triceps brachii brachioradialis Extensor carpi Anconeus Extensor digitorum Extensor crapi Flexor crapi Extensor carpi ulnaris Extensor minimi

Fig.2.3 Muscles of Forearm [27]

13

2.3 Types of Human Arm Joints and Models The state vector of generalized coordinates =( 1, ...., n )T, expresses the configuration (or posture) of the articulated structure, which is the configuration of all its joints. Hence the set of all possible configurations are termed the joint space. The total number of degrees of freedom is n, while the number of joints is k, with k n. For multiple-DOF joint models, there is no simple one-to-one correspondence between a degree of freedom and a joint: hence the generalized coordinates of the jth joint are noted j, whose number of degrees of freedom is noted nj. Hence:
k

n
j =1

=n

(2.1)

As example, for a typical human arm, the number of degrees of freedom (without the fingers) is n50, while the number of joints is k20. Now the problem is reduced to find a parameterization for each joint, and to impose limits on its motion. For this purpose, a minimal set of joint models must be defined[14 ]. 2.3.1 Joint Rotations To model a joint, it is first necessary to define the respective joint frame of each bone in a pair to enable the parametric description of its state. Such description is then fully achieved with a transfer matrix relating its relative frame to its reference frame. Various compositions exist to describe the three-dimensional orientation of a frame with respect to another. The model chooses successive local rotations with respect to the reference frame of the joint: the rotation matrix changes the global frame (X0,Y0,Z0) into the local frame (X3,Y3,Z3) of the joint and it is given by[28]

14

M ( e , e , e ) = M x ( e ) M y ( e ) M z ( e )

(2.2)

The coordinates in the global frame of any vector V and its image vector V' by the above rotation may then be obtained by the relationship:
V ' = M ( e , e , e ) V

(2.3)

Due to the planar symmetry between the left and right sides, it has been necessary to define a different rotation order for each one, in order to respect the convention of "direct coordinate system" and the anatomical rotation order usually considered by practitioners. The intermediate successive rotations around the local axis are shown in Fig. 2.4. for the left arm and in Fig. 2.5. for the right arm[29].

Z3

Z2
e

Zo
e Z =Z 1 o

cos e M z ( e ) = sin e 0 cos e M y (e ) = 0 sin e

sin e cos e 0

0 0 1

Y3=Y2 X2=X1

e e

Y2 Y1 Yo

Xo

X1

X3

0 sin e 1 0 0 cos e 0 0 1 0 cos sin M x ( e ) = e e 0 sin e cos e

Fig. 2.4 Rotations for the Left Arm [29]

15

Z3

Z2
e e

Zo
Z1=Zo

Y3
Y2=Y1 e e

Y1 Yo

Xo

X1

X3=X2

cos e sin e 0 M z ( e ) = sin e cos e 0 0 0 1 0 0 1 0 cos sin M x (e ) = e e 0 sin e cos e cos e 0 sin e M y ( e ) = 0 1 0 sin e 0 cos e

X2

Fig. 2.5 Rotations for the Right Arm [29] 2.3.2 Joints Model The first purpose of a joint model is to compute a local transformation matrix, noted , as a function of the set of generalized joint coordinates j These parameters represent either translational or rotational degrees of freedom. However, translations are not considered here, since they do not appear in the human body (except to a very limited extent). The second purpose of a joint model is to define limits on the joint coordinates j, in order to avoid self-collisions between adjacent segments and also the specification of unfeasible postures. The main joint models are explained next [29]. A. The revolute joint model (one DOF): The revolute joint is the simplest joint model that allows rotational motion: rotation occurs about a single, fixed axis. Arbitrarily choose for the axis of rotation to be the z axis of the local joint frame. Its natural parameterization is the angle of rotation , with respect to a reference configuration: hence, the local transformation matrix of the joint is simply Rz().

16

A revolute joint is typically used as a hinge joint (for flexion purposes). If the axis of rotation is aligned with the distal (moving) segment, a rotation results in a twist of the distal segment about itself. Hence the revolute joint model may be used for two conceptually different motions: flexion and twist [30]. child segment parent segment

Fig.2.6 A revolute joint connecting two segments, performing flexion motion [14] Because of its simplicity, and due to mechanical design considerations, the revolute joint is by far the most used joint in robotic manipulators. In human modeling, it is a convenient model for the flexion of the interphalangeal joints of the hands, for example. It is tempting to combine two or three revolute joints at the same point with different axes of rotation to model more complex joints such as the shoulder [17]. B. The elbow joint model (2-DOF): A situation, where two revolute joints can be easily combined to model a more complex joint is for joints with a flexion/extension motion, combined with a twist of the outgoing segment. Typical examples in the human body are the elbow and knee joint. The two axes or rotation are independent, and joint limits can also be specified independently one each degree of freedom [31]. C. The ball-and-socket joint model (three DOF): A ball-and-socket joint possesses three rotational degrees of freedom. Hence, it is the most mobile of the purely rotational joints. It allows an axial 17

motion (or twist) of the segment (1-DOF), as well as a spherical motion (or swing) that determines its direction (2-DOFs). A mechanical illustration of this joint is given in Fig.2.7. By convention, in the following discussion the moving segment is aligned with the z axis of the local joint frame. Ball-andsocket joints are used to model articulations such as the human shoulder and the wrist [14].

Fig 2.7 A Ball-and-Socket Joint. The Outgoing Segment is Aligned with the z-axis [14] The accurate kinematic modeling of such articulations is a difficult task. First, a clear mathematical description of the allowed relative motion must be given by a proper parameterization: because of the complex nonEuclidean nature of rotations, this must be done carefully, or one may incur in the problem of singularities. Second, the range of motion should be constrained by some joint limits, to restrict the parameter space to some more realistic subset. The situation is complex, because the boundaries on the three independent parameters are generally coupled. These two aspects are discussed in more detail in [14] .

18

2.4 Human Arm Kinematic The development of a high-DOF, kinematic human arm model can be used to predict realistic human arm postures. A model was built upon the 6DOF spine and 5-DOF shoulder [32]. This model falls short of realistic arm postures, however, as a result of spine rigidity and unrealistic skin deformations with shoulder movements. The goal is to develop an improved model that leads to more natural movement and allows for more realistic skin deformations in the shoulder. The model of human arm with full real movements depends on the selection of origin point in human body to built the set of kinematic matrices. If one consider the spine include with human arm, the model will appear with 15-DOF or 21-DOF (see Fig 2.8 [33]), but when he select the origin at the end of configuration of arm ( the shoulder joint) the form will change as shown in Fig 2.9.
L4 L6 L8 L7 z13 z12 z14 z15 z9 z8 L5 z11 L3 L2 L1 z1 zo z2 z5 z4 z3 z7 z10 z6

L9

Figure 2.8 A 15-DOF, Kinematics Human Model [33]

19

In this work, the anthropomorphic arm will be dealt with a 7-DOF assuming the origin is at the shoulder joint. The kinematic chain to be introduced resembles a human arm. Since most anthropomorphic limbs have the same setup, the chain is a representation for all human limbs. The distinction between limbs can be defined through joint limits. Usually one or more joint limits are shifted when considering two identical but mirrorinverted limbs, e.g. a left and a right arm.
x1 1

y1

3 x3 5 z1
e

La=a3 y2 4

y3

Lb=a4

z3

Fig 2.9 Kinematic Chain with Three Joints 2.4.1 Forward Kinematic The homogeneous transformation matrices for the frame transitions are set up with D-H parameters. The description of the rotation and the translation that occur in those frame transitions in between the frames of the joints. The 3 3 rotation matrices (embedded in homogeneous 20

transformation matrices) are denoted by Rs,Re and the translation vectors of length 3 are denoted by ( ta,tb ), respectively. The transformation matrix from the shoulder joint frame to the elbow joint frame is denoted by As, the transformation matrix from the elbow joint frame to the wrist joint frame by Ae, [22].
R s As = 0 0 ts , 0 1 R e Ae = 0 0 te 0 1

(2.4)

As mentioned before, the shoulder joint and the wrist joint are both assigned three DOFs, while the elbow joint only one DOF. The angles (1,,3) specify the rotation around the x-axis, y-axis, and z-axis of the shoulder, respectively. Without loss of generality it will be assumed that XY-Z Euler angles are being used, i.e. the rotation of angle 1 around the xaxis is the first one, followed by the rotation of angle 2 around the y-axis, concatenated with the rotation of angle 3 around the z-axis therefore; the homogeneous matrix for shoulder joint can be written as [3]:
0 R ( , , ) 0 1 1 2 3 T1 = 0 0 0 1 0 Rot ( ) Rot ( ) Rot ( ) z 3 y 2 x 1 = 0 0 0

0 0 0 1

(2.5)

21

Analogously, the rotations of the wrist are specified by the angles 5,,7. The 3 3 rotation matrix R2 of the wrist joint is a function of 5 to 7 and the matrix T2 is the equivalent to the matrix T1, i.e.,
R ( , , ) 2 5 6 7 T2 = 0 0 0 0 0 0 1 0 0 0 1

Rot ( ) Rot ( ) Rot ( ) z 7 y 6 x 5 = 0 0 0

(2.6)

The rotation of 4 around the y-axis of the elbow is realized with the rotation matrix Ry. The according homogeneous transformation matrix is Ty. We can directly express this matrix in the following way [3]:
R ( ) 0 4 y Ty = 0 0 1 0 c 4 0 s 4 0 0 1 0 0 = s 4 0 c 4 0 0 0 1 0

(2.7)

Given that the first joint of our kinematic chain is located at the origin of the world frame, the position and orientation of the end-effector is given by the concatenation of all the homogeneous transformation matrices. Those are the matrices representing the joint rotations and those representing the frame transition in between the joint frames. Since all rotations and

22

translations are presented in homogenous coordinates, the matrices can simply be multiplied [3].
G f = T1 As T y AeT2 .

(2.8)

The orientation and position of the end-effector in the world frame can be represented with the rotational part Rg and the translational part tg, the matrix G takes the form:
R g G= 0 0 tg 0 1

(2.9)

2.4.2 The Inverse Kinematics The inverse kinematics technique is useful both for the manipulation and animation of articulated figures. The focusing on the inverse kinematic problem, and on the issues raised by its solution. Basically, the problem is to determine a joints configuration for which a desired task, usually expressed in Cartesian space, is achieved. For example, the shoulder, elbow and wrist configurations must be determined in order that the hand precisely reaches a position in space. The equations that arise from this problem are generally non-linear, and are thus difficult to solve in general [3]. There is a large number of methods to solve the inverse kinematics problem. Each has its own advantages and drawbacks. They can be compared on the following important criteria: efficiency (speed), robustness, generality, naturalness of the result (except for robot manipulators), and complexity of the method. The choice of a solution method clearly depends on the intended application. For real-time applications, analytic methods are

23

always preferable, therefore; the analytical solution will be implemented in this work in two forms as given below [31]. 2.4.2.1 Analytical Solution of Inverse Kinematic Human Arm The solution for the seven joint angles of the kinematic chain introduced and the chain has one degree of redundancy. This means that even if all seven joint angle values and of course the end-effector position and orientation are known, it is still not possible to describe the position of every joint of the chain. Now will determinate how to analytically describe all DOFs and the degree of redundancy with the help of additional constraints for all human arm joints [3]. Solving for Elbow Joint 4: The elbow joint 4 is the only of the seven joints angle that regulates the distance between the shoulder frame and the wrist frame. It does not depend on the other joints and can be computed using the end-effector position tg, given the matrices As and Ae, i.e. the D-H parameters describing the arm. The translation of the end-effector in the world frame is contained in matrix G. For the sake of readability, Ri(i,i+1,i+2) is referred to as Rj. Following equation (2.8), to get[3]:
G = T1 AsTy AeT2 R 1 = 0 0 0 0 R 0 t s Ry s 0 1 0 0 0 1 0 0 0 0 R 0 te R2 e 0 1 0 0 0 1 0 0 0 0 0 0 1

(2.10)

RR R R R R t = R1 Rs Ry t e + R1t s tg R1 Rs Ry t e + R1t s g 1 a y b 2 i.e. g = = Rg = R1 Ra Ry Rb R2 0 0 1 0 0 0 1 0

24

As before, it can be assumed the shoulder to be positioned at the origin of the global coordinate system. The distance between shoulder and wrist is therefore the length of the translation vector between world frame and end-effector frame, which is the same as the position of the wrist in the world frame tg. As can be seen in equation(2.11), T1 does not contribute to the distance. This is to be expected, since it contains merely the matrix R1, which is the rotation of the wrist. The right hand side as well as the left hand side of equation(2.11) represents the translation vector, which means both sides must have the same length. Taking the dot product of the left hand side and the right hand side of the equation and the definition of D-H parameters the matrix As & Ae [3]:
c 2 s c As = 2 1 s 2 s 1 0 c 3 s c Ae = 3 2 s 3 s 2 0 s 2 c 2 c 1 c 2 s 1 0 s 3 c 3 c 2 c 3 s 2 0

0 s 1 c 1 0 0 s 2 c 2 0

s 1 d 2 c 1 d 2 1 a2 s 2 d 3 c 2 d 3 1 a1

(2.11)

From all above equations the following formula can be determined:

c 4 (2a1c1 a 2 + 2d1c 1 d 2 ) + s 4 (2a1c1c 1 d 2 + 2d1 a1 )
2 2 2 2 = (tg x + tg y + tg z2 ) (a12 + d12 ) (a 2 + d 2 ) (2 s1 a1 s 1 d 2 ).

(2.12)

From the full solution of the above equations, the final form of 4 can be found (see Appendix A):

25

Solving for Swivel Angle ( ) Elbow Position e( ) As explained at the beginning of this section, the matrix G consists of a rotation matrix Rg and a vector tg. This vector is made up of three coordinates in the world frame, the rotation matrix is the concatenation of three rotations. The three coordinates of vector tg together with the three angles of rotation implicitly given by Rg are six constraints. These six constraints are imposed on the end-effector. This means that equation(2.8) has to be solved according to these constraints. These six constraints do not allow to determine the entire kinematic chain unambiguously, the chain has a degree of redundancy. For a fixed end-effector, the elbow can swivel about an angle from position (e) to position e( ) on a circle around the axis that connects shoulder and wrist, see Fig 2.10[3]
w

s z

e e()

Fig 2.10 Elbow arc with center C and radius R. Elbow position e is rotated about the normal of the plane to new elbow position (e() ) [17] The elbow position can be expressed as a function of the swivel angle
, for which the center C, the radius r and the plane of the circle, i.e. the two

26

vectors defining the plane, are used. The normal vector of the plane n is the vector from shoulder to wrist, which is the translation of the end-effector in the world frame tg after being normalized in the Euclidean norm. The first vector u to span the plane is the default elbow position a (the vector from s to an arbitrary elbow position e) projected onto the plane. For this position, the swivel angle is set to (0). The second vector of the plane v is perpendicular to the first one and the normal of the plane. It can be obtained by using the cross-product of the other two vectors. The elbow position e( ) can be calculated as the sum of the center C and a linear combination of u and v [3].
u=

a ( a . n) n a ( a . n) n

n=

sw sw

v = n u

(2.13)

e( ) = C + r (cos( ) u + sin( ) v)

The standard elbow position which equals = 0, e()=e, is known, then for any given swivel angle the current elbow position can be computed using equation(2.13). The detail solution of the swivel angle and the elbow position e( ) and the angles of joints (1 --- 7) can be seen in Appendix A. Case study one: The above algorithm was simulated by Drzevitzky [3] using maple software, which is resimulated in this work by using Matlab software Ver.2008a with the following numerical example. From the summary of algorithms it can be seen that the key for analytical solution is the 27

calculation of (4), (note: literature survey reveals that this reference is the first full real solution of inverse kinematic), but from the simulated results in [3], (see Fig.2.11) and the simulated results by using Matlab Fig(2.12 A), the same results are obtained for (4) is unacceptable as was deduced in reference[3]. Therefore, S. Drzevitzky suggested a trial and error procedure to get the correct value. Fig(2.12 B) shows the results of the simulation for (4). The human limb D-H parameters which was used in the simulation is shown in table(2.1).

4 (rad)

Position of enfeffector in x-axis

Fig.(2.11) The result of [3] for suggested formula.

(rad) (rad)

Fig. 2.12. Simulation of 4 . a: solution of equation(2.12). b: solution of suggested formula of ref [3].

28

Table(2.1). D-H parameters for human limb [3]. D-H parameter value d1 0 d1 0 1 a1 d2 d2 0 1 0 0 2 a2 0 1

The simulation of all steps in algorithm to explain the effect of value of 4 on the final result of solution of inverse kinematic. The simulation has three steps: the first simulate with the first value of 4 in equation(2.12). The second step using the second value of 4 in equation(2.12). The last step, using trial and error for many value 4 and get the good result with the value (4 =1.571). The decision of good result achieved by checking the values of (R1 , R2 , G , the value of joints angle 1 7) matrix and checking the accuracy of algorithm by calculating the error between the target and measuring of the position and orientation of end-effector. The D-H parameters and all initial values for simulation are shown in table (2.2). Table(2.2). D-H parameters and all initial values[3] parame ter 1 d1 1 a1 2 d2 2 a2 value /4 2 /4 2 /4 2 /4 2 parame ter ax ay az ex ey ez tgxo tgyo value 0.5 6.5 6.5 2 0.45 2 4 4 param eter tgzo exo eyo ezo tgx tgy tgx tgx value 4 3 3 3 2.971 1.354 2.04 parame ter tgy tgz value /2 /4

29

Table.(2.3).Analytical results for full simulation of algorithm.

Results\ No. of step R1

One: with value of 4=-4.1574. rad

0.7453 0.6614 0.0839 = 0.3334 0.4788 0.8122 0.5774 0.5774 0.5774 0.1258 0.9550 0.2687 = 0.8049 0.0601 0.5904 0.5800 0.2906 0.7610
0.0824 0.9550 0.3369 2.6041 0.0952 0.3439 0.9342 1.6525 = 0.9920 0.0449 0.7610 1.0614 0 0 0 1 0.0000 0.7071 0.7071 2.9710 0.0000 0.7071 0.7071 1.3540 = 1.0000 0.0000 0.0000 2.3090 0 0 1 0

Two: with value of 4=3.8075. rad

0.7453 0.6614 0.0839 = 0.3334 0.4788 0.8122 0.5774 0.5774 0.5774 0.5574 0.3580 0.7491 0.3201 0.7778 = 0.5409 0.5800 0.2906 0.7610
0.0184 0.9983 0.0565 3.2524 0.0330 0.0570 0.9978 0.8712 = 0.9993 0.0129 0.0338 1.4956 0 0 0 1 0.0000 0.7071 0.7071 2.9710 0.0000 0.7071 0.7071 1.3540 = 1.0000 0.0000 0.0000 2.3090 0 0 1 0

R2

Gf

Angle of Joints (rad) Error in position of endeffector

e x 0.3669 e = 0.2985 y e z 1.2476 ex 0.2814 e = 2.2252 y ez 3.8046
1 = 0.7854, 3 = 0.4206, 6 = 0.6187, 2 = 0.6155, 5 = 2.7768, 7 = 1.7259, 1 = 0.7854, 2 = 0.6155, 3 = 0.4206, 5 = 0.3648, 6 = 0.6187, 7 = 0.4039,

30

table 2.3 (Continued)

Results\No. of step three: with trail value of 4=1.571. rad R1
0.7539 0.6485 0.1054 = 0.3136 0.4961 0.8097 0.5774 0.5774 0.5774 0.5679 0.8144 0.1195 = 0.5681 0.4929 0.6590 0.5956 0.3063 0.7426 0.0208 0.7071 0.7068 3.0154 0.0087 0.7070 0.7071 1.2545 = 0.9997 0.0086 0.0208 2.309 0 0 0 1 0.0000 0.7071 0.7071 2.9710 0.0000 0.7071 0.7071 1.3540 = 1.0000 0.0000 0.0000 2.3090 0 0 1 0

R2

Gf

Angle of Joints(rad) Error in position of end-effector

1 = 0.7854, 3 = 0.3942, 6 = 0.6380,

2 = 0.6155, 5 = 2.7503, 7 = 2.3559,

ex 0.1154 e = 0.0455 y ez 0.0090

The results are shown in Figs(2.11 & 2.12) and table(2.3), are the results obtained by the simulation in ref[3]. From the simulation results it can be seen that the algorithm is complex due to the complexity of the equations. These problems will be considered in chapter four where a new algorithm for IKP is proposed.

31

2.4.2.2. Analytical Solution of Inverse Kinematic Manipulator: Describe the real human arm as a manipulator by imaging the form of arm in space and then writing the D-H parameters tables for each joint of arm. Human arm has three main joints ( shoulder joint with 3-DOFs , elbow joint with one DOFs and wrist joint with 3-DOFs). Fig(2.13) shows some manipulators that represent the human arm which are called robot arm. Every manipulator has fixed initial D-H parameters that describes its posture in spaces[34]

-A-

-B-

-C-

-D-

Fig 2.13 Some types of manipulator of a human arm [34] Solving of IK for human arm as manipulator by analytical solution (see, Fig.2.13.A,B,C), can be done by calculating D-H parameters for every DOF in each joint and then using the direct solution [16]. Case study two: Now, solving the numerical example for above algorithm of manipulator with 5-DOFs , it represents the Lynx 6 robot arm[34]. The D-H

32

parameters for this manipulator are shown in table(2.7) and the initial position and orientation are shown below[34]: Table(2.4). D-H parameters of robot arm. joint 1 2 3 4 5 6 rad /4 /3 /3 -/4 /3 0 dcm 0 0 0 0 0 0
0 1 0 0

acm 0 12.065 12.065 14.249 0 0

rad /2 0 0 - /2 0 0

1 0 initial position = 0 0

0 26.3140 0 0 0 19.9640 0 1

The transformation matrix for the given values in table(2.7), can be calculated in general form:
0.7071 0.7071 A1 = 0 0 0 0.7071 0 0 0.7071 0 1 0 7.899 0 0 1 0.5 0.86 0.86 0.5 A2 = 0 1 0 0
0

0 6.03 0 10.04 1 0 0 1

0.86 0 6.03 0.5 0.86 0.5 0 10.04 A3 = 0 1 1 0 0 0 1 0 0 0.86 0 0.7071 0.5 0 0.7071 0 A5 = 0 1 0 7.899 0 0 1 0

0.70 0.70 ; A4 = 0 0 1 0 0 1 ; A6 = 0 0 0 0

10.07 0 0.70 10..07 0 1 0 0 0 1 0 0 0 0 1 0 0 1 0.70

33

The total transformation matrix, 1 T6 between base of manipulator and the end-effector is:
0.2380 0.9539 0.1830 9.7322 0.9451 0.2709 0.1830 9.7322 1 T6 = 0.2241 0.1294 0.9659 32.4841 0 0 1 0

When the algorithm of analytical solution is applied to calculate the angle of joints, the following results are obtained:
1 = tan 1
9.73 = 0.7854 rad 9.73 c3 = 0.86 ; s3 = 0.5

3 = tan 1

0.5 = 0.5267 rad 0.86 371 99.4 2 = tan 1 = 0.5236 rad 371 + 99.4 4 = 15 30 30.1735 = 45.1735 = 0.7884 rad

5 = tan 1

0.49 = 0.5234 rad 0.86

The accuracy of this algorithm is calculated by comparing the target value of end-effector and the calculated value[34] Table (2.5). Error value for end-effector position. Position value px py pz Calculated value(cm) 9.73 9.73 32.48 Target value(cm) 9.3 9.27 31.85 Error value 0.43 0.46 0.63

34

Chapter Three EMG Signal and Virtual Environment

Chapter Three EMG Signal And Virtual Environment

3.1 Introduction The EMG signal which is generated by the muscles activity is located inside the body and diffuses to the skin surface of human body. The EMG signal must be processed and filtered to be a useful signal for recording, monitoring and controlling. All organisms and instruments measuring of the human called "Man Instrument system". Interpersonal simulation allows users to practice their interpersonal skills by communicating with a virtual human (VH). In current interpersonal simulators, the communication between a user and the VH takes the form of a conversation consisting of bidirectional speech and simple gestures. Use of VHs and interpersonal simulation is rapidly expanding to educate users in medical, physical examinations , education and many application. This chapter has two main parts, the first part presents the anatomy of EMG signal, measurement, analysis, and its processing. The second part presents the basic of VR, and the using of mixed virtual reality(MVR) in human application.

3.2 Physiology of (EMG) Signal Human body is a typical complex system, therefore, the biological signals such as skin surface EMG signal contains a lot of control commands. It is very difficult to obtain the same EMG signals for the same motion even with the same person. Furthermore, each muscle activity for a certain motion is highly nonlinear, because the responsibility of each muscle for the motion varies in accordance with joint angles [35]. 35

One muscle is not only concerned with one motion but also with other kinds of motion. Moreover, activity level of each muscle and the way of using each muscle for a certain motion is different from person to another. Physiological condition of the user also affects the activity level of muscles. In addition to these problems, the activity level of some muscles such as biarticular muscle is affected by the motion of the other joint [36]. The relation between the load acting on the other joint and the change in biarticular muscle activity level is different from person to another. Furthermore, the activity level of muscles is affected by the external load acting on the arm, therefore flexible and adaptive nonlinear control must be applied to control the robot with the skin surface EMG signals. Moreover, real-time control ability is required by the controller for a power assist robot since motion delay gives a lot of stress to the user [23]. Understanding EMG signals implies the understanding and the way they generate bioelectrical signals. It also implies the understanding of the "forward problem", that is, how specific mechanisms and phenomena influence the signals, as well as the more difficult problem is "Inverse problem", that is, how the signals reflect certain mechanisms and phenomena and allow their identification and description. The concept of forward and inverse problem is familiar to physiologists and engineers and is strictly associated to the concept of a system as a set of inputs, transfer function and outputs, and of a model, as a set of descriptions and relations associating, under certain conditions and assumptions, the inputs to the outputs [37]. Motor unit (MU) The central motor system and the concept of the MU are presented in Fig.3.1. The central nervous system is organized in a hierarchical fashion. 36

Motor programming takes place in the premotor cortex, the supplementary motor area, and other associated areas of the cortex. Inputs from these areas, from the cerebellum and, to some extent, from the basal ganglia converge to the primary motor cortex and excite or inhibit the various neurons of the primary motor cortex. The outputs from the primary motor cortex have a powerful influence on interneurons and motoneurons of the brain stem and of the spinal cord [35].

Force=motoneuron + muscle fibers

Motor unit

Fig 3.1 A schematic representation of basic motor control mechanisms and of the motor unit and its components [35] In voluntary contractions, force is modulated by a combination of MU recruitment and changes in MU activation frequency. The greater the

37

number of MUs recruited and their discharge frequency, the greater the force will be. During full MU recruitment the muscle force, when activated at any constant discharge frequency, is approximately (2 - 5) kg/cm2, and in general, this is relatively independent of species, gender, age, and training status [38].

3.3 EMG Signal Fundamentals 3.3.1 Electrical Activity Generation of EMG Signal Electrical activity in the muscles arises from the contraction of the muscle fibers, the structure of which is shown in Fig.3.2. Each muscle fiber contains a bunch of myofibrils (long chains of contractile units). The myofibrils contain long chains of contractile units called sarcomeres, which contribute to the force exerted within the muscles [8]. Each of the myofibrils is chemically activated by local neurons, generating an electrical charge that moves up and down the myofibril, activating the chains of sarcomeres (see Fig.3.3). The charge motion generates an electromagnetic field that induces volume conduction, which enables recording of an electrical signal both internally at the muscle and externally at the surface over it [8].

38

muscle fiber myofibril

sarcomere Fig 3.2 Muscle Fibers Composition [8]

Fig 3.3 Generation of Electric Field in Muscle Fibers [8] A schematic representation of the EMG generation is shown in Fig.3.4. The symbol mp(t, F), myoelectric signal as a function of time (t) and the number of firings (F), represents the physiological EMG and it is not recordable or measured. The detected EMG signal that is utilized in the research is the observed signal m(t, F) that is contaminated with electronic noise (almost white) and has lost some of the high frequency components due to the filtering effects at the electrodes [8]. To conclude, considering the EMG signal as a time varying stochastic process gives the possibility to model it as a zero-mean Gaussian distribution, because EMG is the sum of a large number of MUAPs. This random character of the EMG signal enables the later described 39

approximation of EMG amplitude as the square root of the detected signals variance. In addition, the recorded EMG signal is dependent on the type, geometry, and position of the recording electrodes. The depolarization wave also causes chemical changes that result in a mechanical twitch, which is slower than the electrical response, and delayed by (50-100) msec [39]. Dirac Delta Motor Unit Impulse Trains Motoneuron (firing) Action Potentials S P I N A L C O R D h1(t) h2(t) Physiological EMG Signal

Trains

hi(t) mp (t,F)

hp(t) Detection Site m (t,F) r (t) Observed EMG Signal Electrode and Recording Equipment System Noise n(t)

Fig 3.4. EMG Signal Origin Block Diagram [8] 3.3.2 EMG Characteristics A motor unit action potential, or MUAPs, is a summated action potential as detected from all the muscle fibers in the same motor unit. It is

40

the summation of all the MFAPs produced by fibers of the MU. The shape and characteristics of a MUAP are shown in Fig.3.5. It will cover the following characteristics: Duration / Amplitude / Area / Area-Amplitude, the Area to Amplitude Ratio (AAR) / Size Index Firing Rate / Firing Rate per Motor Unit (FR/MU) / # of Phases / % Polyphasic MUAPs / # of Turns [4]. Duration and amplitude characteristics are considered in this work because of their importance in EMG signal identification and classification.

Turns Satellite Phase Baseline crossing

Amplitude

Rise time Duration Fig 3.5 Characteristics of a EMG signal [4] Amplitude: Amplitude is a more variable characteristic than duration, since it depends strongly on the position of the needle electrode. DQEMG reports the peak to peak voltage, which is a standard measure, though some other labs use the voltage difference between the largest negative peak and the baseline for EMG studies. In clinical conditions physicians currently assess 41

amplitude qualitatively by viewing it on an oscilloscope-like display and comparing the height of the signal with other signals in their memory. They can do this partly because they usually maintain the same visual sensitivity, or vertical scale, on their monitor [40]. Amplitude has a log normal distribution within an individual, the range of which varies with disease classification, the amplitude is consistently reduced or normal (5 to 850 mV) in myopathic cases, but can vary between reduced and very high for neuropathic cases. 26 Number of MUAPS
Polymyositis Normal

20

10

14

28 0

14

28

Time (msec) Fig 3.6 Histograms of MUAP duration in normal and myopathic patients [4] Amplitude also has correlations with other characteristics that may make it valuable to graph it on a two dimensional scatter plot with another characteristic. Amplitude and Duration are positively correlated. Amplitude and AAR are negatively correlated which leads to more separable data distributions when the two are plotted together [41].

42

3.4 The Virtual Reality Axiom A virtual reality is here considered to be a reality created by information processing. By definition it does not exist independently in and of itself, as it depends upon processing to exist. If the processing stops then the virtual reality must also cease to exist. In contrast an objective reality simply is, and does not need anything else to sustain it. This suggests two hypotheses about our reality: 1. The objective reality (OR) hypothesis: That our physical reality is an objective reality that exists in and of itself, and being self-contained needs nothing outside of itself to explain it [42]. 2. The VR hypothesis: That our physical reality is a virtual reality that depends upon information processing to exist, which processing must occur outside of itself. Whatever ones personal opinion, these views clearly contradict. If a world is an objective reality it cannot be virtual, and if it is a virtual reality it cannot be objective. These hypothesis are mutually exclusive, and each has implications, e.g. objective reality suggests the universe as a whole is permanent as it has nowhere to come from or go to. For further details, the reader may refer to ref [43 & 44].

3.5 Virtual Reality Requirements What would be necessary to create a VR that behaves like our world? First one must assume information processing constancy, that the properties of information processing are constant, e.g. information processing in our world involves discrete data and calculable algorithms. It is reasonable to assume that a virtual realitys processing works the same way. The requirements include [18]:

43

1. Finite processing allocations. That the processing that creates a VR that behaves like our world allocates its processing in finite amounts. Apart from the fact that we have no concept of what infinite processing means, finite processing allocation suggests that every quanta of matter, time, energy and space has a finite information capacity: recent observations favor cosmological models in which there are fundamental upper bounds on both the information content and information processing rate.. While the processing power needed to run a universe is enormous it is not inconceivable[45]. 2. Autonomy. Once started, a VR that behaves like our world must run itself without further information input. Most human computer simulations require regular data input to run. In a virtual world that behaves like ours, such external data input would constitute a miracle, and in our world miracles are at best rare. This VR simulation must run itself without miracles, i.e. without ongoing data input. 3. Consistent self-registration. A VR that behaves like our world must register itself consistently to internal observers. Most human computer simulations output data to an outside viewer, but we see our world from within. We register reality when light from the world interacts with our eyes (also in the world). For a virtual reality to register itself internal interactions must be consistent with respect to each local observer. 4. Calculability. A VR that behaves like our world must at all times be calculable. If processing is allocated in finite amounts, local calculations must not tend to infinity. While many mathematical calculations can do just this, a calculable VR is guaranteed to avoid that, these requirements constrain any VR model of our world [44].

44

3.6 VRML Coordinate System The coordinate system used in VRML is the right-handed Cartesian coordinate system. The VRML coordinate system is different from the MATLAB, Aerospace Blockset, and SimMechanics coordinate systems. VRML uses the world coordinate system in which the y-axis points upward and the z-axis places objects nearer or farther from the front of the screen. It is important to realize this fact in situations involving the interaction of these different coordinate systems [46]. From Fig.3.7, it can be seen that the coordinate in MATLAB ( Xm=Xv , Ym=-Zv , Zm=Yv ), see equation(3.1).

Fig.3.7: (a). MATLAB graphics coordinate system. (b) VRML coordinate system.
0 0 X v 1 0 0 X v X v X m 1 Y = 0 cos( / 2) sin( / 2) Y = 0 0 1 Y = Z v v v m Z m 0 sin( / 2) cos( / 2) Z v 0 1 0 Z v Yv

(3.1)

Rotation angles in VRML are defined using the right-hand rule. Imagine your right hand holding an axis while your thumb points in the direction of the axis towards its positive end. Your four remaining fingers

45

point in a counter-clockwise direction. This counter-clockwise direction is the positive rotation angle of an object moving around that axis , this is shown in Fig 3.8. In the hierarchical structure of a VRML file, the position and orientation of child objects are specified relative to the parent object. The parent object has its local coordinate space defined by its own position and orientation. Moving the parent object also moves the child objects relative to the parent object. The measurement units used in VRML in the lengths and distances are measured in meters, and all angles are measured in radians [46].

Fig 3.8 Direction of Rotation [47]

3.7 Virtual Human Body The idea of a virtual atlas for the human body is not new and is possible to find different projects based on several image or model databases. Many of them provide an interactive way of visualization of the structures and some are available over the Internet, some time present interactive models but without detailed structures description, or 3D atlas which describe all structures visualized. Other works intend to offer a detailed library of 2D images and present explanations of specific body regions and traditional 2D

46

human that can offer video animations explaining the organs which explain in two skeleton human shape and muscles human structure, see Fig3.9 [48]. Mixed reality human Real environment (RE) and virtual environment (VE) are at two sides, Mixed reality (MR) is in the middle, Augmented reality (AR) is near to the real environment side. Data created by the computer can augment real environment and enhance users comprehension about environment. Augmented Virtuality (AV) is a term created by Milgram (see Fig.3.10). It means add RE images to VE, such as add texture mapping video on virtual objects. This term can increase virtual objects reality degree, decrease virtual object and real objects differences. But VE is entirely virtual. In AR real object, virtual object and user environment must be seamlessly integrated together[49] . Mixed Reality Humans ( MRHs ) are new type of embodied agent that affords touch-driven communication. By affording touch-driven

communication between a human and an embodied agent, MRHs are able to simulate interpersonal scenarios in which touch is a critical component of effective communication Interpersonal simulation allows users to practice their interpersonal skills by communicating with a VH. In current interpersonal simulators, the communication between a user and the VH takes the form of a conversation consisting of bidirectional speech and simple gestures. Use of VHs and interpersonal simulation is rapidly expanding to educate users in medical domains. In this work, MR is used to present different models for human arm, with its principle explanation. Some of these models are presented as a

47

whole human body to apply the arm movements as a part from the body. This is done by connection VR with Simulink\MATLAB.

Fig 3.9 Structure of human skeleton and muscles [ 25]. Mixed reality Real Environment (RE) Augmented Reality (AR) Augmented Virtuality (AV) Virtual Environment (VE)

Fig 3.10 Milgrams Reality-Virtuality Continuum [49]

48

Chapter Four
Proposed Algorithm and Structures for Virtual Human Arm Movements

Chapter Four Proposed Algorithm and Structures for Virtual Human Arm Movements
4.1 Introduction This work concerns with the implementation of the real human arm movements by using the EMG signals and it presented with mixed virtual reality environment. The proposed system combines various methodologies utilized in previous studies and creates a method for manipulating the human arm in a real manner through three joints. Proposed algorithm and structure are build to simulate this system with all real constrains. This chapter consists of four main parts. The first part explains the proposed algorithm for solving the IKP of human arm by analytical solution. The second part presents the simulator of generating EMG signal for human arm and the measurement of real EMG signal of human arm and its simulation. The third part presents the proposed structures for classification EMG signal and recognition of human arm movements by using FRBP-NN, while the fourth part explains different designs of human arm in virtual reality environment according to mixed reality fact.

4.2 Proposed Algorithm of Human Arm Inverse Kinematics The anatomy of limbs and their joints is indeed very complex, (as evidenced by the debate in the literature on the correct method for modeling joint motion ). A combination of single degree- of-freedom joints such a s a revolute joint can be employed (e.g., an elbow joint can be represented as a revolute joint, while the wrist and shoulder joints can be represented by three intersecting revolute joints). For example, if the 49

resultant motion is rotational, the joint will be modeled as a revolute joint. The effect of a spherical joint is modeled as three revolute joints whose axes intersect at the centre of the sphere. Indeed , all anatomical joints can be modeled using basic kinematics pairs. In this work human arm represented by three joints and two links with 7-DOF as shown in chapter two( see Fig 2.9). The proposed algorithm (see Fig.4.3) for solving IKP with analytical solution depends on real human arm movements, real joints constrain and the algebraic trigonometric relation to solve equation of human arm model analytically. The algorithm consists of: Step one: Calculate D-H parameters and all initialization Consider the human arm shown in Fig 2.9 and modeled as a total of 7-DOF, where the wrist and shoulder joints are represented by three intersecting revolute joints. From the D-H parameters one can get the seven transformation matrices which transfer the movements from joint to another and they are denoted by (Ai) where (i=1,,7), the detail of calculation of (Ai) is explain in chapter two. The real constrain of motion for human arm joints are taken from ref[50] and by private communication with the consultations in Baghdad Medical City, as follows:
2 12 1 4 2 6 ; ; 11 8 3 2 5 2 3 ; ; 2 3 6 2 ; ;

The initialization of human arm model had been done, firstly by setting the shoulder joint as origin point and then define the position and orientation of target(end-effector), finally set initial value for all joints angle and position and orientation of pointer (hand of arm). 50

Step two: determine the lengths ( Ltg ,La ,Lb ) The calculation of these lengths ( Ltg ,La ,Lb ) is very important, where it represents the key for solving angle of elbow as will be explained in the next step. The first length represents the distance from shoulder ( origin point ) to wrist, it can be calculated by the norm of vector tg:

Lt g = norm(t g )

(4.1)

The other two lengths are (La and Lb) have fixed values. It can be taken from D-H parameters table where (La=a3 and Lb=a4) see Fig 4.1. step three: calculate the angle of elbow (4) Next, the angle of elbow joint (4) can be calculated by:
2 L2 + L2 Lt g b a 4 = m cos 2 La Lb 1

(4.2)

Only the negative sign in equation (4.2) will be considered due to the actual constrain of the elbow angle, i.e.:
2 L2 + L2 Lt g b a 4 = cos 2 La Lb 1

51

Envelop of elbow

Lt g
/12 La 4 5/6 Fig 4.1 Form of Human Arm in Elbow Joint Step four: determine the form of equation of shoulder joint It is well known the position of wrist (which represent the position of end-effector ) depends only upon the first four joints and this fact can be written in the equation(4.3) :
0 0 A1 A 2 A 3 A 4 = t g 0 1

W
Lb

(4.3)

The first three matrices ( A1, A2 and A3) represents the transformation of shoulder to elbow while A4 represents the transformation from elbow to wrist. The general form of equation(4.3) is shown in appendix A, and its analytical solution is complex and not available in the literature. However, the work reported in this thesis presents an analytical solution using an algorithm based on equation (4.2), i.e. computing (4) first. Step five: checking the humanely form of arm In this step the human arm for a given D-H parameters based on 7DOF set of axes in space. This description of the arm is sufficient to describe the jobs that the arm have to do. The validity of the solution of 52

equation (4.3) will be based on its unique solution as obtained by comparing its two sides. The next step is to determine the angles of the shoulder. If equation(4.3) has no solution, this means it represents a manipulator with 7-DOF, and it can be solved by using the methods given in references[14,15,16,17]. Usually numerical methods are used if and only if all transformation matrices have no singularity. Step six: find the shoulder joint angles The shoulder joint is spherical joint and the solution of the rotation angles analytically is very complex due to the cross-coupling between two or three axes; therefore the analytical solution will depend upon two sets of equations the first set are shown in equation(4.3) and the second set are shown in equation(4.4).
0 e x 0 e A1 A2 A3 = y 0 e z 1 1

e x La s1c 2 e = L s ; & y a 2 e z La c1c2

(4.4)

The position of elbow ( ex , ey , ez ) in equation(4.4) must be determined in order to use them for solving the joint angles. From Fig4.2, it can be seen that the position of elbow is represented by a circle with center o and radius R. d1 s La R e Fig 4.2 Elbow Position Control 53 o Lb tg LH w H RH

start Find D-H Parameters and initialization all variables

Calculate La, Lb & Ltg Calculate 4 Eq.(4.2) Apply Eq.(4.3) is there analytical solution for Eq.(4.3)

Yes

No

Find the solution for 1 2 3 Find the form In Eq.(4.7) Find the solution for 5 6 7

Yes

Is there Singularity of matrices A1---A4

No solution

No Find the solution as manipulator

end Fig 4.3 The Proposed Algorithm for Inverse Kinematic by Analytical Solution 54

By comparing equations(4.3) & (4.4), and geometric relations in Fig.4.3, the following equations are determined:
Lb s 4 c3 = t g x c 2 t g z s 2 Lb s 4 c3 = t g y c1 + t g x s1 s 2 + t g z s1c2 e x = t g x c2 t g z s2 e y = t g y c1 + t g x s1 s 2 + t g z s1c 2

(4.5)

From the above equations the relation of shoulder joint angles can be obtained as:
1 = tan 1
ex ey ey cL s2 = 1 a ex s1 La c1 0 otherwise

s 2 = tan 1 2 ez L a s1c2 t g + c1c2 t g + s 2 t g x z z 3 = tan 1 c1t g x + s1t g y

(4.6)

Step seven: solving for the wrist angles Once the values of 1, 2, 3 , and 4 have been calculated the wrist angles can be isolated by rearranging of the following equation:
Rw = R5 R6 R7 = (R1R2 R3 R4 ) Rg = Rend effector
1

c5c7 + s5 s6 s7 R5 R6 R7 = c6 s7 s5c7 + c5 s6 s7

c5 s7 + s5 s6c7 c6c7 s5 s7 + c5 s6c7

s5c6 ef11 ef12 ; R ef22 s6 end effector = ef 21 ef31 ef32 c5c6

ef13 ef23 ef33

(4.7)

55

There are two values for 6 that can be found from equation(4.7) which are:
sin1 (ef23 ) 6 = ( + sin1 (ef )) 23

(4.8)

According to the value of cosine of 6 can be find the values of 5 and 6 are:
ef13 c6 1 5 = tan ef 33 1 ef31 c6 5 = tan ef 11 c6 0 ; else c6 = 0 ef21 7 = 0 c6 1 7 = tan ef 22 c6

if

(4.9)

Now all angles ( 1,, 7 ) are calculated, therefore the solution can be checked by calculated the forward kinematic and then calculate the error between the results and the target values to determined the accuracy of solution, this is done by equation(4.10).
G forward = A1 A2 A3 A4 A5 A6 A7 t g x forward t g y forward t g z forward err x err y err z = G forward ( 4,1 : 3) t g t g x x forward = t g y t g y forward t t g z forward gz

(4.10)

56

4.3 Human Arm EMG Processing It is imperative that the raw EMG signal can be monitored in realtime, as it is recorded and as the electrodes are placed on the subject. Ideally, this monitoring must be performed as the EMG signal is recorded as it is often difficult to differentiate between signal and noise if any processing has been done to the EMG signal. One disadvantage of using some computerized collection systems is that many do not provide the ability to see a raw EMG signal in real time, Fig 4.4, shows an unprocessed EMG signal[51]. Amplitude (mV) Window of EMG signal for Biceps muscle

Time (sec) Fig 4.4 Real EMG Signal [simulated by EMGlab] A novice electromyographer may have some trouble in

determining if any problems exist in the EMG signal. However, there are several items that can be quickly spotted a wavering base line is a common indication that low frequency movement artifacts are present. Large, individual spikes can be also indicative of motion of the pickup electrode on the skin surface. Other things to look for are common signals across all channels (possibly a poor ground reference electrode) and/or an underlying 50 or 60 Hz line power signal superimposed on the signal [51]. Filtering the EMG signal There are two basic filters that can be applied to EMG signals : high-pass and low-pass. Other filters such as notch and band-pass filters are just combinations of these two basic filters. Fig.4.5, shows zoom in of window for EMG signal for Biceps EMG signal about time(0-0.45 msec). 57

Amplitude (mV)

Zoom in EMG signal for Biceps muscle

Time (sec) Fig.4.5. EMG Signal Without Filtering. Actual EMG data has frequency 10-15 Hz or higher, depending on

the activity (10 Hz for normal walking and 15 Hz for more rapid movements). Clinical EMG data is usually low-pass filtered at 300-600 Hz for surface EMG, or 1,000 Hz or higher for fine wire EMG recordings because of including the measuring environment. The choice of whether to filter the data or not, and the filter points to use, depends partly on the quality of the raw data and partly on the intended use of the processed data [51]. EMG data for clinical motion analysis use tend to be more heavily filtered partly to remove motion and other artifacts that may be unavoidable in clinical subjects, and partly because the timing of muscle activity is (in most cases) more significant than the fine details of the content of the EMG activity. Fig.4.6, display the affect of low pass (625,1250)Hz and high pass (2500Hz) filters on EMG signal. Filtered with low pass freq.(625Hz)

Amplitude (mV)

Filtered with low pass freq.(1250Hz)

Filtered with high pass freq.(2500Hz)

Time (sec) Fig 4.6 Filtered EMG Signal (for the signal shown in Fig.4.5) 58

Amplitude Normalization The timing of muscle activity can be easily determined from the raw EMG. The onset and cessation of EMG activity can then be correlated to the stance and swing periods of gait. The difference in the EMG amplitude of a single muscle represents varying levels of activity. As more muscle strength is required, additional motor units are added, thus visually the EMG signal has larger amplitude and becomes denser. Normalization of the EMG signal amplitude is required when attempting to compare the signals from different muscles. One simple method of amplitude normalization is to normalize each of the displayed EMG channels to the maximum of the signal throughout the current trial within the channel. Thus, the muscle effort can then be displayed as a simple percent of the maximum achieved by that muscle in the current trial. This is a simple method that works very reliably for most common EMG signals. This type of EMG signal normalization is called Maximum Amplitude Normalization is considered in this work by equation(4.11).
s(k ) ; + K n * max (s ( k ) )k =1: N

s n (k ) =

max (s ( k ) )k =1:N

K n = 0 .45;

(4.11)

Where: sn : represents the normalized signal. K n : represents the gain for tolerance of the signal normalization, obtained by trial and error. An alternative normalization method is to make an individual recording from each muscle, while the subject is attempting to generate a maximum contraction for the individual muscle (or muscle group). Each EMG channel is then normalized to the maximum amplitude of the recorded EMG signal during the test. This method is more complex and time consuming than the simpler Amplitude Normalization method but is considered to provide a more accurate comparison of the recorded EMG

59

signal. The type of EMG signal normalization is called Maximum Voluntary Contraction Normalization often abbreviated to MVC[8]. 4.4 Simulation of Generated EMG Signal The best modeling of clinical EMG signals was achieved in algorithm by Hamilton at 2005[9] .This algorithm is simulated by using Matlab software and using the GUI approach to get full mathematical simulated model for generating real EMG signal of a specific human muscle as shown in Fig4.7 [52]. This simulator has many options used with rearrangement to generate EMG signal for human arm muscle. The option of the simulator after rearrangement can be summarized as: Muscle: This popup shows the muscle being simulated. One can select from the list ( which adding for human arm muscles), of already defined muscles or select "Custom." to define a new one. Clicking the edit button allows one to modify the muscle parameters. Helping for specification of human arm muscles is added to this window to help the user for generation EMG data more nearest to clinical data, Fig 4.8 shows the window for this option.

Fig 4.7 The EMG Signal Simulator 60

Fig 4.8 Window for Select or Design Human Arm Muscle Electrode: This popup shows the electrode being simulated. The user can select from the list of already defined electrodes or select "Custom..." to define a new one. Clicking the edit button allows the user to modify the electrode parameters. The "Electrode" panel allows the user to save new or modified sets of electrode parameters. Helping for types of electrode, advantages and disadvantages for each type is added to the window to help the user for using the electrodes. Selecting "Multiple Electrodes" allows the user to simulate simultaneous recordings from more than one electrode. Select "Add" from the number popup to add a new electrode. Each electrode can be of a different type. If the user specify multiple electrodes, then the program creates separate data files for each electrode. The signal from the first electrode is still named filename.dat, and the signals from the other electrodes are named filenameI.dat, where "I" is the electrode number. Fig 4.9 shows the window for this option. 61

Fig 4.9 Window for Select the Type of Electrode Advanced: This allows the user to specify some advanced simulation preferences. Include all units: This causes the firing patterns of all the active motor units to be included in the annotation file, not just the ones closest to the electrode. Output directory: Specifies the directory in which to write the data files. The default directory is the data subdirectory in the simulator directory. Output root filename: Specifies the root filename for the output files. Signal duration: Specifies the length of each signal, in seconds. Contraction: Selects a contraction. To add additional contractions select "Add" from the popup. %MVC: Specifies the contractile level for the selected contraction. Position: Specifies the x, y, and z electrode locations (in mm) for the selected contraction. The z coordinate is the distance from the muscle endplate along the muscle axis. Use the "electrode" popup to specify

62

locations for multiple electrodes. Note that you can specify different locations for each electrode in each contraction. Delete: Deletes the current contraction. Run Simulation. Runs the specified set of simulations (one simulation per electrode per contraction). The simulations can take a fair amount of time, depending on the specified signal duration. The trace statements from the simulation routines are displayed in the command window. Cancel. Quits the simulator and returns to Matlab. After press the Run Simulation button , each simulation may take a minute or more. The program creates the following files for each contraction: Filename.dat EMG signal. Filename.hea header file ( allows signals to be read by another software). Filename.eaf. annotation file These file can be use to analysis the EMG signal.

4.5 Simulation of Generated EMG and Measured EMG Signals The program, which explains the graphic of the files obtained from the EMG simulator, is called EMGlab. This program is build in Matlab software and the description of run EMGlab in Matlab command window as follows [53]. The program runs in a single Matlab window, which is divided into five panels: At the top is the signal panel, which displays a segment of the EMG signal. Below that is the template panel, which displays the MUAP templates. Below that on the left is the firing panel, which displays the firing patterns of the identified MUs. To the right is the close-up panel, which displays a section of the EMG signal at an expanded scale. At the very bottom is the navigation panel which displays a thumbnail of the 63

EMG signal. The buttons on the edges of the panels are used to change the display characteristics[54]. Fig.4.10 shows the output of EMGlab with data of EMG signal. It receive the data that are generated by EMG simulator and the practical measured data and explain the decomposition of the signal and register the data of this signal. The inserted symbols in the window are: +, - zoom in or out vertically <, > scroll left or right, and ||, | | zoom in or out horizontally Other buttons on the firing and close-up panels are described below. At the top of the window are pull-down menus for selecting the channel and high-pass filtering.

Signal and its decomposition Decomposition for each peak Zoom at specific place

Fig 4.10 The Output of EMGlab for EMG Signal

64

4.6 EMG Recognition System The recognition system of EMG patterns consists of three stages. The first, time series data for EMG is measured by electrodes in the input part or generation by EMG simulator and then processing will be applied to this signal such as filtering, amplification and amplitude normalization. Fig4.11 summaries these processes.

Measured EMG

OR

EMG Generator ( Simulator )

Filtering of EMG Amplification of EMG Amplitude normalization of EMG Collection data of EMG 1 I Identification part Learning algorithm

Signal processing part

1 Target EMG for learning

Fig 4.11 Flowchart for Processing EMG Signal and Identification of NN 65

There are needle type electrodes and surface type electrodes to measure EMG signal . The needle (insertion)-type electrode can measure EMG from a specific muscle because of sticking muscle directly. But, it is not preferable for the subject and in danger of sick contagion. As compared with this, the surface-type electrode is preferable for the subject and not in danger of sick contagion and pain. On the other hand, it is difficult to use in practice. The second stage is identification on FRBP NN (see Fig 4.12) by EMG signal, it is done by parallel form of identify NN, where the data of EMG arrangement by set with diminution (PI). The length of I represents the number inputs to NN, which denote number of muscles that share the movement of a human arm. The numbers of muscles for all real movements about 22 muscles. The length of P represents the number of samples used in learning the NN [55].

I m

Fig 4.12 FRBP Network with I inputs, one hidden layer of(m) unit and (n) outputs [55] The rule used for forward NN is delta rule and then backward by fast momentum back-propagation algorithm, by using the error between the output of NN and the data of raw EMG, this process was explained in previous section and it is shown in Fig.4.11.

66

The Third stage is test recognition system, that is done by taking new raw EMG signal and recollecting this data using the same procedure in signal processing part as shown in Fig.4.11. After that, setting the NN by biases and weight to identify NN in second stage above, enter the new data to forward NN. The output of the recognition NN is compared with output of the identify NN, the difference between them is entered to check for amplitude in order to decide human arm muscle, where the identification of NN is done by all muscles of human arm. This procedure is summarized by Fig 4.13. Forward NN ( without Learning ) Reprocessing of EMG data same as input part

EMG i/p Signal

I m

The decision of movement of HA according to action of certain muscle

Data Base of NN identifier Fig 4.13 Block Diagram of Neural Network for Recognition Movement

67

4.7 Object Design in Virtual Reality The requirements for design in VRML are explained in chapter three. In HVR there are many designs and every one is related to the function of design. In this work we are interested in the design of an object to implement real movements of human arm. In the real human arm there are about 10 main movements and about unknown coupling joints movements. The biology of these movement and its kinematics was explained in chapter two. The real control and drive for these movements were explained in chapter four. The implementation of all this work can be done either as a practice object or in VA approach. This section presents the design of real human arm to connect with another program to run it. Before starts, the explanation of procedure for design in VRML will presents.

4.7.1 Object Structure Design in VRML The design in VRML depends on the information of designer and his imaging for object. There are two choices for design in VR, the first is the standard configuration such as sphere, cone, cylinder,etc and the second choice is free design by select indexed face set button to get many configuration with free rearrangement of points; therefore every real form design in the second choice. Fig.4.14, explain the design of six main parts of real skeleton human arm. The procedure for design has two stages, it starts with building these parts every one alone and check the shape similar to related real human arm part. These bones of human arm cannot be simulated in VR when the standard shape in VR library is used, where the bones are not uniform shape; therefore the design was achieved by using indexed face set in VR.

68

The sequences of design for each part of human arm by using indexed face set in VR is very difficult, where the using of this choice for design must be controlled for all points in form. Fig.4.15 shows the design of scapula and clavicle bones in same form, there are many points in this form every point has proper position to get true final shape . The second stage in the design is very important, where the connection between all parts of design will be achieved to get the final object after defining its origin point which represents the starting point. The origin is set at the scapula and clavicle bones then the humerus bone is connected to the bottom children the scapula and clavicle bones. The same procedure was followed other parts.

shoulder joint

clavicle

humerus elbow joint

scapula

ulna

a
hand

radius wrist joint

Fig 4.14 Built Virtual Right Human Arm The three spheres, that are used in Fig.4.15, denoted by shoulder, elbow and wrist joints, used in design give the ability for object to implements the real movements such as rotation around axes and twist and so on. 69

points of design

Fig 4.15 Design Object in Indexed Face Set in VR 4.7.2 Human Body Project in VRML There are many designs for human body, where every one is built according to function for implementation of human body utility practice project. Hence according to required movements of human body there are two forms of human body design as follows: 4.7.2.1 Skeleton of Human Body The skeleton human arm can be designed in VR as one unit, this design is used as dummy skeleton. It can not do any function of human skeleton such as arm, leg movement. This design is shown in Fig.4.18a, this object used in VR of carton movie and same application. The other design is to build the skeleton by collecting each bone alone and then group them by selecting the origin of design and connect the bones same as in a real skeleton human body. This design has wide range of implementation in medical, education, and cinema applications. The using of real skeleton in these application is very difficult and the 70

similarity of the design with real skeleton gives it high reliability in practical implementation, (see Fig.4.16b.).

Fig 4.16a Design Skeleton Human Body as One Unit

Fig 4.16b Design Skeleton Human Body

71

4.7.2.2 Complete Human Body The object built is by the same methods used in human skeleton as one unit or same parts for real human body, as shown in Fig4.17. The importance of this design is that it can recognize the sex of the human from the design by adding the final form that classifies the human to male, female, girl, boy , old man and old woman. Also, the human can be classified according to his job by adding cloths proper to a job such as teacher, solider,.etc.

Fig 4.17 Human Body for Girl and Boy [56]

72

Chapter Five Simulation Results and Discussion

Chapter Five Simulation Results and Discussion

5.1 Introduction This chapter presents a proposed GUI package, which includes the simulation of IKP, EMG simulation and its processing, and finally human arm virtual model implementation. This package was built using MATLAB Ver.R2009a package. 5.2 Proposed Package of a Complete Virtual Human Arm Driven by EMG Signal This package presents the overall system design simulation and it contains many windows, each one has some related function. The presentation of the results depends on the sequence of running the windows and display them by tables, curves and movies. The starting window of GUI is shown in Fig 5.1. It explains the overall idea of the work. Button Next is used to transfer to the next window shown in Fig 5.2.

Fig 5.1 Starting Window in GUI Simulation System

73

Fig 5.2 represents the system window with three main parts. The help of each part can be executed by selecting its button. Fig 5.3(a,b,c) presents the help windows of the specified subject. Sample of help frames are given in appendix B.

Fig 5.2 The Structure of the Simulated System

-a-

-b-

-cFig 5.3.a,b,c The Helping Windows in GUI System Moreover, Fig 5.2 includes System Run button, where complete simulation may be started.

74

5.3 Measurement of Real EMG Signal Fig 5.4 represents the EMG signal window, where the EMG signal has two main sources, either measurement of real EMG or generated by using EMG simulator. Each one has advantages and disadvantages from the point of view of accuracy and reality.

Fig 5.4 GUI of EMG signal in Recognition System Fig 5.5 represents the real measurement system of EMG signal. Locally (i.e. in Iraq) there are many problems associated with the acquisition of real EMG measurement. The systems in the educational hospitals in AL- Kadhimiyaa , AL Yarmook, Baghdad Medical City, and some private Clinical centers do not justify this research requirements.

sensors stage

amplifier stage

Processing and monitoring stage

Fig 5.5 Real Measuring System

75

Actually there are many sensor types (electrodes) of EMG signal, but locally there is only a needle electrode type. The usual procedure for using this type is by inserting the needle in the proper placement of muscle to touch the fiber and sense the EMG signal. Fig 5.6 represents sample photos for the real EMG signal measurement by needle electrode at Baghdad Medical City hospital.

Fig 5.6 Real Measurement EMG signal by Using Needle Electrode Measurement of real EMG signal by needle electrode has many problems such as high pain to the patient and side effects on muscles. Needle electrode is used only for up normal muscles. Human arm movement recognition requires number of surface channels equal to the number of recognized muscles. For example, four channels must be available at the same time to recognize the wrist joint movements. When selecting the first button in Fig 5.4, real EMG data are obtained, taking into consideration all the above mentioned technical problems. The output on the screen of Micromid measuring system in Baghdad Medical City hospital is shown in Fig 5.7. 76

Fig 5.7 Screen Graph of Real EMG Measurement System for Miled Resistance on Deltiod Muscle of Right Human Arm The measured EMG data in Baghdad Medical City hospital are stored in a file format (filename.TRC). This data can be read and collected using EMGlab program as explained in chapter four. These measurement data are taken from every muscle for three cases: no resistance (no activity of muscle), low resistance (Miled) and high resistance. EMG simulator is used to overcome the problems of locally real measuring system ( needle electrode and single channel ). The EMG data, which is generated by the simulator, depends on three main parameters: muscle specifications, electrode types and electrode position. In this work only muscle specifications are taken into consideration, i.e. taking fixed values for electrode type and position. There are four muscle parameters which affect the EMG data. The selected values of these parameters depend upon knowledge of the user about the muscles. All these parameters represent biological specification of muscle: number of MUs in muscle, muscle fiber density, muscle fiber cross section area, and MU diameter ( minimum and maximum values). 77

The degree of reliability of generated EMG

data depends on the

closeness of the value to those of the real biological muscle. Fig.5.8 shows the window settings of the specifications of biceps muscle of 20 years old human with a normal body. The real value of the muscle parameters can be obtained by studying the anatomy of muscles in detail. Depending on the practical data obtained from medical table and consultation with the specialist [57], the muscle parameters adopted are as shown in table 5.2.

Fig 5.8 Setting the Specification of Muscle Table 5.2 Standard Parameters of Muscle Human Arm Muscle name biceps triceps deltoid flexor carpi radial extensor carpi radial flexor carpi ulna extensor carpi ulna No. of MU 400 350 450 200 250 100 150 Fiber Density ( per mm2 ) 15 10 20 7 8 5 6 Fiber Area ( mm2 ) 0.0075 0.0055 0.0085 0.0030 0.0035 0.0015 0.0025 Range MU diameter ( mm ) 2 - 10 29 3 - 11 1.5 - 8 1.5 8 26 26

78

Fig.5.9 represents the generation of EMG signal by EMG simulator. There are two choices to get the EMG data. The first by EMG simulator with specified values for the parameters of the muscle. The results are data files with three types of format: filename.hea, filename.eaf and filename.dat. The details of the EMG simulator was explained in chapter four and the GUI window for running EMG simulator is shown in Fig.5.10. The second choice in Fig.5.9, is used to produce data of EMG signal by selecting a specific muscle of the human arm. The data is calculated by selecting the specification of normal human body as given in table.5.2.

Fig 5.9 Generation Window for EMG Signal

Fig 5.10 Running Window for EMG simulator with Deltiod Muscle 79

Finally the main difference between the two sources of EMG signals is the reality of the signal where, the first source is the measurement of real EMG signal direct from muscles considering all real constrains while the second source obtained by EMG simulator with specific data of parameters of muscles. The processing of collected EMG data, was explained in chapter four. Filtering, normalizing, and then collecting the data is done by the GUI window shown in Fig 5.11.

Fig 5.11 Window for Processing on EMG Signal As an example for using EMGlab real data measurements for deltoid muscle will be used and its results will be compared with the results of EMGlab by EMG data for deltoid muscle generated by EMG simulator. Fig.5.12, presents real EMG deltoid signal and the steps of filtering by low pass filter with cut-off frequency 625Hz, 1250Hz and by high pass filter with cut-off frequency 2500Hz.

80

-a-

-b-

-c-

-d-

Fig 5.12 Real Deltiod EMG Signal with high resistance a. Unfiltered signal b. Filtering signal with low pass filter (625Hz) c. Filtering signal with low pass filter (1250Hz) d. Filtering signal with high pass filter (2500Hz) The same procedure is applied to the EMG data obtained by simulation and the results are shown in Fig 5.13.

81

-a-

-b-

-c-

-d-

Fig 5.13 Deltiod EMG Signal Generated and Its Filtering a. Unfiltered signal b. Filtering signal with low pass filter (625Hz) c. Filtering signal with low pass filter (1250Hz) d. Filtering signal with high pass filter (2500Hz) By comparing Figs 5.12 & 5.13, it can be noticed that the EMG simulator has effective ability to generate EMG signal for specific muscle similar to real data of EMG signal and the ability of EMGlab to collect 82

the signal and filtering the signal by two type of filters. The effect of using low pass filter and high pass filter is shown in above Figs. They cancel the noise and smooth the signal. As it was mentioned earlier, there are two parameters that influence the EMG signal estimation, these are the amplitude and period of the signal. The amplification of EMG signal ( first choice in Fig5.5 ) is used with the real measuring system only. The amplitude of signal is sometimes very low, where this phenomena occurs for many reasons, some of them are clinical, and the rest is caused by the position of the EMG sensor. Hence, after measuring this signal it will be amplified by certain gain in the range of (10 - 10000). The normalization of EMG signal is needed when it is used with NN approach to recognize the movements of the human arm. Two method for normalization were explained in chapter four. For NN approach, which is used in human arm movement recognition, EMG signal normalization is needed. Maximum amplitude normalization (equation(4.11)) method was used in this work. . 5.4 Neural Network Recognition System Fig 5.14 represents the GUI window for calling NN recognition system. Clicking the identification button initiates the identification process. In this design number of samples are chosen to be (1600 samples) and this represents the minimum value with acceptable error for identification. The number of samples is adjusted by trial and error. The simulation of the block diagram shown in Fig.4.11 is activated by using the first button in Fig.5.14. The EMG signals are used with identification process generated by EMG simulator for main seven muscles of human arm (biceps, triceps, deltoid, flexor carpi radial, extensor carpi radial, flexor carpi ulna and extensor carpi ulna ).

83

Mechanical activation of these muscles will cause the corresponding movements of human arm ( supintion of elbow, flexion of forearm, abduction of arm at shoulder, flexion of wrist, extension of wrist ,abduction of wrist and adduction of wrist ). Figs 5.15 and 5.16 present the EMG identification data with the dimension (16007). These data are used as an input to FRBP NN identifier.

Fig 5.14 Window of NN Recognition System

84

Amplitude (V) 3500 3000 2500 2000 1500 1000 500 0 -500 -1000 -1500

200

400

600

800 1000 1200 No. of samples

1400

1600

Fig 5.15 Selected Samples of Deltoid EMG Signal for Identification -a-b-

-c-

-d-

-e-

-f-

Fig 5.16 Zoom in of 1600 samples from EMG Signal of (a). Triceps muscle (b). Biceps muscle (c). Extensor Crapi Radail muscle (d). Extensor Crapi Ulna muscle (e). Flexor Crapi Radail muscle (f). Flexor Crapi Ulna muscle 85

The recognition of movement under the real complex constrains is very complex. The recognition design will be implemented with different procedures depending on the two sources of EMG signal. When selecting the measurement of real EMG button from Fig 5.4 the processed window of Fig 5.17 is obtained, while selecting the button for generation of EMG data by EMG simulator the processed window of Fig 5.9 is obtained. In this work the following points give the EMG sources: a. measurement of real EMG signal from normal arm muscles. b. measurement of up normal signal from ill arm muscles ( such as neuropathic and myopathic ). c. EMG signal generated by EMG simulator with standard specification of arm muscles(by using biology tables) d. EMG signal generated by EMG simulator by assuming random parameters of simulator.

Fig 5.17 GUI of Real EMG Measurement From the above description of information about EMG ( specifications and measuring methods ), the procedure to recognize movement can be done by using the following steps: 86

Step one: The identification system has two procedures depending on the reliability of the measured signal. If the signal measured by sources ( b & d), it must be classified as EMG signal for human arm or unknown EMG signal. The structure used for NN identifier is FRBP-NN with dimensions (7-30-7), values of learning rate (r=0.01) and momentum term (mu=0.015). This is shown in Fig.4.11. The NN identifier has seven inputs with EMG signals, (shown in Figs 5.15&5.16 ), thirty neurons in hidden layer and seven outputs. The training of NN was achieved by (1600) samples with accepted error by mean square equal to (0.0005). After the learning of network all parameters of NN are stored in order to use them with NN classifier. The outputs of NN identifier are shown in Figs.5.18&5.19.
v

3000

Identification of EMG signal for Bicepe Muscle NN output EMG Signal

2000

Amplitude ( v

1000

)
0 -1000 -2000 -3000 0

200

400

600 800 1000 No. of samples

1200

1400

1600

Fig 5.18 Identification process for NN

87

3500 3000 2500 Amplitude ( v )

Identification of EMG signal for Deltiodl Muscle NN output EMG signal

v 2500

Identification of EMG signal for FlexorCrapi Radail Muscle EMG signal NN output

2000 1500 1000

1500 1000 500 0 -500

Amplitude ( v )
200 400 600 800 1000 No. of samples 1200 1400 1600

2000

500 0 -500

-1000 -1500 0
v 2500

-1000 0
v 3500

200

400

600 800 1000 No. of samplrs

1200

1400

1600

Identification of EMG signal for Extensor Crapi Ulna Muscle EMG signal. NN output

Identification of EMG signal for Flexor Crapi Ulna Muscle

2000 1500 1000

3000 2500

EMG signal NN output

A m plitude ( v )

Amplitude ( v )

2000 1500 1000 500 0 -500

500 0 -500

-1000

-1000 0
v

200

400

600 800 1000 No. of samples

1200

1400

1600

-1500 0
v 3500

200

400

600

800

1000

1200

1400

1600

No. of samples
Identification of EMG signal for Triceps Muscle

3000 2500 2000 1500 1000 500 0

Identification of EMG signal for Extensor Crapi Radail Muscle NN output EMG signal

3000 2500 Amplitude ( v )

Amplitude ( v )

2000 1500 1000 500 0 -500

-500 -1000 0

-1000

200

400

600 800 1000 No. of samples

1200

1400

1600

-1500 0

200

400

600 800 1000 No. of samples

1200

1400

1600

Fig 5.19 Identification of NN for six EMG human arm muscle signals

88

Step two : Fig 5.20 represents the GUI window of this step. Any tested signal is applied in the forward direction of the NN in the proposed structure and its output is compared with NN identifier data base to classify the tested signal as human arm EMG signal or unknown EMG signal. The comparison procedure is as follows: For each sample the accepted amplitude difference between tested signal and data base is less than ( 2.5% ). If the number of accepted samples is greater than ( 90% ), the tested signal will be classified as human arm muscle EMG signal and this result will appear in the decision label in Fig.5.20.

Fig 5.20 Neural Network for classified EMG signal Step three : This step will start the operation of NN to recognize the movements of the human arm depending on the EMG signal. The signal after steps (1)&(2) may be received from any of the four sources as explained above. The signal from sources (a)&(c) will enter directly to this step. The recognition algorithm is built on two biological facts. The first one is the amplitude of EMG signal depending on four factors for the generation of EMG and these parameters differ for any given muscle. The 89

second fact is the activation time of muscles which can never occur simultaneously. These facts can be seen in Fig 5.21, which shows EMGs for the seven muscles during the same period.
4000
max 3 max 2 max 6 max 5 max 4 max 7

3000 2000 1000 0 -1000 -2000 -3000 0

EMG1

max 1

Amplitude ( v )

200

400
EMG2

600
EMG3

800
EMG4

1000
EMG5

No. of samples 1200 1400 1600

EMG6 EMG7

Fig 5.21 EMG Signals for Seven Muscles The procedure for recognition starts with the identification of NN with maximum value of normalized EMG data for seven channels. These channels are connected to the human arm at the same time. This environment of measurement can not be found locally. The reasons for that was explained in the previous section; therefore the EMG used was obtained by using EMG simulator. In this step, the NN identifier has the same structure of NN identifier in step one with learning rate (r=0.9), momentum term (mu=0.95) and (500) iteration. Fig 5.22 represents the NN identification results.

90

0.7 0.6
tricps muscle bicepe muscle

deltiod muscle

0.5
extensor crapi radial muscle

flexor crapi radial muscle

Normalized Amplitude

0.4
extensor crapi ulna muscle

flexor crapi ulna muscle

0.3 0.2 0.1 0 0

50

100

150

0.7 0.6 0.5

200 250 300 No.of iteration

deltiod muscle

350

400

450

500

bicepe muscle tricps muscle flexor crapi radial muscle extensor crapi radial muscle

Normalized A mplitude

0.4 0.3 0.2 0.1 0 0

flexor crapi ulna muscle extensor crapi ulna muscle

10

15

20 25 30 No.of iteration

35

40

45

50

Fig 5.22 Identification for Normalized Max. Value of Amplitude for Seven EMG signals Step four : The finial step of the design for movement recognition system is achieved by recognizing the normalized maximum amplitude for test

91

signal by using the same NN in forward direction only. After that the output of net is compared to the output of NN identifier. The difference must be less then (0.05%) for any one of the seven values, thus relating it to human arm muscle. Depending on muscle recognition the human arm movements may be recognized. Fig 5.23 represents the GUI window of this step.

either - or

Fig 5.23 GUI for Simulation of Recognition System by EMG Signal 92

5.5 Kinematics of Human Arm The case under consideration was shown in Fig.2.12, with the D-H parameter values given in table 5.3: Table.5.3. Numeric Value for D-H Parameters of Human Arm. Frame (joint) 1 2 3 4 5 6 7 i
(rad)

di
(cm)

aj
(cm)

j
(rad)

1 2 3 4 5 6 7

0 0 0 0 0 0 0

0 0 20 25 0 0 0

/2 /2 - /2 /2 - / 2 /2 - /2

The position of end-effector is assumed to be [29.71 , 13.45 , 23.09] in cm, with the orientation as ( , /2 , /4) in x-axis, y-axis and z-axis respectively. All initial joint angle values are assumed to be zero. As in the proposed algorithm explained in section 4.2, and according to table.5.3: La=20 cm and Lb=25 cm. After setting of all initial condition in first step, equation(4.1) is applied to calculate Ltg=39.95 cm. The rotation matrix and position of end-effector can be calculated as:
0 0.7071 0.7071 29.71 0 0.7071 0.7071 13.45 Gendeffector = 1 0 0 23.09 0 0 1 0

(5.1)

Considering Fig.4.1, and equation(4.2), (4) can be calculated as(4=1.5728 rad). Fig 5.24 gives the values of (4) with the position of end-effector in x-axis (0:0.1:2) and zero in (y,z) axes.

93

3.5 3 the value of theta4 ( rad ) 2.5 2 1.5 1 0.5 0 0

note: the value of E.F in y & z axes are set to zero

0.2

0.4

0.6 0.8 1 1.2 1.4 position of E.F. in x-axis

1.6

1.8

Fig 5.24 Form of (4), with t gy = t gz = 0 and t gx = 0 : 0.1 : 2 When applying equation(4.3), to the case under consideration the following equations are obtained:
(c1 s 2 c3 c1 s 3 )a 4 c1 s 2 a3 = t gx c 2 c3 a 4 c 2 a 3 = t gy ( s1 s 2 c3 + s1 s 3 )a 4 s1 s 2 a 3 = t gx where : s i = sin( i ), ci = cos( i )

(5.2)

It is quit clear that the above equations have unique analytical solution where, the values of shoulder joint rotations 1 , 2 , and 3 can be expressed as:
1 = tan 1
t gz ; t gx

3 = cos 1 tan(1 )

a cos( 3 ) 2 = tan 1 4 a3 t gy

a 3t gz a3t gx a 4 t gx a 4 t gz

(5.3)

94

Substituting the numerical values of the D-H parameters given in table.5.3 into the set of equation(5.3), the following values for shoulder joint are obtained: 1=0.7854 rad , 2=-0.6155 rad , 3=0.4243 rad To find the value of the wrist joint angles , with Rw as in equation(4.7) the following is obtained:
c5 s 6 c7 s5 s 7 Rw = c6 c7 s 5 s 6 c 7 c5 s 7 c5 s 6 s 7 s 5 c 7 c6 s 7 s5 s 6 s 7 c5 c7 c5 c 6 s6 s5 c6

(5.4)

Applying the other form of equation(4.7), Rw may be calculated as:

(R1 R2 R3 R4 )

0.5785 0.0015 0.8157 R g = 0.5661 0.7192 0.4028 0.5873 0.6947 0.4152

(5.5)

Consider equations(5.4) and (5.5) for computing matrix Rw , the wrist joint angles are calculated by comparing both sides, that is:
c5 s 6 c 7 s 5 s 7 Rw = c6 c 7 s 5 s 6 c 7 c5 s 7 c5 s 6 s 7 s 5 c 7 c6 s 7 s 5 s 6 s 7 c5 c 7 c5 c6 0.5785 0.0015 0.8157 s6 = 0.5661 0.7192 0.4028 s5 c6 0.5873 0.6947 0.4152

5 = tan 1

Rw (3,3) ; 5 = 0.4708rad . Rw (1,3) Rw (2,2) , 7 = 0.4146rad . Rw (2,1)

7 = tan 1 6 = tan 1

Rw (2,3) , 6 = 0.904rad . Rw (3,3) / sin( 5 )

In applying equation(4.10) the forward kinematic may be calculated using the above calculated angles ( 1,, 7 ) as follows:

95

G forward

0.7071 0.7071 29.712 0 0 0.7071 0.7071 13.452 = A1 A2 A3 A4 A5 A6 A7 = 1 0 0 23.094 0 0 1 0

(5.6)

The errors in position of end-effector are calculated by comparing equations 5.1 and 5.6, while the orientation values are similar. These errors are:
e t 2.4 10 3 gx e t gy = 2 10 4 4 e t gz 4 10

(5.7)

From the values of error, it can be seen that the accuracy of calculation of IK is acceptable as in equation(5.7). Human learning sense test is considered through many repetition of the above calculation method. When a human tries to touch a known position and his neural system is familiar with this position; the same results will be obtained with each trial. The result is shown in Fig 5.25.
p o sitio n (cm ) in x -ax is 40 30 20 0 40 20 0 0 40 30 20 0 0.5 1 1.5 2 2.5 3 No. of return run. 3.5 4 4.5 5 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 position of E.F o/p model initail position

p o sitio n (cm ) in z-ax is

p o sitio n (cm ) in y -ax is

Fig 5.25 Simulation for IKP by proposed algorithm 96

If the position of end-effector changes without changing the D-H parameters and initial condition, the result is shown in Fig.5.26. From this figure it can be seen that the position error will decrease with calculation repetition similar to real human sense. The envelope of movements in this model, related to human arm real constrain is calculated with the following condition:
2 L 2 + L 2 Lt g a b

2 L a Lb
o/p model initail position of model

(5.8)

position of E.F

40 position (cm) in x-axis 35 30 0 40 position (cm) in y-axis 20 0 0 40 position (cm) in z-axis 30 20 10 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

0.5

1.5

2.5

3.5

4.5

0.5

1.5

2.5

3.5

4.5

No. of return run.

Fig 5.26 Simulation with different values of position of end-effector The GUI that simulate the kinematics of this design is shown in Fig 5.27. The kinematic system design is shown with two forms. The forward kinematics and the inverse kinematics are presented by simulink under MATLAB. The use of simulink is to connect the simulation with VR design. When the first button is selected, forward kinematics human arm model will only run the forward part while when the second button is 97

selected forward kinematics human arm model will run all parts of the model shown in Fig 5.28.

Fig 5.27 GUI system for kinematics and VR for Human Arm
Given endeffector position Computing endeffector position

Fig 5.28 Simulink Flowchart of Forward and Inverse Kinematics 98

5.6 Human Arm Movements Implementation in VR The VR technique in this work is used as an environment similar to real or practical environment. In this design there are three choices to implement the movements in the VR which are explained next. 5.6.1 Virtual Human Body Model Demo The design of VR models for human body is explained in chapter four. The simulation of human body is achieved by connecting the VR with simulink/ MATLAB, where the commands for movement will be calculated in simulink and then given to VR model . The command is calculated by using the real constrain in human arm joints, and every movement is checked separately. Then all these data are collected by lookup table in MATLAB workspace. The simulation of movement will take the data from lookup table by sequences to implement all real movements for the two human arms (right and left). The movements implemented in this design represent the main movements (14) in human arm such as abduction-adduction , flexionextension , and internal-external rotation of the upper arm. The flexionextension , and pronation - supination of the forearm, the ulna-radial deviation ( abduction-adduction of wrist ) , and flexion-extension of the hand. Fig.5.29, present the simulink design for the system. The output is shown in Fig.5.30. The details of Fig.5.29 and the residue of the results are shown in Appendix B.

99

Fig 5.29 Virtual Human Body Model driven by Simulink

0.1 sec

0.2 sec

0.3 sec

0.4 sec

0.5 sec

0.6 sec

0.7 sec

0.8 sec

0.9 sec

0.10 sec

0.11 sec

0.12 sec

Fig 5.30 Simulated Results for Human Body Model 100

5.6.2 Virtual Kinematic Human Arm The skeleton of a human arm in VR was shown in details in section (4.7.1). This design will now be simulated and the instruction of movements achieved by applying a proposed algorithm will be as explained previously, see Fig 5.31

Fig 5.31 Virtual Human Arm driven by a proposed kinematics algorithm 101

The example for simulation results of this model is shown in Fig.5.32. From these results, it can be seen that the solution for IKP by analytical solution has many advantages such as the ability to implement the system in VR and then can be done in practice with artificial limps. This movement is implemented according to all real constrains of real movement of human arm ( the value of joints angle and the real envelop of movements ).

0.1 sec

0.2 sec

0.3 sec

0.4 sec

0.5 sec

0.6 sec

0.7 sec

0.8 sec

0.9 sec

0.10 sec

0.11 sec

0.12 sec

0.13 sec

0.14 sec

0.15 sec

0.16 sec

0.17 sec

0.18 sec

0.19 sec

0.20 sec

Fig 5.32 Simulation Virtual Skeleton Human Arm 102

5.6.3 Virtual Human Arm driven by EMG Signal The GUI for simulation of this design is shown in Fig 5.33. The implementation of movement based EMG signal starts with the preprocessing on the EMG signal and NN identification for seven EMG signals of seven different muscles, then classifies the signal, and finally recognizes the movements. The overall simulation was explained in this chapter, and it can be summarized in the following steps: Step one: receives the signal from any sources as shown in Fig 5.4. Step two: applies the procedure shown in Figs(5.45.13) preprocessing the data and discussing the other types of signal. Step three: applies the procedure for identification of network and signal classification, (see Figs(5.145.23)).

Fig 5.33 GUI for Simulation of Recognition of Movements of Human Arm Now, the final step is the recognition of the human arm movements which is produced by contraction of the seven muscles. When applying the first step in Fig.5.33, the recognized signal result is obtained according to a certain movement and the data for movement or the EMG 103

signal for other muscle in human arm is prepared. The system will show message the signal is not arm defined muscle. These two results are explained in GUI shown in Fig.5.34(a&b).

-a-

-b-

Fig 5.34 Simulation Results for step one of movements recognition If the result of step one is the same as in the Fig.5.34b, it can not be applied in the next step, and another signal must be tested. The next step is to simulate the instruction to execute virtual human arm by simulink program, and recall the data obtained in first step from workspace and then enter to the system shown in Fig.5.29. The period for running the system is two seconds to execute the order. To explain the operation of the design the following case study will be considered. Case Study: The case under consideration is dealt with by using generated EMG signal by the simulation of deltoid muscle, which controls the abduction

104

of human arm at the shoulder. After running the EMG simulator with specification of deltoid muscle we will get the first signal shown in Fig 5.18. The data for this signal are stored with the name ( deltoid_test.mat ) in workspace. This signal will be used in the complete example. The steps of simulation for this design are presented in details in Fig.5.35(ah). From the above figure, it can be seen that the use of NN recognition based on normalized amplitude EMG signal after classifying the signal as EMG signal by using FRBP-NN with samples of EMG about (1600) samples captured from the monitoring of EMG signal is a very powerful method for moving virtual human arm. From these results it can be said that using analytical solution has a suitable accuracy in presenting the real human arm movements.

click click

-a-

-b-

105

Fig 5.37(Continued)

click

-c-

-d-

Double click

click

-e-

-f-

Fig 5.35(af) Procedure for running the simulation of Virtual Human Arm Driven by EMG Signal

106

Chapter Six Conclusions and Suggestions for Future Work

Chapter Six Conclusions and Suggestions for Future work

6.1 Conclusions From the obtained results it can be concluded that: 1. The analytical solution of IKP is an effective method to simulate human arm and it has the ability for adding any real constrain to the model without any complexity. The proposed algorithm has the ability to solve IKP for anthropomorphic limbs by analytical solution with high accuracy when compared with those available in the literature. 2. Using an EMG simulator overcomes the problems of hardware unavailability in the real EMG measuring system, where the local real measuring system has only a single channel with needle electrode. The use of EMGlab gives the facility of obtaining an EMG signal from the given data either from real measuring system or an EMG simulator. 3. High accuracy NN classifier is used to distinguish between the human arm muscle EMG signals. Human arm movements recognition system based on EMG signals using FRBP-NN is a good method from the point of view of its simplicity and accuracy and it can be implemented in many medical applications such as the use of artificial limbs and high accuracy robot applications. 4. The use of mixed reality human ( MRH ) adds more realities to the VR model to approximate the real sense of human arm movement similar to that of natural human performance.

107

5. The connection between MATLAB / simulink and VR model gives the design a facility to calculate the human arm movements by many methods of simulation through providing a suitable command to VR model. The relation between the MATLAB and VR offers a VR model to execute all movements run directly from VR through a generation of virtual model animation. 6. Design of the proposed package based GUI technique gives the ability to connect all three parts under consideration, simulate individual parts by transportation between them, and finally presents the three parts simultaneously as one animation system. Moreover, many helps are included in this package with explanations by notes, graphs and movies enabling this package to be used as an education tool to teach human arm kinematic, its EMG signal and VR animation.

6.2 Suggestion For Future Work 1. Applying an optimal control approach in calculation of human arm posture based EMG signal. 2. Using of augmented reality ( AR ) technique to have more reality for virtual human arm model. 3. Implementation of the proposed package as assistance with manufacturing of artificial limbs.

108

References

References: M. W. Spong, S. Hutchinson, and M. Vidyasagar, Robot Modeling and Control, John Wiley and Sons, INC.2006. [2] A. Benitez, G. delos Santos, D. Vallejo, Forward Kinematics for Virtual Agents. Engineering Letters, 15:2, EL, 15-2, 2008. [3] S. Drzevitzky, Symbolic Solutions for Inverse Kinematics Problems for Anthropomorphic Limbs, Diplomarbeit Thesis, Germany, March, 2008. [4] A. K. G. Murphy, Effective Information Display and Interface Design for Decomposition-based Quantitative Electromyography , M.Sc thesis, University of Waterloo, Canada, 2002. [5] C. Joslin, I. S. Pandzic, N. Magnenat, Trends in Networked Collaborative Virtual Environments, MIRALab- University of Geneva 24 Rue Du General-Dufour, CH1211, Geneva-4, 2001. www.miralab.unige.ch/repository/papers/100.pdf, last my visit to web in 5/9/2009. [6] P. L. Gribble, L. I. Mullin, N. Cothros, Role of Cocontraction in Arm Movement Accuracy, J Neurophysiol 89, 23962405, January 22, 2003. [7] N. BU, EMG-Based Motion Discrimination Using a Novel Recurrent Neural Network , Journal of Intelligent Information Systems, 21:2, 113126, 2003. [8] O. Bida, Influence of Electromyogram (EMG) Amplitude Processing in EMG-Torque Estimation , M.Sc Thesis , WORCESTER POLYTECHNIC INSTITUTE ,Electrical Engineering , January 2005. [9] A. Hamilton-Wright, D. W. Stashuk, Physiologically Based Simulation of Clinical EMG Signals , IEEE Transaction on Biomedical Engineering, VOL. 52, NO. 2, PP 171-185, February 2005. [10] M. B. I. Reaz, M. S. Hussain, F. Mohd-Yasin, Techniques of EMG signal analysis: detection, processing, classification and applications , Biol. Proced. Online 2006; 8(1): 11-35. doi:10.1251/ bpo115 March 23, 2006. [11] Y. Hou, J. M. Zurada, Estimation of the Dynamic Spinal Forces Using a Recurrent Fuzzy Neural Network , IEEE Transaction on Systems, man, and Cybernetics-Part B: Cybernetics, Vol. 37, NO. 1,PP 100-109, February 2007. [12] C. Cipriani, F. Zaccone, S. Micera, On the Shared Control of an EMG-Controlled Prosthetic Hand: Analysis of UserProsthesis Interaction , IEEE Transaction on Robotic, Vol. 24, NO. 1, PP 170-184, February 2008 110 [1]

[13] N. A. Shrirao, N. arender, P, Reddy, Neural network committees for finger joint a ngle estimation from surface EMG signals, BioMedica l Engineeri ng OnLine 8 :2, 2009. [14] P. Baerlocher, Inverse Kinematics Techniques of the Interactive Posture Control of Articulated Figures ,Ph.D. thesis, Ecole Polytechnique Federale de Lausanne, 2001. [15] X. Wu, L. Ma, Z. Chen, Y. Gao, A 12-DOF Analytic Inverse Kinematics Solver for Human Motion Control , Journal of Information & Computational Science, PP 137-141, 30-August 2004. [16] M. Mihelj, Human Arm Kinematics for Robot Based Rehabilitation , Robotica, Volume 24, Issue 03, pp 377-383 , May 2006. [17] M. Kallmann, Analytical inverse kinematics with body posture control , Computer Animation and Virtual Worlds (in press) Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/cav.176, April 2007. [18] R. R. Powell, Future Cyborgs: Human-Machine Interface for Virtual Reality Application, Air Command and Staff. A Research Report, April 2007. [19] Y. Tsepkovskiy, L. Antonov, Cv. Kocev, F. Palis, N. Shoylev, Development of a 3D and VRML Virtual Hand Models for Different Mechanical Gripper , Journal of the University of Chemical Technology and Metallurgy, 43, PP 159-164, 2008. [20] B. Whitworth, The Physical World as a Virtual Reality , Massey University, Albany, Auckland, New Zealand E-mail: bwhitworth@acm.org . [21] A. Kotranza, B. Lok, A. Deladisma, C.M.Pugh and D.S.Lind, Mixed R eality Humans: Evaluating Behavior, Usability, and Acceptability , IEEE Transaction on Visualization and Computer Graphics, Vol. 15, NO. 3, PP 369-382, May/June 2009. [22] P. Beylot , P. Gingins , P. Kalra , N. M. Thalmann , W. Maurel , D. Thalmann 3D Interactive Topological Modeling using Visible Human Dataset, MIRALab Copyright Information , MIRALab, University of Geneva, Computer Graphics Lab. 1998. [23] T. S. Buchanan, D. G. Lioyd, Muscle Activity is different for Humans Performing Static Tasks which Require Force and Position Control, Neuroscience Letters 194, pp 61-64, 1995. [24] C. O. Keefe, Muscle Activation Patterns in the Trans Tibial Amputee, monash rehabilitation technology research unit, project 1998.

111

[25] Software Package, 3D Virtual Human Antomay Studio ( 3D VHAS), Ver.0.9.1. , 2009. [26] T. Tsuji, O. Fukuda, M. Kaneko, Pattern Classification of TimeSeries EMG Signals Using Neural Networks , Int. J. Adapt. Control and Signal Processing, PP 829-848, 2000. [27] A. Aubel, Anatomically-Based Human Body Deformations, Ph.D. thesis, Ecole Polytechnique Federale de Lausanne, 2002. [28] M. Mihelj, Inverse Kinematics of Human Arm Based on Multisensor Data Integration, J Intell Robot System , 47, pp139 153, 2006. [29] W. Maurel, P. Beylot, D. Thalmann, P. Gingins, P. Kalra, P. Hoffmeyer, N. M. Thalmann, A Biomechanical Musculoskeletal Model of Human Upper Limb for Dynamic Simulation, Computer Graphics Laboratory, Swiss Federal Institute of Technology, DILIG, EPFL, CH-1015 Lausanne, Switzerland. http://ligwww.epfl.ch/~maurel/Publish/EWCAS96/EWCAS96.pdf last my visit to web in 5/9/2009. [30] A.N. Bhat, H.M. Lee, J.C. Galloway, Toy-oriented changes in early arm movements IIJoint kinematics , Infant Behavior & Development, 30, PP 307324, 2007. [31] N. I . Badler, D. Tolani, Real-Time Inverse Kinematics of the Human Arm , Copyright Massachusetts Institute of Technology. Reprinted from Presence, Volume Issue 4, pages 393 -401. University of Pennsylvania, 1996. [32] A. M. Mughal, A Theoretical Framework for Modeling and Simulation with Optimal Control System of Voluntary Biomechanical Movements ,PhD Candidate Dept. of Applied Science, UALR , September 2006. [33] K. Farrell, Kinematic Human Modeling and Simulation Using Optimization-Baed Posture Prediction., M.Sc Thesis, Mechanical Engineering, University of Iowa, December 2005. [34] B. Koyuncu, M. Gzel, Software Development for the Kinematic Analysis of a Lynx 6 Robot Arm , International Journal of Applied Science, Engineering and Technology 4;4, PP 228-233 www.waset.org Fall 2008. [35] R. Merletti, P. Parker, Electromygraphy: Physiology, Engineering and Noninvasive Applications , ISBN 0-47-67580-6, copyright , PP 1-29, 2004. [36] M. B. Reaz, M. S. Hussain, Techniques of EMG Signal Analysis: Detection, Processing, Classification and Applications , Biological Procedures Online; 8(1): 11-35. doi:10.1251/bpo115 March 23, 2006.

112

[37] H. Reucher, G. Rau, J. Silny, Spatail Filtering of Noninvasive Multielectrode EMG:n Part I-Introduction to Measuring Technique and Applications , IEEE Transaction on Biomedical Engineering, Vol. 34, no. 2, PP 98-105, February 1987. [38] K. Manal, T. S. Buchanan, A one parameter neural activation to muscle activation model: estimating isometric joint moments from electromyograms, Journal of Biomechanics 36 , PP 11971202, 2003. [39] V. R . Mankar, A. A. Ghatol, Design of Adaptive Filter Using Jordan/Elman Neural Network in a Typical EMG Signal Noise Removal ,Hindawi Publishing Corporation Advances in Articial Neural Systems Volume 2009, Article ID 942697, 9 pages doi:10.1155, 2009. [40] O. A. Alsayegh, EMG Based Human Machine Interface System , IEEE Transaction of Biomedical Engineering, 0-7803-6535-4/00/ pp 925-928, 2000. [41] M. M. Lowery, N. S. Taflove, A Multiple Layer Finite-Element Model of the Surface EMG Signal , IEEE Transaction of Biomedical Engineering, Vol. 49, no.5, PP 446-454, May 2002. [42] L. Mesin, D. Farina, Simulation of Surface EMG Signals Generated by Muscle Tissues with Inhomogeneity Due to Fiber Pinnation , IEEE Transaction on Biomedical Engineering Vol. 51, no. 9, PP 1521-1529, September 2004. [43] T. Mazuryk, M. Gervautz, Virtual Reality: History, Application, Technology and Future , Institute of Computer Graphics, Vienna University of Technology, Austria, 1997. [44] C. A. Wingrave, D. A. Bowman, Personalized Nuance-Oriented Interaction in Virtual Environments , VCirginia Tech, 660 McBrude Hall 0106 Blacksburg, VA 24061, 2002. [45] D. Finkenzeller, M. Baas, S. Thuring, VISUM: A VR System for the Interactive and Dynamics Simulation of Mehatronic Systems , Virtual Concept, Biarrtz-France, November 5-7, 2003. [46] Virtual Reality Toolbox 4 Users Guide, The MathWorks, at www.mathworks.com, July 2008. [47] R. Carey, G. Bell and C. Marrin, ISO/IEC 14772-1:1997 Virtual Reality Modeling Language (VRML97), At http://www.vrml.org/Specifications/VRML97, August 2008. [48] R. W. Lindeman, J. L. Sibert, Towards Usable VR: An Empirical Study of User for Immersive Virtual Environments , Institute for Computer Graphics, The George Washington University, DC, 20052, PP 64-71, 1999. [49] Y. Guo,Q. Du, Yi Luo, W. Zhang, Lu Xu, Application of Augmented Reality Gis in Architecture , The International 113

[50]

[51]

[52] [53]

[54] [55]

[56] [57]

Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B5. PP 331-336, Beijing 2008. K. Abdel-Malek, W. Yu, Human Placement for Maximum Dexterity, Department of Mechanical Engineering The University of Iowa City, last my visit to web in 15/8/2009.: http://www.engineering.uiowa.edu/~amalek/papers/Humandexterit ySAE.pdf. Motion Lab Systems, Inc, A software user guide for EMG Graphing and EMG Analysis EMG Analysis , Updated Thursday, February 26, 2009. A. Hamilton, D. W. Stashuk, Production of Simulated EMG Using a Physiological Model , Simulator: Howto 2, June, 2007. K. C. McGill, Z. C. Lateva, H. R. Marateb, EMGLAB: An interactive EMG decomposition program , Journal of Neuroscience Methods, NSM-3970, PP 13, 2005. EMGLAB software Version 0.9 Users Guide, The MathWorks, at www.mathworks.com, May 2008. Y. I. Al-Mashhadany, Locally Recurrent Neural Network (LRNNs) Controller for Dynamical System , First Information Technology Conference, University of Technology, ITAHOI,552, Baghdad, 21-23 April 2009. www.3dcadbrowser.com, download by subscription in this Web. Private Consultation in Baghdad Medical City. -----------------------------------------------------------

Published papers: [*] M. Z. Al-Faiz, Y. I. Al-Mashhadany, Analytical Solution for Anthropomorphic Limbs Model (IK of Human Arm), IEEE Symposium on Industrial Electronics and Applications (ISIEA 2009), October 2009, Kuala Lumpur, Malaysia.

[**]M. Z. Al-Faiz, Y. I. Al-Mashhadany, Human Arm Movements Recognition Based on EMG Signal MASAUM Journal Of Basic and Applied Sciences (MJBAS) Volume 1 Issue 2, PP 164-171, (September 2009)

114

Appendices

Appendix A: Details of Classical Analytical Solution[3]. The final form to get elbow joint angle (4) can be obtained from the solution of equation(2.12) shown below :
c 4 (2a1c1 a 2 + 2d1c 1 d 2 ) + s 4 (2a1c 1c 1 d 2 + 2d1 a1 )
2 2 2 2 = (tg x + tg y + tg z2 ) (a12 + d12 ) (a 2 + d 2 ) (2 s1 a1 s 1 d 2 ).

(A.1)

From the full solution of above equations we get the finial two forms of 4 (see ref 3):The first form:

4 = 2 arctan( 0 .5
2 4 cos( 2 ) d 2 a1 cos( 1 ) + 4 a 2 d 1 + 2 ( 4tg x a1 sin( 1 ) sin( 2 ) d 2 a14 d 14 4 4 4 4 2 2 tg x a 2 tg z d 2 4 a 2 a1 sin( 1 ) sin( 2 ) d 2 + 4tg z a1 sin( 1 ) sin( 2 ) d 2 4 d 2 a sin( ) sin( ) d + 2 a 2 d 2 + 2 a 2 tg 2 + 2 a 2 tg 2 + 2 a 2 tg 2 + 2 a 2 tg 2 1 2 2 2 1 1 1 1 1 x y x z 2 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 a d 2 a a 2 a d + 2 d tg + 2 d tg + 2 d tg 2 d d 2tg tg 1 2 1 1 1 1 1 1 2 x y z y x 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2tg y tg z + 2tg y d 2 + 2tg y a 2 2tg x tg z + 2tg x d 2 + 2tg x a 2 + 2 a 2 tg z 2 2 2 2 2 2 2 2 2 2 4 2 a 2 d 2 + 2tg z d 2 + 4 (cos( 2 )) d 2 d 1 + 4 a 2 a1 (cos( 1 )) tg y + 3 4 (cos( 2 )) 2 d 22 a12 (cos( 1 )) 2 4 a13 sin( 1 ) sin( 2 ) d 2 4 d 2 a1 sin( 1 ) sin( 2 ) 2 1 2 4 a1 (sin( 1 )) 2 (sin( 2 )) 2 d 22 + 4tg y a1 sin( 1 ) sin( 2 ) d 2 ) 2 ) 2 2 2 2 2 2 2 a1 + d 1 tg y tg x + a 2 tg z + d 2 2 cos( 2 ) d 2 d 1 + 2 a1 sin( 1 ) sin( 2 ) d 2 2 a 2 a1 cos( 1 )

(A.2)

Or the second form:

A-1

 4 = 2 arctan( 0.5
2 4 cos( 2 ) d 2 a 1 cos( 1 ) + 4 a 2 d 1 2( 4tg x a 1 sin( 1 ) sin( 2 ) d 2 a 14 d 14 4 4 4 4 2 2 tg x a 2 tg z d 2 4 a 2 a 1 sin( 1 ) sin( 2 ) d 2 + 4tg z a 1 sin( 1 ) sin( 2 ) d 2 4 d 2 a sin( ) sin( ) d + 2 a 2 d 2 + 2 a 2 tg 2 + 2 a 2 tg 2 + 2 a 2 tg 2 + 2 a 2 tg 2 x y x z 1 1 1 2 2 2 1 1 1 1 1 2 a 2 d 2 2 a 2 a 2 2 a 2 d 2 + 2 d 2 tg 2 + 2 d 2 tg 2 + 2 d 2 tg 2 2 d 2 d 2 2tg 2 tg 2 x y z y x 1 2 1 1 1 1 1 1 2 1 2 2 2 2 2 2 2 2 2 2 2tg y tg z2 + 2tg y d 22 + 2tg y a 2 2tg x tg z2 + 2tg x d 22 + 2tg x a 2 + 2 a 2 tg z2 2 2 2 2 2 2 2 2 2 2 4 2 a 2 d 2 + 2tg z d 2 + 4(cos( 2 )) d 2 d 1 + 4 a 2 a 1 (cos( 1 )) tg y + 2 2 2 2 3 3 4(cos( 2 )) d 2 a 1 (cos( 1 )) 4 a 1 sin( 1 ) sin( 2 ) d 2 4 d 2 a 1 sin( 1 ) sin( 2 ) 2 1 2 2 2 2 4 a 1 (sin( 1 )) (sin( 2 )) d 2 + 4tg y a 1 sin( 1 ) sin( 2 ) d 2 ) 2 ) 2 2 2 a 12 + d 12 tg y tg x + a 2 tg z2 + d 22 2 cos( 2 ) d 2 d 1 + 2 a 1 sin( 1 ) sin( 2 ) d 2 2 a 2 a 1 cos( 1 )

(A.3) The final form of swivel angle and the elbow position e( ) as follow:

A-2

 = arctan 2 (
(tg x e y a x tg z e x a y e z tg y a x e y tg x a z + tg y e x a z + e z tg x a y )tg x (tg x e y a x tg z e x a y e z tg y a x e y tg x a z + tg y e x a z + e z tg x a y )tg x (tg x e y a x tg z e x a y e z tg y a x e y tg x a z + tg e a + e tg a )tg z x y x y x z tg z2 e x a x + tg z2 e y a y tg e a tg tg e a tg z z x x z z y y tg z e x a x tg z tg z e y a y tg z 2 tg y e y a x tg x + tg x e z a z 2 tg x e x a y tg y + tg y e z a z + 2 2 e y tg x a y + e x tg y a x
2 1 2 2 2 2 a x tg y a x tg z + tg x a y tg y + tg x a y tg z + 2 2 a y tg x a y tg z2 tg y a x tg x tg y a z tg z + + 2 a tg 2 a tg 2 tg a tg tg a tg z x z y y x x y z z 2 1 2 2 2 a x tg y a x tg z2 + tg x a y tg y + tg x a y tg z + 2 2 2 a y tg x a y tg z tg y a x tg x tg y a z tg z + + , 2 2 2 a tg a tg tg a tg tg a tg z x z y y x x y z z 2 1 2 2 2 2 a x tg y a x tg z + tg x a y tg y + tg x a y tg z + 2 2 a y tg x a y tg z2 tg y a x tg x tg y a z tg z + a tg 2 a tg 2 tg a tg tg a tg 2 z x z y y x x y z z 1 2 2 2 2 a x tg y a x tg z + tg x a y tg y + tg x a z tg z ( 2 2 2 + a y tg x + a y tg z tg y a x tg x tg y a z tg z ) + a tg 2 + a tg 2 tg a tg tg a tg 2 z x z y z x x z y y

A.4 )

A-3

The form of elbow position can be get by the following equations:

2 2 = tgx + tg y + tgz2

( + a =
=1
2

tg, a = axtgx + aytg y + aztgz

2 1 2 2 + d12 a2 d2 2 2 1

( + a

2 2 2 2 + d12 a2 d2 + a2 + d2 a12 d12

)(

(A.5)

2 v1 = ax + axtgx + tgx aytg y + tgx aztgz 2 v2 = ay + tg y axtgx + aytg y + tg y aztgz

2 2 v3 = aztgx aztg y aztgz2 + tgz axtgx + tgz aytg y + aztgz2

v=

4 4 2 2 2 2 2 tgx + tg y + tgz4 + 2(tgx tg y + tgx tgz + tg ytgz2 )

v1 + v2 + v3

(A.6)

tg, a tg x + a x tg x v(cos( ) + sin( )) , tg y + a y tg a tg y v(cos( ) + sin( )) e( ) = tg + a tg, a tg v(cos( ) + sin( )) z z z

(A.7)

Solving for Shoulder Rotation Matrix R1: Two coordinate systems are set up: one for the current arm position and one for the desired arm position. The idea is to treat the shoulder rotation as the rotation from the plane containing the current triangle .s eold wold to the plane containing the new triangle .s e new w new . For our purposes it is easier to consider the frames defined by
A-4

these triangles and compute the rotation in between these two frames. This rotation is then the rotation performed by R1 . The rotation of the shoulder R1 is the change of bases. This change can be computed by placing the vectors of the bases in homogeneous transformation matrices at the position of the rotation matrices. The shoulder rotation is the rotation from the first frame to the second frame. This is equivalent to the concatenation of the rotation from the first frame to world frame and the rotation from the world frame to the second frame.
x goal T1 = 0 0 0 x y z 0 1 0 0 0 0 0 0 1

y goal 0

z goal 0

(A.8)

We are able to extract the combined rotations of the shoulder in form of the 3 3 matrix R1(1,2,3). To further extract the values for the angle joints 1,2, and 3, the combined rotations need to be decomposed, Rs can be defined as the concatenation of three rotations in a specific order. Therefore, these matrices take the following form:
0 ci + 2 si + 2 0 ci +1 0 si +1 1 0 s . 0 0 c s 1 0 = i + 2 c i + 2 0 i i 0 0 1 si +1 0 ci +1 0 s i ci ci + 2 ci +1 si + 2 ci + ci + 2 si +1 si si + 2 si + ci + 2 si +1ci = s i + 2 ci +1 ci + 2 ci + si + 2 si +1 si ci + 2 s i + si + 2 si +1ci s i +1 ci +1 si ci +1ci

Rrot

(A.9)

The final expression of analytical solution of shoulder joint angles are:

A-5

1 = arctan 2(c 2 s1 , c 2 c1 ), 2 = arctan 2( s 2 , c 2 s1 / sin(1 )), 3 = arctan 2( s3 c 2 , c3 c 2 ),

(A.10)

Solving for Wrist Rotation Matrix R2: Having set up analytical expressions for 1,.,4 and , it is finally possible to compute R2. Recall that the orientation and position, i.e. the constraints, of the endeffector are given by G from equation (2.13). This equation can thus be solved for T2:
T2 = (T1 AsTy Ae ) 1 G RRR R R1 Rs Ry t e + R1t s 1 s y e = 0 0 1 0
1

R g 0 0

t g 0 1

(A.11)

From the definitions of the respective matrices, the inverse of a homogeneous transformation matrix can in general be expressed analytically as:
R t 0 1
1

RT = 0

RT t 1
R R R R t R R R R t 1 0 0 R ( , , ) 0 0 2 5 6 7 0 0 1 0 0 0 1
T e T y T s T 1 g T 1 T s T y T e e

( A.12)

RT RT RT RT R e y s 1 g T2 = 0 0 0 RT RT RT RT R e y s 1 g = 0 0 0

(A.13)

Because T2 only has a rotational part, we can reduce equation (2.24) and express R2 as:

A-6

R2 = ( R1 Rs R y Re ) 1 R g = ReT R T RsT R1T R g y

(A.14)

Since the shoulder rotation matrix R2 is the same concatenation of rotation matrices as R1, except of course that R2 is a function of 5,6, and 7, it is possible to derive the values for these angles as done for the shoulder joints 1,2, and 3, to get:
5 = arctan 2(c6 s5 , c6 c5 ), 6 = arctan 2( s 6 , c6 s5 / sin( 5 )), 7 = arctan 2( s 7 c 6 , c7 c6 ),

(A.15)

Last but not least, analytical representations for the joint angles 5,6, and 7. All representations from this section can now be merged into a single solution for the Inverse Kinematics problem of the given chain defined through end-effector position tg and orientation Rg as well as the swivel angle .

A-7

Appendix B: Some Output of System Design: The output of atypical help of the developed package built to simulate the design is shown in Fig.B.1.

Fig.B.1. Help Output for Muscles Demo.

B-1

Fig.B.2. Help Output for Skeleton Human Arm.

Fig.B.3. Help Output for Neuron Instruction.

B-2

Fig.B.4. Output of Virtual Human Arm Model

B-3

-a-

-b-

B-4

Variable name

Data

x-axis in shoulder ( Xs)

x-axis in shoulder ( Xs)

x-axis in shoulder ( Xs)

Data

x-axis in elbow ( Xs)

x-axis in elbow ( Xs)

x-axis in elbow ( Xs)

Data

xaxis in wrist ( Xs) 0 0 0 0 0 0

xaxis in wrist ( Xs) 0 0 0 0 0 0

xaxis in wrist ( Xs) 0 0 0 0 0 0

Type of movement

y1 y2 y3 y4 Y5 Y6 -

sine sine sine 0 0 0

0 +1 -1 0 0 0

0 0 0 0 0 0

+1 0 0 0 0 0

0 0 0 sine sine sine

0 0 0 +1 0 0

0 0 0 0 1 -1

0 0 0 0 0 0

0 0 0 0 0 0

Abduction of arm at shoulder Flexion of arm at shoulder Extension of arm at shoulder Flexion and Extension forearm at elbow Proration forearm at elbow Supination forearm at elbow

-c-

Fig.B.5. Details of Simulink Human Body Demo Instruction. a. Shoulder joint Instruction. b. Wrist Joint Instruction. c. Elbow joint Instruction.

B-5

Fig.B.6. Output of Virtual Human Body Demo Model

B-6

Appendix C: Sample Programs in Matlab Editor Ver.2009a:

% %%%%%%%%%%%%%%%%%%% program No.one %%%%%%%%%%%%%%%%%%%%%%%% %% This program used to solve Two main kinematic taskes: %%%% %%%%% 1. slove inverse dynamic for real human arm and %%%%%% %%%%% the solution according to the classical algorithm %%%%% %%%% this algorithm define by Stephanie Drzevitzky 2008 %%%%% %%%% 2. solve the forward kinematic to check the values %%%%% %%%% of joint angles which are get by inverse kinematic %%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
clc clear %%%%%%%%%%%%%%%% INVERSE KINEMATIC ALGORITHM %%%%%%%%%%%%%%%%%%%% % Define Limb via matrix A & B with Denavit-Hartenberg parameters: % Define the End-Effector position with vector tg: %----------------------------------------------------------------a1=20;%input('enter the parameter a1 D_H='); a2=20;%input('enter the parameter a2 D_H='); d1=20;%input('enter the parameter d1 D_H='); d2=20;%input('enter the parameter d2 D_H='); th1t=pi/4;%input('enter the parameter theta1 D_H='); th2t=pi/4;%input('enter the parameter theta2 D_H='); aL1=pi/4;%input('enter the parameter alpha1 D_H='); aL2=pi/4;%input('enter the parameter alpha2 D_H='); tgx=input('enter the end affecter in x axis tgx='); tgy=input('enter the end affecter in y axis tgy='); tgz=input('enter the end affecter in z axis tgz='); %-----------------------------------------------------% Define all initail values: ax=5;ay=65;az=65; a=[ax ay az];tho5=pi/6;tho6=pi/4;tho7=pi/6; c1t=cos(th1t);c2t=cos(th2t);caL1=cos(aL1);caL2=cos(aL2); s1t=sin(th1t);s2t=sin(th2t);saL1=sin(aL1);saL2=sin(aL2); ex=20;ey=-4.5;ez=20;tgxo=40;tgyo=40;tgzo=40;exo=30;eyo=30;ezo=30; tg=[tgx tgy tgz]; Ltg=norm(tg,2); eold=[exo eyo ezo]; %******************************************************* % Define the rotation of end-effector: thgx=pi;%input('enter the angle for rotate in of the end-affector x-axis ='); thgy=pi/2;%input('enter the angle for rotate in of the end-affector y-axis ='); thgz=pi/4;%input('enter the angle for rotate of the end-affector in z-axis =');

C-1

sx=sin(thgx);cx=cos(thgx); sy=sin(thgy);cy=cos(thgy);sz=sin(thgz);cz=cos(thgz); Rgch=[ cz*cy sz*cy -sy -sz*cx+cz*sy*sx cz*cx+sz*sy*sx cy*sx sz*sx+cz*sy*cx -cz*sx+sz*sy*cx cy*cx ];

tg=[tgx;tgy;tgz];

Gc=[Rgch,tg]; Gc(4,4)=1; Gch=Gc; %******************************************************* % calculated for matrix A : % A transformation from 0th to 1st frame : % rotation matrix around x-axis by angle alpha_o: Rotx_aL=[ 1 0 0 0 ; 0 caL1 -saL1 0 ; 0 saL1 caL1 0 ; 0 % transation in x-axis by a1 Transx_a=[ 1 0 0 a1 ; 0 1 0 0 ; 0 0 1

1 ];

0 ; 0 0 0 1 ];

% rotation matrix around z-axis by angle theta1: Rotz_th=[ c1t -s1t 0 0 ; s1t c1t 0 0; 0 0 1 0 % transation in z-axis by d1 Transz_d=[ 1 0 0 0 ; 0 1 0 0 ; 0 0 1 d1 ; 0 0 0 1 ];

; 0 0 0 1 ];

% transformation matrix from 0th frame to 1st frame is ( A matrix ): A=Rotx_aL*Transx_a*Rotz_th*Transz_d Ra=A(1:3,1:3); ta=A(1:3,4); Lta=norm(ta,2); Lu=Lta; %pause; % calculated for matrix B: % A transformation from 1st to 2nd frame : % rotation matrix around x-axis by angle alpha: Rot1x_aL=[ 1 0 0 0 ; 0 caL2 -saL2 0 ; 0 saL2 % transation in x-axis by a2 Trans1x_a=[ 1 0 0 a2 ; 0 1 0 0 ; 0 0 1

caL2

0 ; 0

1 ];

0 ; 0 0 0 1 ];

% rotation matrix around z-axis by angle theta: Rot1z_th=[ c2t -s2t 0 0 ; s2t c2t 0 0; 0 0 1 0 % transation in z-axis by d1 Trans1z_d=[ 1 0 0 0 ; 0 1 0 0 ; 0 0 1 d2 ; 0 0 0 1 ];

; 0 0 0 1 ];

% transformation matrix from ith frame to ith+1 frame is ( B matrix ): B=Rot1x_aL*Trans1x_a*Rot1z_th*Trans1z_d Rb=B(1:3,1:3); tb=B(1:3,4);

C-2

Ltb=norm(tb,2); Lf=Ltb; % define the buffers for storge: Fia1=[];Fi1=[];Fi2=[];the4=[];the1=[];the2=[];the3=[];T1F=[];TyFa=[]; the4a=[];the1a=[];the2a=[];the3a=[];the5=[];the6=[];the7=[];TT1F=[]; FiF=[];Gfr=[];E2=[];TT1F=[];T2F=[];TyF=[];EFiF=[];Ecf=[];the4q=[]; tgxF=[];tgyF=[];tgzF=[];thez4=[];tgyF1=[];tgzF1=[];tgxF1=[];TGF=[]; T1fr=[];T2fr=[];Tyfr=[];tgyF2=[];tgzF2=[];tgxF2=[]; as=3; %******************************************************* % define the condition for envelope of moveing of arm: mm1=abs(Lta.^2+Ltb.^2-Ltg.^2); mm2=abs(2*Lta*Ltb); if mm1 > mm2 disp('this coordinate ot of range movement') stop end N=10; %******************************************************* % Solve elbow angle theta4 : Eqn 2.2.7: % Choice the value of elbow angle from two values: %------------------------------------------------------for ii=1:N; tg=[tgx tgy tgz];

tgxF1=[tgxF1;tgx]; tgyF1=[tgyF1;tgy]; tgzF1=[tgzF1;tgz]; Ltg=norm(tg,2); th4n1=2*atan2(0.5*(-4*caL2*d2*a1*c1t+4*a2*d1-2*(4*tgx.^2*a1*s1t*saL2*d2 ... -a1.^4-d1.^4-tgx.^4-a2.^4-tgz.^4-d2.^4 ... -a2.^2*a1*s1t*saL2*d2+4*tgz.^2*a1*s1t*saL2*d2 ... +4*d1.^2*a1*s1t*saL2*d2+2*a2.^2*d1.^2+2*a1.^2*tgx.^2 ... +2*a1.^2*tgy.^2+2*a1.^2*tgz.^2-2*a1.^2*d1.^2-2*a1.^2*a2.^2 ... -2*a1.^2*d1.^2+2*d1.^2*tgx.^2+2*d1.^2*tgy.^2 ... +2*d1.^2*tgz.^2-2*d1.^2*d2.^2-2*tgy.^2*tgx.^2-2*tgy.^2*tgz.^2 ... +2*tgy.^2*d2.^2+2*tgy.^2*a2.^2-2*tgx.^2*tgz.^2+2*tgx.^2*d2.^2 ... +2*tgx.^2*d2.^2+2*tgx.^2*a2.^2+2*a2.^2*tgz.^2-2*a2.^2*d2.^2 ... +2*tgz.^2*d2.^2+4*caL2.^2*d2.^2*d1.^2+4*a2.^2*a1.^2*c1t.^2 ... +tgy.^4+4*caL2.^2*d2.^2*a1.^2*c1t.^2-4*a1.^3*s1t*saL2*d2 ... -4*d2.^3*a1*s1t*saL2-4*a1.^2*s1t.^2*saL2.^2*d2.^2 ... +4*tgy.^2*a1*s1t*saL2*d2).^0.5),(a1.^2+d1.^2-tgy.^2-tgx.^2 ... +a2.^2-tgz.^2+d2.^2-2*caL2*d2*d1+2*a1*s1t*saL2*d2-2*a2*a1*c1t))

% or

C-3

th4n2=2*atan2(0.5*(-4*caL2*d2*a1*c1t+4*a2*d1+2*(4*tgx.^2*a1*s1t*saL2*d2 ... -a1.^4-d1.^4-tgx.^4-a2.^4-tgz.^4-d2.^4 ... -a2.^2*a1*s1t*saL2*d2+4*tgz.^2*a1*s1t*saL2*d2 ... +4*d1.^2*a1*s1t*saL2*d2+2*a2.^2*d1.^2+2*a1.^2*tgx.^2 ... +2*a1.^2*tgy.^2+2*a1.^2*tgz.^2-2*a1.^2*d1.^2-2*a1.^2*a2.^2 ... -2*a1.^2*d1.^2+2*d1.^2*tgx.^2+2*d1.^2*tgy.^2 ... +2*d1.^2*tgz.^2-2*d1.^2*d2.^2-2*tgy.^2*tgx.^2-2*tgy.^2*tgz.^2 ... +2*tgy.^2*d2.^2+2*tgy.^2*a2.^2-2*tgx.^2*tgz.^2+2*tgx.^2*d2.^2 ... +2*tgx.^2*d2.^2+2*tgx.^2*a2.^2+2*a2.^2*tgz.^2-2*a2.^2*d2.^2 ... +2*tgz.^2*d2.^2+4*caL2.^2*d2.^2*d1.^2+4*a2.^2*a1.^2*c1t.^2 ... +tgy.^4+4*caL2.^2*d2.^2*a1.^2*c1t.^2-4*a1.^3*s1t*saL2*d2 ... -4*d2.^3*a1*s1t*saL2-4*a1.^2*s1t.^2*saL2.^2*d2.^2 ... +4*tgy.^2*a1*s1t*saL2*d2).^0.5),(a1.^2+d1.^2-tgy.^2-tgx.^2 ... +a2.^2-tgz.^2+d2.^2-2*caL2*d2*d1+2*a1*s1t*saL2*d2-2*a2*a1*c1t))

% choice as value according to three choices in algorithm: % as=input('enter 1 or 2 for select the angle of rotate elbow='); if as==1 th4=th4n1; elseif as==2 th4=th4n2 else th4=1.571;%input('enter the correct value of theta4='); end the4=[the4;th4]; % if elbow position e is known but swivel angle unknown: % solve Eqn: 2.2.16: %-------------------------------------------------------------------%define the elbow position: ex=6;ey=1;ez=6; % to calculate the swivel angle phi is : Fia= atan2((abs(((tgz*ey*ax-tgz*ex*ay-ez*tgy*axey*tgx*az+tgy*ex*az+ez*tgx*ay)*tgx)./ ... ((abs(-ax*tgy.^2-ax*tgz.^2+tgx*ay*tgy+tgx*az*tgz).^2+abs(ay*tgx.^2+ ... ay*tgz.^2-tgy*ax*tgx-tgy*az*tgz).^2+abs(az*tgx.^2+az*tgy.^2-tgz*ax*tgx ... -tgz*ay*tgy).^2).^0.5)).^2 ... +abs(((tgz*ey*ax-tgz*ex*ay-ez*tgy*axey*tgx*az+tgy*ex*az+ez*tgx*ay)*tgy)./ ... ((abs(-ax*tgy.^2-ax*tgz.^2+tgx*ay*tgy+tgx*az*tgz).^2+abs(ay*tgx.^2+ ... ay*tgz.^2-tgy*ax*tgx-tgy*az*tgz).^2+abs(az*tgx.^2+az*tgy.^2-tgz*ax*tgx ... -tgz*ay*tgy).^2).^0.5)).^2 ... +abs(((tgz*ey*ax-tgz*ex*ay-ez*tgy*axey*tgx*az+tgy*ex*az+ez*tgx*ay)*tgz)./ ...

C-4

((abs(-ax*tgy.^2-ax*tgz.^2+tgx*ay*tgy+tgx*az*tgz).^2+abs(ay*tgx.^2+ ... ay*tgz.^2-tgy*ax*tgx-tgy*az*tgz).^2+abs(az*tgx.^2+az*tgy.^2-tgz*ax*tgx ... -tgz*ay*tgy).^2).^0.5)).^2).^0.5, ... (tgz.^2*ex*ax+tgz.^2*ey*ay-tgz*ez*ay*tgy-tgz*ez*ax*tgx-tgz*tgx*ex*az ... -tgz*tgy*ey*az-tgy*ey*ax*tgx+tgx.^2*ez*az-tgx*ex*ay*tgy+tgy.^2*ez*az ... +ey*tgx.^2*ay+ex*tgy.^2*ax)/ ... ((abs(-ax*tgy.^2-ax*tgz.^2+tgx*ay*tgy+tgx*az*tgz).^2+abs(ay*tgx.^2+ ... ay*tgz.^2-tgy*ax*tgx-tgy*az*tgz).^2+abs(az*tgx.^2+az*tgy.^2-tgz*ax*tgx ... -tgz*ay*tgy).^2).^0.5)) Fia1=[Fia1;Fia] ni=tg./Ltg; LLa=norm(a,2); ai=a./LLa; vb=ai-(dot(ai,ni)*ni); Lvb=norm(vb,2); ui=vb./Lvb; vi=cross(ni,ui); aL=acos((Lta.^2+Ltg.^2-Ltb.^2)/(2*Lta*Ltg)); CC=cos(aL)*Lta*ni; % Eqn 2.2.10 CC_a=((Lta.^2+Ltg.^2-Ltb.^2)/(2*Lta*Ltg))*Lta*ni RR=sin(aL)*Lta RR_a=(1-((Lta.^2+Ltg.^2-Ltb.^2)/(2*Lta*Ltg)).^2).^0.5*Lta ; % Eqn 2.2.11 eFi=CC+RR*((cos(Fia)*ui)+(sin(Fia)*vi)); EFiF=[EFiF;eFi]; %************************************************************************ % to calculate R1 ,R2 ,th1 --- th7 : % Define the previous elbow position eold % Define the previous end-effector position tgold %-----------------------------------------------------------------------% solve sholuder rotation matrix R1 Eqn 2.2.18 c4=cos(th4);s4=sin(th4); nor_eo=((abs(exo)).^2+(abs(eyo)).^2+(abs(ezo)).^2).^0.5; nor_en=((abs(ex)).^2+(abs(ey)).^2+(abs(ez)).^2).^0.5; h1=norm(eold,2); %used norm matlab function to check the difference Xold=[a1 saL1*d1 caL1*d1]; h3=norm(Xold,2); wold=[tgxo tgyo tgzo] qq1=c1t*c4*a2+s1t*saL2*d2+c1t*s4*caL2*d2+a1 ; % used Eqn 2.2.20 qq2=((caL1*s1t*c4+saL1*s4)*a2)-caL1*c1t*saL2*d2+(caL1*s1t*s4-saL1*c4)*caL2*d2saL1*d1; qq3=((caL1*s1t*c4-saL1*s4)*a2)caL1*c1t*saL2*d2+(caL1*s1t*s4+saL1*c4)*caL2*d2+saL1*d1;

% eqn of normal vector

C-5

wwold=[qq1

qq2

qq3]

% used Eqn 2.2.20

h4=norm(wwold,2); ex=eFi(1);ey=eFi(2);ez=eFi(3); enew=[ex ey ez] h2=norm(enew,2) ; wnew=[tgx tgy tgz]; X=eold./h1; xgoal=enew./nor_en; XX=Xold./h3; y1=wold-(dot(wold,X)*X); y1n=norm(y1,2); Y=y1./y1n; yy1=wwold-(dot(wwold,XX)*XX); yy1n=norm(yy1,2); YY=yy1./yy1n; y1g=wnew-(dot(wnew,xgoal)*xgoal); y1gn=norm(y1g,2); ygoal=y1g./y1gn; Z=cross(X,Y); ZZ=cross(XX,YY); zgoal=cross(xgoal,ygoal); rt1=[xgoal;ygoal;zgoal]'; rt1(4,4)=1; R1a=rt1; rt2=[X;Y;Z]';rt2(4,4)=1; rrt2=[XX;YY;ZZ]';rrt2(4,4)=1; R2a=rt2'; RR2a=rrt2'; disp('solve by used Eqn 2.2.18'); T1=R1a*R2a % solve by used Eqn 2.2.18 disp('solve by used Eqn 2.2.20'); TT1=R1a*RR2a; % solve by used Eqn 2.2.20 T1F=[T1F;T1]; TT1F=[TT1F;TT1]; %*************************************************************** % solve for T2 matrix Ry=[c4 0 s4;0 1 0;-s4 0 c4]; Ty=Ry;Ty(4,4)=1 TyF=[TyF;Ty]; R1=T1(1:3,1:3); th1=atan2(R1(3,2),R1(3,3)); s1=sin(th1); th2=atan2(-R1(3,1),R1(3,2)/s1); th3=atan2(R1(2,1),R1(1,1));

C-6

%************************************************************** R2=Rb'*Ry'*Ra'*R1'*Rgch; ra2=R2; ra2(4,4)=1; T2=ra2 T2F=[T2F;T2]; th5=atan2(R2(3,2),R2(3,3)); s5=sin(th5); th6=atan2(-R2(3,1),R2(3,2)/s5); th7=atan2(R2(2,1),R2(1,1)); %****************************************************************** % Inverse Kinematics: the1=[the1;th1] the2=[the2;th2] the3=[the3;th3] the5=[the5;th5] the6=[the6;th6] the7=[the7;th7] %*************************************************************** %%%%%%%%%%%%%%%% INVERSE KINEMATIC ALGORITHM %%%%%%%%%%%%%%%%%%% %*************************************************************** s1=sin(th1);c1=cos(th1);s2=sin(th2);c2=cos(th2); s3=sin(th3);c3=cos(th3);s4=sin(th4);c4=cos(th4); s5=sin(th5);c5=cos(th5);s6=sin(th6);c6=cos(th6); s7=sin(th7);c7=cos(th7); %************************************************************* %disp(' THE SHOLUDER MATRIX:'); %disp(' Rotate around X_axis by theta1:'); Rot_sx=[ 1 0 0 ; 0 c1 -s1 ; 0 s1 c1 ]; %disp(' Rotate around Y_axis by theta2:'); Rot_sy=[ c2 0 s2 ; 0 1 0 ; -s2 0 c2 ]; %disp(' Rotate around Z_axis by theta3:'); Rot_sz=[ c3 -s3 0 ; s3 c3 0 ; 0 0 1 ]; RSxyz=Rot_sz*Rot_sy*Rot_sx; R1f=RSxyz; T1f=RSxyz; T1f(4,4)=1 T1fr=[T1fr;T1f]; %disp(' THE ELBOW MATRIX :'); %disp(' Rotate around Y_axis by theta4:'); Rot_ey=[ c4 Ry=Rot_ey; 0 s4 ; 0 1 0 ; -s4 0 c4 ];

C-7

Tyf=Rot_ey; Tyf(4,4)=1 Tyfr=[Tyfr;Tyf]; %disp('THE WRIST MATRIX:'); %disp(' Rotate around X_axis by theta5:'); Rot_wx=[ 1 0 0 ; 0 c5 -s5 ; 0 s5 c5 ]; %disp(' Rotate around Y_axis by theta6:'); Rot_wy=[ c6 0 s6 ; 0 1 0 ; -s6 0 c6 ]; %disp(' Rotate around Z_axis by theta7:'); Rot_wz=[ c7 -s7 0 ; s7 c7 0 ; 0 0 1 ]; RWxyz=Rot_wz*Rot_wy*Rot_wx; R2f=RWxyz; T2f=RWxyz; T2f(4,4)=1 T2fr=[T2fr;T2f];

Gf=T1f*A*Tyf*B*T2f Gfr=[Gfr;Gf]; %pause; tgxa=Gf(1,4); tgya=Gf(2,4); tgza=Gf(3,4); ee1=tgx-tgxa ee2=tgy-tgya ee3=tgz-tgza %pause; tgx=tgxa tgy=tgya tgz=tgza tgxF=[tgxF;tgxa]; tgyF=[tgyF;tgya]; tgzF=[tgzF;tgza]; %**************************************************** end ii=1:N;tgxo(1:ii)=4;tgyo=tgxo;tgzo=tgxo; figure; plot(ii,tgxF1,'--.r',ii,tgxF,'-.og',ii,tgxo,'*');title('tgx'); figure; plot(ii,tgyF1,'*r',ii,tgyF,'-.og',ii,tgyo,'*');title('tgy'); figure; plot(ii,tgzF1,'.r',ii,tgzF,'-.og',ii,tgzo,'*');title('tgz');

C-8

% % % % % %

----------------------------------------------------------program No.Two ------- identification of Neural Network by EMG signal ------The signal get from seven muscles ------FAST MOMENTUM BACKPROBAGATION ALGORITHM ------------------------------------------------------------

clear clc tic %----------------------------------------------------% LOAD DATA FOR emg SIGNALS: load yydat % REARRAGEMENTS OF DATA TO INPUT NN: ip=[ yo1 yo2 yo3 yo4 yo5 yo6 yo7 op=ip; mm=max(max([ip])); mm1=mm+(0.45.*mm); ipn=ip./mm1; opn=op./mm1; % -------------------n=7; % input neurons p=30; % hidden neurons m=7; % output neurons alpha=.01; % learning rate mu=.015; % momentum term %-----------------------------Y=[];ww3=[];bb3=[];ww4=[];bb4=[];opp=[]; N=max(size(ip)); % % % % ---------------------- initialization --all initial cond. = 0 --------------------for i=1:n; for j=1:p; w3(i,j)=0.00001;%rand*1.65-1*.001; dw3(i,j)=0; end end for j=1:p; b3(j)=0.0;%rand*1.65-1*.001; db3(j)=0; end for i=1:m; for j=1:p; w4(i,j)=0.0;%rand*1.65-1*.001; dw4(i,j)=0; end end

];

C-9

for j=1:m; b4(j)=0.0;%rand*1.65-1*.001; db4(j)=0; end % -------------------% -- input to sys. --% -------------------yd(1)=0; % initial condition iter=0;B=40; Y1=[ 0 0 0 0 0 0 0 ]; for kk=1:N-1; for i=1:n; x(kk,i)=ipn(kk,i); end % -------------------% forward propagation % -------------------for j=1:p; % bipolar activation function [-1,1], bounded zin=0 ; % processing (n-p) for i=1:n; zin = zin + x(kk,i) * w3(i,j); end zin = zin + b3(j); z(j) = ( 1 - exp(-1 * zin) ) / ( 1 + exp(-1 * zin) ); end for i=1:m; % processing (p-m) yin(i)=0 ; for j=1:p; yin = yin + z(j) * w4(i,j); end yin(i) = yin(i) + b4(i); y(i) = ( 1 - exp(-1 * yin(i)) )*B / ( 1 + exp(-1 * yin(i)) );%yin(i);% end Y1=[Y1;y]; ww3=[ww3;w3]; ww4=[ww4;w4]; bb3=[bb3;b3]; bb4=[bb4;b4]; opp=[opp;opn]; % -------------------% backward propagation % -------------------for i=1:m; errin1(i) = ( opn(kk,i) - Y1(kk,i) ) * ( 1 + y(i) ) * ( 1 - y(i) ); end % 1st input

C-10

% ---------for j=1:p; sign=0; for i=1:m; sign = sign + errin1(i) * w4(i,j); end errin2(j) = sign * ( 1 + z(j) ) * ( 1 - z(j) ); end % ---------for j=1:p; for i=1:m; dw4(i,j) = alpha * errin1(i) * z(j) + mu * dw4(i,j); end end % ---------for i=1:n; for j=1:p; dw3(i,j) = alpha * errin2(j) * x(i) + mu * dw3(i,j); end end % ---------for j=1:p; db3(j) = alpha * errin2(j); end % ---------db4 = alpha * errin1; % ---------for i=1:n; for j=1:p; w3(i,j) = w3(i,j) + dw3(i,j); end end % ---------for i=1:m; for j=1:p; w4(i,j) = w4(i,j) + dw4(i,j); end end % ---------for i=1:p; b3(i) = b3(i) + db3(i); end % ---------b4 = b4 + db4; % -------------------% - error processing % -------------------% iter=iter+1; for i=1:m; we(kk,i) = abs( opn(i) - Y1(kk,i) ); % mean square error error(kk,i) = ( .5 * we(kk,i) * we(kk,i) ); end %error

C-11

%pause; % -------------------end % end of main for loop Y2=Y1.*mm1; figure subplot(411);plot(error(:,1)); subplot(412);plot(error(:,2)); subplot(413);plot(error(:,3)); subplot(414);plot(error(:,4)); figure subplot(311);plot(error(:,5)); subplot(312);plot(error(:,6)); subplot(313);plot(error(:,7)); figure hold on plot(Y2(:,1)); plot(op(:,1),'--r'); hold off figure hold on plot(op(:,2),'--k'); plot(Y2(:,2),'m'); hold off figure hold on plot(Y2(:,3)); plot(op(:,3),'--r'); hold off figure hold on plot(op(:,4),'--k'); plot(Y2(:,4),'m'); hold off figure hold on plot(Y2(:,5)); plot(op(:,5),'--r'); hold off figure hold on plot(op(:,6),'--k'); plot(Y2(:,6),'m'); hold off figure hold on plot(op(:,7),'--g'); plot(Y2(:,7)); hold off time_learning=toc w3f=ww3; b3f=bb3; w4f=ww4; b4f=bb4; Yn=Y2; % SAVING NN DATA TO USED IN NN CLASSIFICATIO ..... save ydat_idenf w3f b3f w4f b4f Yn Y1 time_learning

C-12

 : ) (. . . : . .

 . , . ) ( EMG . . . : : . : ) ( . . ) ( EMG ) ( EMG ) ( EMG ) .( EMGlab

) (

) 9991 (

...

9002

0341