5

CHAPTER 2 USING EMG DATA IN PROSTHETIC HAN CONTROL Prosthetic hands give hope for amputees to recapture their ability to perform complicated physical movements of the lost human hands. Electromyography (EMG) is a suitable approach for human-machine interface in the prosthetic hands control. The first artificial hand controlled by EMG was developed in Russia by A. E. Kobrinski in 1961. The Otto Bock Orthopedic Industries developed an EMG controlled multi-finger hand in 1965 [44]. In 1980s and 1990s, high-tech solutions were emerging which used new material and miniaturized components. Users can comfortably wear the lightweight electrically powered prosthetic hands during an eight hours day [44]. Integrated with modern computer system the future prosthetic hand probably can accept command direct from the human nerve system and perform multiple tasks. This technique can also be used in robot hand control such as remote dextrous manipulation devices [29], or teleoperation. 2.1 Electromyography (EMG) The Electromyography (EMG) is the signal measured by placing conductive elements or electrodes on the skin surface, or invasively within the muscle. Surface EMG, or skin EMG is used more often because its non-invasive character makes it safer and easier to use. The Electromyography (EMG) is useful in the field of both medicine and engineering. This section describes the relevant background knowledge on muscle architecture and how the muscle electricity is generated and detected.

6 2.1.1 Muscle Physiology The skeletal muscles implement our body movements. They are attached to adjacent bones via tendons, and function to induce movement at the joint formed where the two bones meet. Skeletal muscle comprises the largest single organ of the body. Each of these individual muscles is composed of single cells or fibers embedded in a matrix of collagen. The muscle cells are roughly cylindrical, with diameters between 10 and 100 um but up to a few centimeters long. They may be arranged in parallel and bound by a connective tissue envelope into a homogeneous bundle. A myofiber is a multinucleated single muscle cell. Its basically water with some dissolved ions separated from the extra-cellular space that is mostly water with some dissolved ions. It generates a potential difference across its cell membrane by having different concentrations of ions. The fibers are excitable cells. Excitation signals are received at the synapse. Then a rapid depolarization occurs and is coupled with a contraction. Its a process during which electrochemical events occur. The action potentials are propagated along the sarcolemma, or cell membrane, toward the end of the fiber and downward from the surface into the transverse tubular system. The propagation of the action potential along a nerve or muscle fiber includes the flow of ions and gives rise to extra-cellularly recordable potential gradient. These potential gradients, moving in both time and space, constitute the electricity as recorded from active muscle fibers [37]. Thus the small currents are generated prior to the generation of muscle force. The myofibers are the smallest complete contractile systems and are arranged in functional units called motor units. A motor unit is simply the cell body (motor neuron), its axon and all of the muscle fibers that it innovates (attaches to provide the signal for contraction). Each motor unit has a characteristic innervation ratio that is the number of muscle cells (muscle fibers)

7 controlled by one neuron. The number of muscle fibers belonging to a single motor unit varies widely from muscle to muscle. One whole muscle has many motor units. 2.1.2 Acquisition of EMG As the brain's signal for contraction increases, it both recruits more motor units and increases the "firing frequency" of those units already recruited. All muscle cells within one motor unit become active at the same time. By varying the number of motor units that are active, the body can control the force of the muscle contraction. When individual motor contract, they repetitively emit a short burst of electrical activity known as the motor unit action potential (MUAP). It is detected by electrodes on the surface of the skin in proximity of the motor. The detection is illustrated in the following figure [27]:

Figure 2-1: Detection of the motor unit action potential (MUAP) The function unit of a muscle is the motor. All the fibers which belong to one motor are activated at the same time. The motor unit action potential (MUAP) is the electrical response to the impulse from the axon. A MUAP looks like the following figure.

Figure 2-2: Action potential (AP) of one motor unit [26] The primary factors that determine the shape of a MUAP are the diameter and geometrical arrangement of the muscle fibers, the tissue filtering effect, and the properties of the recording electrode and instrumentation [27]. The contraction of a muscle recruits a number of motors during a period of time. When several motor units are active (the timing of the electrical burst between distinct motor units is mostly uncorrelated), a random interference pattern of electrical activity results. The time between successive bursts is somewhat random for each motor unit. EMG technology enables us to record the action potentials from an entire muscle or a large portion of it by putting electrodes on the surface on the skin. It is a summation of the interfered motors MUAP, which is also the summation of small currents from the fibers belonging to variant motors. Therefore it looks quite different with MUAP. The following Figure shows how the detected EMG signals and its spectrum appear. We can observe that the EMG signal of greatest amplitude is detected by put the electrode at proper locations on the surface the muscle [25].

Figure 2-3: Detection of EMG Signal. It is well established that the amplitude of the EMG signal is stochastic (random) in nature and can be reasonably represented by a Gausian distribution function. The instantaneous value of EMG signal contains no information. The random nature of the action potential of the motor makes EMG to be stochastic. The usable energy of the signal is limited to the 0 to 500 Hz frequency range. Usable signals are those with energy above the electrical noise level [25]. There are many factors that influence the EMG signals detection. These include the electrode structure and its placement on the surface of the skin above the muscle. The physiological, anatomical and biochemical characteristics of the muscle can also influence the EMG signals [25]. 2.2 Research in SDSU

Dr. Marko Vuskovic led an ongoing research in San Diego State University (SDSU) on the application of EMG signal on the prosthetic hand control [46]. The following is the system overview:

10

Figure 2-4: Control of the multifunction prosthetic hand using surface EMG signals The EMG amplifier system used in this study was the Musle Tester ME3000 Professional System manufactured by Mega Electronics, Ltd. The relevant muscles are the extensor muscle on the upper forearm. Four electrode sets were placed above the following muscles: extensor pollicis group (thumb), extensor communis digitorun (index, middle) and extensor carpi ulnaris (little) [47][48]. EMG data are collected off-line from the four electrodes, or four channels, and stored into a database. As we know, human hands usually form a basic grasp shape before grasping an object [44]. This is called preshaping. Six grasps types are repeatedly tested: lateral, precision

11 (one opposition), cylindrical with small aperture, cylindrical with large aperture, spherical with small aperture and spherical with large aperture. These motions were performed by grasping six appropriate objects: small cylinder (SC), large cylinder (LC), small ball (SB), large ball (LB), small disk (SD) and a key (SK). Currently only the healthy human hand is involved in the project. The research work in SDSU implemented the feature extractor and grasp classifier modules in the figure 2-4. The raw signals are obtained off-line and stored as data files available for the various experiments. The goal of this project is to apply the system to an amputees hand in the future. The hand control will be processed in real-time with the support of embedded microprocessor and special hardware. But the project is at experimental stage right now. All the experiments are performed off-line, not real time, not the involving real amputees limb and not equipping special hardware. The future control scheme can be described as the following: The information collected from the amputees remaining muscle is employed to train a classifier. The classifier recognizes the contraction patterns specific to each amputee after the training. The EMG signals from the remaining muscles of the amputee are measured with the multiple electrodes. The robot hand performs suitable grasp motion immediately after the preshaping type is determined. In this approach the amputee imagines the motion of the lost hand and presumes the motion. The nerve system sends action potentials to the muscle cells. Then small currents are released immediately. The EMG signals are detected with the electrodes on the skin surface of the amputees remaining limb. The raw analog EMG signals are collected and pre-processed (filtering, sampling, etc.) and then converted to the raw digital EMG data. These digital data are sent to an integrated microprocessor system for the feature extraction and classification. Then

12 signals for position and force control are computed. The motors drivers produce the actual hand motion based on the control signals. This control scheme is capable of recognizing a multitude of volitional prehensile movements, making the hand control transparent in a sense that the subject can move the hand in a most natural way as if he/she would do with a healthy hand [49]. The feature extractor and grasp classifier modules are important to this control scheme because the real time response depends much on if a shorter time period of signal can be used to produce a good classification accuracy. In order to facilitate the experiment a real robot hand is added as the test tool in SDSU. The following figure shows the robot hand in SDSU robotic laboratory. It is a human-sized robotic hand with four fingers and a thumb. Each finger can perform flexion and extension through three joints. The thumb has three joints also--two of them perform flexion and extension, while the third joint performs thumb abduction and adduction about an axis parallel to the wrist [46].

Figure 2-5: SDSU Hand

13 2.3 Classification of EMG Patterns The classification of EMG patterns is the mapping of the measured EMG signals into the human motions or gestures. Bernard Hudgins and Philip Parker [36] proposed in 1993 a new approach to the control of a multifunction prosthesis based on the classification of myoelectric patterns. They suggested that there is a considerable structure in the EMG signals temporal waveform during the onset of a contraction by observing the ensemble average of EMG signals during the initial 300 ms of the motions. Blake Hannarford and Steven Lehman [35] established in 1986 that there exists a consistent spectral shift for the EMG signals recorded during rapid movements. The spectrums are computed by the windowed Fourier transform, or the short time Fourier transform. Both central frequency and the variance of the spectral energy distribution grew with time. Its similar to the voiceprint in the speech recognition. This consistency suggested the existence of EMG patterns. The classification of EMG patterns is widely accepted and implemented due to the structure property of the EMG signals. The concept is attractive because a control scheme based on signal dynamic would eliminate the delay associated with signal averaging and thus increases the time available for signal analysis [36]. Kevin Englehart and Bernard Hudgins described the classification as a multiple stage process [40] in the following figure:

Figure 2-6: Break down the classification of the EMG signal into multiple stages

14 Feature extraction is the first stage of the process. It plays a crucial role for the classification performance. We will introduce it in more details in the next section of this chapter. Feature selection methods attempt to determine the best subset of the original feature set. Feature projection methods attempt to determine the best combination of the original features [40]. After these initial tasks the raw EMG data are converted to a data set that can represent the EMG signals in a most efficient way to discriminate the motions or gestures caused by muscle contraction. Usually organized as feature vectors, these representations are meaningful and ready to be used by the classifier. The classification task in the figure includes linear discriminant analysis (LDA) classifier and multilayer perceptron (MLP) classifier. LDA and the MLP are easy to implement and are representatives of statistical and neural classifiers respectively, which were well understood [40]. Pattern classifiers with adaptability are desired in this application because of the nature of EMG signals. As we introduced in chapter 1 there are many factors influencing the acquisition of EMG signals. Significant differences and variances of features are expected for different individuals. Similarly there can be differences in EMG signals of a single person measured at different times. The patterns might be completely altered for different individuals, while for one individual the changes upon time might be some shifting, increasing or growing of the patterns. The classifiers need to be able to deal with these differences and variances while maintaining a certain level of stability. Dr. Vuskovic introduced a new type of simplified fyzzy ARTMAP Network (SFAM) [49] [50]. The network is an Adaptive Resonance Theory Based neural network that is focused on using Mahalanobis distance.

15 The Mahalanobis distance is a normalized distance. It is invariant of the unit size of the cluster. The use of covariance matrix and Mahalanobis distance in SFAM made the clustering simple and efficient. It can be computed in a recursive manner [49][50]. The cluster described by Mahalanobis distance is basically assumed to have a Gaussian distribution. Furthermore, the assumption is somehow consistent with the fact that the amplitude of the EMG signal is stochastic (random) in nature and can be reasonably represented by a Gausian distribution function [25], if only the temporal information in EMG is used for the clustering here. All the experiments in this work employed the Mahalanobis distance based SFAM as the pattern classifier. 2.4 Feature Extraction Approaches For EMG Signals Feature extraction is a process by which signal attributes are computed and collected into a compact vector format. Feature extraction can be considered as data compression that removes irrelevant information and preserves relevant information from the raw data. [15] In general the methods for feature extraction are crucial to the successful classification. A successful classification system depends not only on reliable, stable and adaptable classifiers, those as SFAM, but also on the strategies and methods to extract and select features from the raw data. Classifiers perform better on feature vectors which can represent observed objects more efficiently such that the relevant information to patterns are emphasized the most. There are two basic approaches for EMG feature extraction. One is temporal approach and the other is frequency approach. The temporal approach identifies the attributes of the raw EMG signal that characterize its temporal structure relative to a specific muscular function [48]. Temporal approach uses the information in time domain as feature values without doing any further transformation. Hudgins

16 proposed some temporal features: mean absolute value, zero crossings, slope sign changes, and waveform length [36]. Some other methods are proposed in previous work in SDSU [44] such as enveloping parameters, square Integral, and amplitude of the first burst. In the presented work temporal methods called multiple time widows are proposed and tested. These methods are:
Multiple hamming windows Multiple trapezoidal windows Multiple Slepian windows

The raw EMG signals are segmented by these windows. The energy contained inside a window is extracted as a feature value. The change of the EMG energy with time is thus emphasized. Some encouraging classification rates have been achieved. The result is compared with the previously used square integral method. The frequency approach identifies the attributes of the raw EMG signal from its spectrum in frequency domain. The spectrum is the transformed sequence based on Fourier theory or on wavelet theory. The feature sets derived from time-frequency representations are explored and presented in several papers [33][40][41][42][43]. These feature extraction methods include the short time Fourier transform (STFT), the wavelet transform, and the wavelet packet transform. The short time Thomsons transform (STTT) has been proposed and has shown to be superior over STFT. The STTT is based on D.J. Thomsons power spectral estimation theory [34]. In the presented work both the STFT and STTT are experimentally evaluated and the results are compared. The following measurements are used as feature values: The moments of the spectrum

17 The spectral magnitude averages covering each frequency region

The Slepian windows, or the discrete prolate spheroidal sequences (DPSS), are used in both multiple time windows and STTT. The encouraging performance suggests that DPSS are fit for EMG signal for feature extraction. We will introduce the temporal approach and frequency approach in detail in chapter 4 and chapter 5.