Proceedings of the 2005 IEEE International Conference on Robotics and Automation Barcelona, Spain, April 2005

Development of the NAIST-Hand with Vision-based Tactile Fingertip Sensor

Jun Ueda, Yutaka Ishida, Masahiro Kondo, Tsukasa Ogasawara Robotics Laboratory, Graduate School of Information Science, Nara Institute of Science and Technology (NAIST), Nara, 630-0192, Japan uedajun@is.naist.jp
Abstract This paper introduces a multingered robotic hand eNAIST-Handf and a grip force control by slip margin feedback. The developed prototype nger of the NAISThand has a new mechanism by which all 3 motors can be placed inside the palm without using wire-driven mechanisms. A method of grip force control is proposed using an incipient slip estimation. A new tactile sensor is designed to active the proposed control method by the NAIST-Hand. This sensor consists of a transparent semispherical gel, an embedded small camera, and a force sensor in order to implement the direct slip margin estimation. The structure and the principle of sensing are described. Index Terms Multingered Robotic Hand, Robot Design, Tactile Sensation, Incipient Slip, Slip Margin Feedback

In contrast, recently developed hands become more compact by embedding motors and bevel gears in hand itself without using the wire mechanism[6][7][8][9]. The improvement of actuators, especially on compactness and eciency, is considered as its background. In these structures, the actuators for driving the MP joint are embedded in the palm, and the ones for the PIP and DIP joint are embedded in the ngers link. However, there is a limit of miniaturization of the actuators. This factor imposes a severe restriction on choosing sucient actuators especially for embedding in ngers link. Therefore, the ultimate ngertip force is limited even if sucient torque is generated by the MP joint. Considering the above issues, the NAIST Hand was designed under the following concepts for good maintainability and relaxing the restriction of the space for actuators. All actuators are embedded in the palm. All joints are driven by gears and link mechanisms without using wires. Each nger is designed in modules.

I. Introduction The manipulation by multingered robot hands has been started in the early stage of the robotics research and is still actively studied. Especially, the dextrous manipulation using tactile sensation is considered important for achieving human-like dexterity; however, the current stage of this research is far from the goal. The NAIST Hand project has been started at Nara Institute of Science and Technology in 2002. Especially, in regard to the tactile sensation and manipulation, we have proposed a vision-based slip margin estimation and grip force control by its direct feedback[1]. The NAIST Hand is developed as a platform for these researches. This hand has 4 ngers and each nger has 3 degrees of freedom (DOF). Its specially designed gear mechanism has relaxed the restriction on the space for actuators, i.e., all three actuators can be embedded in the palm without using wire-driven mechanism. A vision-based tactile ngertip sensor is also presented. It consists of a transparent semispherical gel, an embedded small camera, and a force sensor in order to implement the direct slip margin estimation. II. Basic Concept of the NAIST Hand Multingered robot hands with almost the same size and degrees of freedom as human have already been developed in 1980s[2][3]. The wire driving system was adopted for these hands; however it required a huge actuator unit. This caused a problem when attaching the hand to a robot arm. Moreover, the maintainability is not good for cutting or tension control of the wires. Recent tendon-driven hands[4][5] have become more ecient, but it seems the problem is still remaining.

As shown in Fig. 1, the NAIST Hand has 4 ngers, 12 DOF in total. Each nger has 3 DOF; 2 DOF for MP joint, 1 DOF for PIP joint, the DIP joint is couples with the PIP. Since each nger is designed in modules, the conguration of the ngers are easily changed depending on a target task.

Fig. 1. NAIST Hand: 4 Finger Dextrous Hand

0-7803-8914-X/05/$20.00 2005 IEEE.

2343

III. Mechanical Design A. Finger Module A nger module has 3 joints (MP, PIP and DIP) as dened in Fig. 2. The MP joint has 2 DOF for adduction/abduction and exion/extension motion. The PIP joint has 1 DOF exion/extension motion. The DIP joints exion/extension motion is coupled with the PIP motion by a linkage. This is based on the knowledge in physiology that humans PIP and DIP joint synchronize. As a result, one nger has independent 3 DOF. According to the design concept, all 3 actuators are embedded in the palm; Motor 1, 2, and 3 for MP(adduction/abduction), MP(exion/extension), and PIP(exion/extension) respectively. For the motor 2 and 3, two DC motors (2.6[W]) are used with harmonic drive gear(CSF-mini, gear-ratio 50:1) and encoder(360 pulse/rev). The output shaft of the motor 2 is connected to the gear coaxial with the motor 3 by two pulleys and a timing belt. For the motor 1, a geared motor(1.9[W], gear-ratio 1670:1) with encoder(16pulse/rev) is used. Considering that the torque required for the exion/extension motion should be larger than for the adduction/abduction motion, a more compact motor is used for the motor 1. At the ngertip, a plate of insulator is attached for a force sensor. The length of the nger is approximately 150[mm] for this model. Note that this is not the lower bound since the link length is not aected by the actuators size. For the same reason, we also note that the size of the actuators used in this model is not the upper bound but used tentatively. B. 3-Axes Driving Gear Mechanism Generally, when the MP joint has 2 DOF, it seems dicult to drive the PIP joint by link mechanism from an actuator embedded in the palm. Therefore, in conventional hand mechanisms, the motors for the PIP/DIP joints have been embedded in the ngers link. This imposes a severe restriction on choosing sucient actuators. It causes a bottleneck to achieve a sucient ngertip force even if the MP joint torque is enough. As a matter of fact, this problem can easily be solved by reducing the DOF of the MP joint to one, or by using wire mechanism; however this contradicts with the aim. In the NAIST Hand, all actuators can be placed in the palm by specially designed gear mechanism shown in Fig.3. As shown in Fig. 2, almost the same size of motors can be used for the motor 2 and 3 which determine the ultimate ngertip force. Three actuators independently drive 3 axes; the adduction/abduction MP aa , and exion/extension MP f e of the MP joint, and the linkage driving angle rod that determines the exion/extension motion of the PIP joint as dened in Fig. 2 . These axes are given by: MP aa 1 0 0 1 /n1 MP f e = 2 1 0 2 /n2 rod 0 1 1 3 /n3

Motor1: MP (adduction/abduction) DIP Joint PIP Joint (coupled with PIP)

Motor2: MP (flexion/extension)

MP Joint Motor3: PIP (flexion/extension) Insulator for Force Sensor

Fig. 2. Finger Module (without Fingertip)

where 1 , 2 , and 3 are angles , and n1 , n2 , and n3 are the gear ratio of the motors 1 to 3 respectively(see Fig. 5). As shown in Fig.3, the rotational axes of MP aa , MP f e , and rod intersect at one point. Since the rotational axes of the PIP joint and that of rod always keep parallel even if the 2 DOF of the MP joint are driven, the PIP joint can be driven from the actuator on the palm using the link mechanism. Fig. 4 shows the gear mechanism in detail. Two dierent pairs consisting of three bevel gears are combined in the MP joint. The motor 1 directly drives MP aa . The output shaft of the motor 2 is connected to the bevel gear coaxial with the motor 3 by two pulleys and a timing belt, then drive MP f e through two bevel gears. The output shaft of the motor 3 passes through the gear and pulley connecting with the motor 2. It then drives rod through two bevel gears.
PIP(DIP) Flexion

rod

MPaa MPfe
Fig. 3. 3-Axes Driving Mechanism: MP(Adduction/abduction and exion/extension) and PIP(exion/extension)

C. PIP and DIP Joint Driving Fig. 5 shows the link mechanism that enables the coupling between P IP and DIP . As described in the previous section, the PIP joint is driven from the actuator on the palm using a link mechanism, which also drives the DIP joint. Fig.6 shows the coupling motion of the PIP and the DIP. Since it is not a complete parallel link mechanism, rod is not strictly linear to P IP . Similarly, P IP can not be strictly linear to DIP . The length of the links are adjusted by trial-and-error design to achieve approximately linear transmission as shown in Fig. 7. The linear approximations

(1)

2344

140 120

PIP [ deg ]

Motor1

100 80 60

Drive shaft of Motor3 Timing belt

40 20
Pulleys Motor3 Motor2

40

60

80 [ deg ]

100

120

rod
140

(a) MP and PIP angle

Fig. 4. 3-Axes Driving Mechanism in Detail

120

DIP [ deg ]

100 80 60 40 60 80 100 120 140

without including oset are given as: P IP 1.03rod , DIP 0.87P IP . In addition, the working range of the PIP joint is 0 P IP 90[deg] corresponding to the DIP joints 0 DIP 78.1[deg], which approximately follows physiology.
1

PIP [ deg ]
(b) PIP and DIP angle Fig. 7. Joint Angles

3
rod PIP

DIP

Fig. 5. Coupling Link Mechanism

Fig. 6. Flexion of PIP and DIP Joint by Link Mechanism

IV. Vision-based Tactile Fingertip Sensor A. Tactile Sensor for Robot Hand For achieving robotic grasping inspired by human tactile sensation, a number of tactile sensors have been developed. Maekawa et al. developed a nger-shaped sensor by which the contact point of the ngertip can be mea-

sured[10]. Ferrier et al. presented a reconstructing method of the shape of a deformable membrane from the camera image[11]. Howe et al. developed a tactile sensor with accelerometers which detect minute and high-frequency vibrations at the onset of a partial slip[12]. Tremblay et al. developed an improved version of above sensor[13]. The sensor skin is covered with small nibs which vibrate when the incipient slip occurs. This acceleration-based approach is eective for detecting the beginning of the incipient slip; however it seems dicult to measure the degree of the incipient slip since the acceleration measurement is largely aected by noise problem. In order to solve this problem, several tactile sensors have been developed, e.g., based on ultrasonic emission[14] and internal strain distribution sensing[15]. Maeno et al. proposed a method of grip-force control. It is based on the measurement of the internal strain distribution in the contact between a ngertip and a rigid plate, aiming at the case where the friction coefcient is unknown[16]. However, it is dicult to increase the spatial resolution because many strain gauges should be located inside the sensor. Considering the problems mentioned above, we have proposed a vision-based tactile sensation and grip-force control[1]. The proposed method has the following advantages: Highly accurate sensing by vision-based contact area measurement Direct estimation of the slip margin without knowing the friction coecient A stabile grip force by a direct feedback of the slip margin However, in the developed sensor, the camera was not embedded in the sensor. The contact area was measured outside from the ngertip. In this paper, we present a newly developed ngertip sensor for the NAIST Hand with

2345

Grip force control Camera Transparent gel

Robotic gripper

Traction force (disturbance)

Fig. 8. Stable Grasp with Vision-based Tactile Sensation

mal force raised to the power of 1/3[17] 1 . The normal force around the boundary is smaller than the normal force around the center. Therefore a slip between the elastic object and the rigid plate occurs from the boundary region. When the whole contact surface slips, the elastic object begins to slip completely to the rigid plate, which is called the gross slip. The partial slip that occurs before the gross slip is called the incipient slip[20]. In this paper, the contact region where a partial slip occurs is called as the slip region. The contact region where the objects surface is stuck is called as the stick region. The distance c from the contact center to the boundary between the stick region and the slip region is given by: c = a(1 )1/3 (2)

Elastic object

fg

fl

where = fl /fg is the tangential force coecient. is the friction coecient of the contact area. The tangential pressure becomes maximum on the boundary between the stick region and the slip region. Suppose the elastic object deforms by applying fg and fl . A relative displacement shown in Fig. 9(b) is occurred by the deformation of the elastic object. The analytic solution is given by the following equation[18]: 3fg 16a 2 G

Rigid plate
Fig. 9. Contact model of an elastic object and a rigid plate: Applying a normal and tangential force

1 1 2/3

(3)

where G = E/{2(1 + )}. E and are Youngs modulus and Poissons ratio of the elastic object. C. Direct Estimation of Slip Margin In (3), G and are known material constants of the elastic object. Then, , , fg , and a are needed to estimate by inverse solution of (3). As will be shown in the following sections, we use a camera to measure the contact surface, then and a are measured from this image. Additionally, fg is obvious since it is the output of the actuator itself. The remaining unknown variable is the friction coefcient ; however we focus on the grip-force control without knowing , therefore the following transformation of (3) is applied dividing both sides by fl . fl 3 1 16a 2 G 1 1 2/3

a transparent semispherical gel and an embedded small camera. The basic concept of the sensing and control is shown in Fig. 8. By estimating the slip margin during the incipient slip, a stable grasping is realized against the change of the traction force without occurring the gross slip. B. Principle of Sensing The contact between an elastic sphere and a rigid plate is called Hertzian contact[17] and various analysis have been presented[18]. The Hertzian contact is only applicable when (1) both bodies in contact are linear elastic, and (2) the contact deformation is small. In this paper, the above assumptions are applied and the nonlinear case is not considered to simplify the discussion. Fig. 9 shows the schematic diagram of the contact when applying the grip (normal) force fg and the traction (tangential) fl . In this analysis, we consider the situation where the contact holds, therefore fg > 0. Considering the sphere is symmetry around the direction of fg , we can assume that fl denotes the absolute value of the tangential force without losing the generality, i.e., fl > 0. When the elastic sphere is pressed and slid on a rigid plate, the sphere deforms depending on fg and fl . The radius of the contact area a is proportional to the nor-

(4)

The following equation is obtained by collecting : 3 2 + (1 32 ) + (3 2) = 0 (5)

1 Xydas et al. presented that the radius of contact is proportional to the normal force raised to the power of from 0 to 1/3[19]. In this sense, the Hertzian contact model is only a part of this model. However, this model has not been expanded to the case where the normal and the tangential force are simultaneously applied. In this paper, an analytic solution of the deformation of the elastic object is required for this case. Therefore, the analytic result based on the classic Hertzian model shown in (3) is applied.

2346

1.6 1.4

Partial slip

Estimated

1.2 1 0.8 0.6 0.4 0.2 0 0 1 2 3

b
Gross slip

of the contact area a, and the relative displacement of the feature point are required for the slip margin estimation. As revealed in the gures, there values can be detected. The relative displacement can be calculated by the eccentricity[21] which is an index of the deformation of the elastic semisphere.

a
4 5 6 7 8

Compact CCD camera LED

Traction force [N]

Fig. 10. Estimation of

where = 16Ga/(6 3 )fl . The solution of (5) a and b are obtained as follows: (1 32 ) {(1 32 )2 43 (3 2)}1/2 23 2 (1 3 ) + {(1 32 )2 43 (3 2)}1/2 b = 23

a =

(6) (7)

Force sensor

Transparent Feature point(dot) semispherical gel

Fig. 11. Prototype Fingertip Sensor

Fig. 10 shows an example of the plot obtained by two solutions a and b . In calculation of Fig. 10, fl = 10.0[N], = 0.5 are used. Since the admissible traction force f l is given by fl = fg = 5[N], a partial (incipient) slip occurs in the range where 0 < fl < 5, then the gross slip occurs where fl 5. Recall = fl /fg , then it is clear that a corresponds to in the incipient slip condition. In contrast, b corresponds to in the gross slip condition. Based on these analyses, a in the incipient slip condition can be calculated from (6) by measuring , fl , and a. By using this estimated a , a grip-force control during the incipient slip can be applied, i.e., it is unnecessary to estimate by slipping the object once in the gross slip condition. D. Prototype Sensor Unit In order to apply the estimation algorithm presented in the previous section, a prototype version of the ngertip sensor is developed. The basic structure is shown in Fig. 11. A transparent semispherical gel is used to measure the contact area by a small CCD camera from inside of the ngertip. A feature point(dot) is drawn on the apex of the gel. A LED is embedded in the gel which improve the contrast of the contact area image. A force sensor is supporting the unit to measure the external force applied on the gel. The sensor unit is attached on the NAIST Hands ngertip as shown in Fig. 13. The prototype unit is relatively big compared to the nger. We are now improving its structure and size. Fig. 12 (a) shows the initial, i.e., without tangential force, image of the camera when the sensor is pressed against a rigid wall. Fig. 12 (b) shows detected edge of the contact area and the feature point by image processing. Fig. 12 (c) and (d) show the image after applying the traction (tangential) force. Note again that the radius

(a) Initial image

(b) Initial image (binarized)

(c) Applying traction force

(d) Applying traction force (binarized)

Fig. 12. Image of contact area

Fig. 13. Prototype Sensor attached on NAIST Hand Fingertip

2347

V. Conclusion In this paper, a multingered robot hand NAIST Hand is presented which is developed in our hand project. For achieving human-like dexterity (Fig. 14), this hand has 12 DOF in total; 4 nger modules, and 3 DOF for each nger. We have designed a special gear mechanism by which all 3 actuators can be embedded in the palm, and the restriction on the space for actuators is relaxed. The kinematic analyses, especially for coupling of the PIP and DIP joint, are presented. A vision-based tactile ngertip sensor is also developed to implement the direct estimation algorithm of the slip margin. This project is just in progress. We have developed a direct and intuitive input system as shown in Fig.15. We are now tackling new research topics, e.g., a CPG-based manipulation[22], and a detection method of humans whole hand manipulation[23]. References
[1] A. Ikeda, Y. Kurita, J. Ueda, Y. Matsumoto, T. Ogasawara, Grip Force Control for an Elastic Finger using Vision-Based In-

Fig. 14. NAIST Hand Grasping an Object

cipient Slip Feedback, Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pp.810815, 2004. [2] M. T. Mason and J. K. Salisbury, Robot Hands and the Mechanics of Manipulation, The MIT Press, Cambridge, MA, 2nd edition, 1986. [3] S.C. Jacobsen , J.E. Wood, D.F. Knutti, K.B. Biggers, The Utah-MIT Dextrous Hand: Work in Progress, Int. J. Robotic Research, vol. 3, no. 4, pp. 21-50, 1983. [4] L. Biagiotti, F. Lotti, C. Melchiorri, G. Vassura, Mechatronic design of innovative ngers for anthropomorphic robot hands, Proc. IEEE Int. Conf. Robotics and Automation, pp.31873192, 2003. [5] M. V. Weghe, M. Rogers, M. Weissert, and Y. Matsuoka, The ACT Hand: Design of the Skeletal Structure, Proc. IEEE Int. Conf. Robotics and Automation, pp.33753379, 2004. [6] H. Kawasaki, H. Shimomura, Y. Shimizu, Educational-industrial complex development of an anthropomorphic robot hand Gifu hand, Advanced Robotics, Vol. 15, No. 3, pp. 357-363, 2001. [7] J. Butterfass, M. Grebenstein, H. Liu, G. Hirzinger, DLR-Hand II: Next Generation of a Dextrous Robot Hand II, Proc. IEEE Int. Conf. on Robotics and Automation, 2001. [8] A. Namiki, Y. Imai, M. Ishikawa, and M. Kaneko, Development of a High-speed Multingered Hand System and Its Application to Catching, Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pp.26662671, 2003. [9] J. Lee, Y. Youm and W. K. Chung, The Development of POSTECH HAND 5, Proc. IEEE Int. Conf. Robotics and Automation, pp.33863390, 2004. [10] H. Maekawa, , K. Tanie, and K. Komoriya, A Finger-Shaped Tactile Sensor Using an Optical Waveguide, Proc. IEEE Int. Conf. Systems, Man and Cybernetics, Vol. 5, pp. 403408, 1993. [11] N. J. Ferrier, R. W. Brockett, Reconstructing the Shape of a Deformable Membrane using Image Data, Int. J. Robotics Research, Vol. 19, No. 9, pp. 795816, 2000. [12] R. D. Howe and M. R. Cutkosky Sensing skin acceleration for texture and slip perception, Proc. IEEE Conf. Robotics and Automation, pp. 1416, 1989. [13] M.R. Tremblay and M.R. Cutkosky, Estimating Friction Using Incipient Slip Sensing During a Manipulation Task, Proc.IEEE Int. Conf. Robotics and Automation, pp. 363368, 1993. [14] H. Shinoda and S. Ando, Ultrasonic Emission Tactile Sensor for Contact Localization and Characterization, Proc. IEEE Int. Conf. Robotics and Automation, Vol.3, pp.25362543, 1994. [15] T. Maeno, T. Kawai and K. Kobayashi, Analysis and Design of a Tactile Sensor Detecting Strain Distribution inside an Elastic Finger, Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pp. 16581663, 1998. [16] T. Maeno, S. Hiromitsu and T. Kawai, Control of Grasping Force by Detecting Stick/Slip Distribution at the Curved Surface of Elastic Finger, Proc. IEEE Int. Conf. Robotics and Automation , pp. 38963901, 2000. [17] K.L. Johnson, Contact Mechanics, Cambridge University Press, 1985. [18] R.D. Mindlin, Compliance of elastic bodies in contact, Trans. ASME, J. Applied Mech., Vol.16 of E, pp.259268, 1949. [19] N. Xydas and I. Kao, Modeling of Contact Mechanics and Friction Limit Surface for Soft Fingers in Robotics with Experimental Results, Int. J. Robotics Research, Vol. 18, No. 8, pp.941950, 1999. [20] R. S. Johansson and G. Westling, Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects, Exp. Brain Res., Vol.56, pp.550564, 1987. [21] Y. Kurita, A. Ikeda, J. Ueda, Y. Matsumoto and T. Ogasawara, A Novel Pointing Device Utilizing the Deformation of the Fingertip, Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pp.1318, 2003. [22] Y. Kurita, J. Ueda, Y. Matsumoto, T. Ogasawara, CPG-Based Manipulation : Generation of Rhythmic Finger Gaits from Human Observation, Proc. IEEE Int. Conf. Robotics and Automation, pp.12041209, 2004. [23] M. Kondo, J. Ueda, Y. Matsumoto, T. Ogasawara, Perception of Human Manipulation Based on Contact State Transition, Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pp.100105, 2004.

Fig. 15. Motion Transfer to the NAIST Hand

2348