Gesture controlled robot

Introduction:-

In today’s age, the robotic industry has been developing many new trends to increase the efficiency,

accessibility and accuracy of the systems. Basic tasks could be jobs that are harmful to the human,

repetitive jobs that are boring, stressful etc. Though robots can be a replacement to humans, they still

need to be controlled by humans itself. Robots can be wired or wireless, both having a controller device.

Both have pros and cons associated with them. Beyond controlling the robotic system through physical

devices, recent method of gesture control has become very popular. The main purpose of using gestures

is that it provides a more natural way of controlling and provides a rich and intuitive form of interaction

with the robotic system. This mainly involves Image Processing and Machine Learning for the system or

application development. Beyond this, it also requires some kind of hardware for interfacing with the

system for gesture control. There are some systems that have been developed in the same field using

various techniques.

Programming and control an industrial robot through the use of the robot teach pendant is still a

tedious and time-consuming task that requires technical expertise. Therefore, new and more

intuitive ways for robot programming and control are required. The goal is to develop

methodologies that help users to control and program a robot, with a high-level of abstraction

from the robot specific language. Making a robotic demonstration in terms of high-level

behaviors (using gestures, speech, manual/human guidance, from visual observation of human

performance, etc.), the user can demonstrate to the robot

what it should do.In the robotics field, several research efforts have been directed towards

recognizing human gestures, recurring to vision-based systems motion capture sensors or using

finger gesture recognition systems based on active tracking mechanisms . Accelerometer-based

gesture recognition has become increasingly popular over the last decade. The low-moderate cost

and relative small size of the accelerometers make it an effective tool to detect and recognize

human body gestures. Several studies have been conducted on the recognition of gestures from

acceleration data. However, the specific characteristics of an industrial environment (colors, non-

controlled sources of light, infrared radiation, etc.), the safety and reliability requirements, and

the high price of some equipment has hampered the deployment of such systems in industry.

Given the above, the teach pendant continues to be the common robot input device that gives

access to all functionalities provided by the robot (jog the manipulator, produce and edit

programs, etc.). In the last few years the robot manufacturers have made great efforts to make

user friendly teach pendants, implementing intuitive user interfaces such as icon-based

programming color touch screens, a 3D joystick (ABB Robotics), a 6D mouse (KUKA Robot

Group) or developing a wireless teach pendant (COMAU Robotics). Nevertheless, it remains

difficult and tedious to operate with a robot teach pendant, especially for non-expert users. In this

project is proposed an accelerometer-based gesture recognition system to control an robot in a

naturalway. A 3 axis accelerometer are attached to the human arm to detect the human gestures.
Objective:-

 The main objective of this project is to develop user friendly robotic vehicle which can be

controlled by hand gestures instead of buttons or teach pendant.

 To make this vehicle wireless which add the advantage of better control and portability

and accuracy.

 To add a wireless camera and a metal detector module, this makes this robot versatile

which can be used in various applications.

Research methodology:-

A Gesture Controlled robot is a kind of robot which can be controlled by hand gestures not by

button system. This robot consists of accelerometer mounted on operator’s hand. Signals from

accelerometer are processed by microcontroller & sent to transmitter mounted on hand itself.

Robotic vehicle consists of receiver, Microcontroller & motor controller. Receiver receives

signal from transmitter & these signals are sent to microcontroller & micro controller gives

specific commands to motor controller based on the signals received from receiver. Detailed

structure of the project is as given below.

Whole system is divided into following subdivisions.

 Transmitter module

1) Accelerometer

2) Microcontroller

3) Transmitter

 Receiver module

4) Receiver

5) Microcontroller

6) Motor controller

7) Robotic vehicle

8) Wireless Camera and Metal detector module.

This project is further broadly divided into two modules:

Transmitter module:

The transmitter module is an assembly of different components which are mounted on the hand

of the human. These components are collectively responsible for detecting the human gestures,

converting them into equivalent electronic signal and sending it wirelessly to the receiver module

where it is processed. The transmitter module mainly consists of following components.

Accelerometer: An Accelerometer is a kind of sensor which gives an analog data while moving

in X,Y,Z direction or may be X,Y direction only depends on the type of the sensor. This sensor is

mounted on the hand.This sensor is responsible for detecting the gestures of the hand. When

there is change in gesture of the hand the accelerometer outputs the particular value to the analog

pins. The detected gestures are sent as respective analog values to the microcontroller.

Microcontroller: The microcontroller is responsible for analyzing and processing the data. It

consists of a on chip computer which has the capacity to process the data. It receives analog

signals from accelerometer Converts into digital form & sends to the transmitter device.

Transmitter: The transmitter is advice which is responsible for sending the Radio signals

wirelessly. When the microcontroller gets the analog data from the accelerometer it converts it

into digital and send the respective data to the transmitter. The transmitterdevice receives signals

from microcontroller & sends RF signals to the receiver mounted on robotic vehicle.

This total transmitter module assembly is mounted on operator’s hand.

Receiver module:
The receiver module is assembly of different components which are mounted on the robot to be

controlled. This consists of different components which bring about the entire gesture control

motion of the robot. The receiver module consists of

Receiver: Similar to transmitter the receiver module receives the RF signals from the

transmitter. It decodes the respective signals by sending the signals to the microcontroller. The

microcontroller then analyses the respective signals and commands the robot accordingly.

Microcontroller Receives signals from Receiver decodes them & gives appropriate commands

to motor controller. It analyses the respective signals and the decides whether the robot should

move front, back, left or right. Then the motor controller is signaled which helps the robot to

navigate.

Motor controller Operates motor as per the commands received from microcontroller. It

consists of H-bridge IC which can be used for directional control of the motors.

Wireless Camera and metal detector module:The wireless camera is mounted on the robot

which helps in audio video wireless transmission. The output of the camera is directly fed to the

computer, which can be recorded if necessary. This makes this robot more perfect for multiple

applications. The robot also includes the metal detector module mounted on it which can be used

for detecting metal. When there is the presence of metal the metal detector module buzzers up

lighting an led. Thus this robot can also be used for detecting metal.

Receiver module assembly is mounted on Robotic vehicle

Robotic vehicle will operate as per hand gesture as follows

Hand gesture Robot movement

Forward

Reverse

Right turn

Left turn
 Working of Gesture controlled robot
The receiver receives the signal and
decodes it into a command
A sensor mounted on hand
detects the hand gestures

The microcontroller board

The microcontroller board converts the
converts the detected signal into
detected signal into equivalent
equivalent electronic signal
electronic signal and commands the
robot

The robots moves as per the

The Rf module sends the signal instructions from
wirelessly to the receiver. microcontroller.
Applications:

1. This gesture controller robot can be used in high risk areas such as nuclear power plants where

the human intervention is not possible due to radio hazards.

2. This principle can be used to prepare the friendly vehicle for physically handicapped where the

person is partially paralyzed and cannot operated the wheel chair on his own.

3. The robots can be used to replaced conventional teach pendant based robots which are

very difficult to operate for non-experienced people

4. The robot can be used for video surveillance and metal detection using he wireless

camera and metal detector mounted on the robot.

5. With the wireless camera mounted the robot can also be controlled out of the line of sight

of the controller which can be used to replace conventional remote controlled robots

which demand line of sight.

 References

1) Theory of machines by R.S.Khurmi.

2) Robots, androids, and animatrons by John Iovine.

3) Arduino Robotics by John David Warren.

4) Micro mechanical Transducers by M.H.Bao.

5) Atmel AVR Microcontroller Primer by Steven F. Barrett, Daniel J. Pack

6) Programming and Customizing the AVR microcontroller by DhananjayGadre

7) The robotics Primer by Maja.J .Mataric.