Sign Language Translator

Submitted as a term project (Design Phase) for DS302

Project SupervisorDr. Amarnath M. Mr. K.K. Balakrishnan

Submitted byNeelesh Srivastava (2011088) P.B Shruti (2011096) Ritesh Kashyap (2011128) Vishakha Singh (2011176) Hemant Singh Rathore (2011249)

PDPM Indian Institute of Information Technology, Jabalpur, MP

ACKNOWLEDGEMENt
We owe a debt of gratitude to many people for their benevolence and support. To begin with we would like to thank our DS 302 course instructors: Dr. Amarnath M. and Mr. K.K. Balakrishnan for their constant guidance. We are also indebted to Mr. Prasad R. Padase , Mr. Vimal Abinash and Mr. Vipin Yadav for guiding us during the whole design project and for providing us with their expert and valuable advice. Lastly, I would like to thank the design fraternity of IIITDM-J.

- Neelesh Srivastava P.B Shruti Ritesh Kashyap Vishakha Singh Hemant Singh Rathore

AbstracT:
The prime focus of our project is to design a SIGN LANGUAGE TRANSLATOR (SLT) to reduce the communication barrier between Hard of Hearing and Speaking people (HI) and the Normal people. A child with early onset of hearing impairment does not recognise sounds and hence it becomes very difficult for him to reproduce the sounds using voice. Since such a child is devoid of common expression medium i.e. speech it becomes very difficult for him to communicate with others. As he grows up, he tries to learn how to communicate by learning some conventional Sign Language. But that too becomes a waste as normal people generally are not aware of how to use of Sign Language. So, ultimately, such a person has to adopt some tedious way of communication such as Pen and Paper, which he is not comfortable with. Sign language translator can reduce the problems of this community by a great extent as they will be able to communicate their thoughts using a language they are comfortable in. Also, it will be translated to a language understandable by those who do not know the sign language yet wish to communicate with these people.

Table of Contents

Acknowledgement Abstract List of Contents List of figures List of Tables 1. Introduction 1.1 Current situation 1.2 Critical issues 2. Problem Definition 2.1 Specifications and constraints 2.2 Area of research 2.3 Methodology 3. Concept Generation 3.1 SLT in LCD mode 3.2 SLT in speaker mode 3.3 Frequency based vibrating alarm 3.4 Mobile Sign Language Translator 4. Concept evaluation and selection

2 3 4 5 6 7 10 10 11 11 11 12 13 13 14 15 17 18

5. Embodiment design 20 5.1 Product architecture 20 5.1.1 Modular architecture 21 5.1.2 Integral Architecture 23 5.1.3 Assembly of components and devices 25 5.1.4 Different levels of assembly 25 5.2 Configuration design 27

5.3 Parametric Design 5.3.1 CAD Model 5.3.2 PCB Design 5.3.3 Cost Analysis 6. Conclusion 7. Future Scope

31 31 38 38 39 39

List of figures
Fig 3.1: SLT in LCD Mode Fig 3.2: SLT in Speaker Mode FIG 3.3: Frequency based vibrator FIG 3.4: Mobile SLT Fig 4.1: Final Concept Fig 5.1: Estimated list of items to be arranged Fig 5.2: Sketch of camera Fig 5.3: Sketch of LCD Fig 5.4: Sketch of speaker Fig 5.5: Sketch of stand Fig 5.6: Sketch of PCB Fig 5.7: Orientation of PCB Fig 5.8: Assembling in suitcase Fig 5.9: More compact assembling 26 Fig 5.10: Signal flow Fig 5.11: Assembling of component (top view) Fig 5.12: Side view Fig 5.13:Case Fig 5.14: Drawer Fig 5.15: Lid Fig 5.16: Final casing Fig 5.17 Compact design Fig 5.18: Dimension of the case Fig 5.19: Speaker and camera Fig 5.20: PCB Circuit

LIST OF TABLES:
Table 3.1: Concept selection and evaluation Table 5.1: Standard Devices Used Table 5.2: Cost Analysis Table 5.2: Standard components used

1.Introduction
According to a survey conducted by ASHA School run by the Army Wives Welfare Association, detection of hearing loss at birth or early childhood is very difficult. Even after the child is diagnosed with a problem of hearing, he receives hearing aid after a span of few years. This makes the child lose his golden period of learning. Also, a child with early onset of hearing impairment does not recognise sounds and hence it becomes very difficult for him to reproduce the sounds using voice. Since such a child is devoid of common expression medium i.e. speech it becomes very difficult for him to communicate with others. Due to this communication barrier, these children face problems in all aspects of life. In day to day activities Being in a group and trying to express something to people around, is very difficult. Education: Traditional school environments are difficult for a deaf person to navigate without the aid of interpreters. Emergencies: Fire alarms and other emergency alarms are of no help to a hearing impaired person. Driving is a major problem for those who cannot hear. They have to be even more vigilant on streets.

The problems faced in communicating directly using pen & paper, finger spelling and signing Pen and paper - The disadvantage of this is that these people should always carry a notepad to the workplace. Also it becomes very cumbersome to write everything. Finger spelling - The disadvantage of this method is that spelling letters of each and every word is time consuming and no one usually gives so much of time and attention while at work or school. Signing is more complex and it takes time and most people have to take courses, before they are comfortable in using this second language. The objective of this work is to design a low cost, portable mechanism which can help the HI people to communicate with others. Design focuses on attributes like easy operability, portability. The design critically examines and considers the following factors-size of the product, comfort, ease of usage, ease of fabrication, cost effectiveness and aesthetics. Designs are created for a purpose to fulfill the customer needs and aspirations. Design intent is the arrangements of features and dimensions of a design. Design intent governs that relationship between features in a part and parts in assemblies. The intent of each component of a design is to work as a solution to the design problem. Designer should create design representation for himself so that he/she can not deviate from the objective. Design representation is something designer produces and used for communication with himself, with client and makers. The design representation reminds the designer of his intention so that the designer plays with in the design space. The representations are used to answer a variety of hypothetical questions posed by the clients and the

manufacturers. Design representation will act as a good communicator between the designer and the manufacturer. It has all the required information that manufacturer wants from the designer. A document contains all required information of the project which is sketches (all types) & drawings.

1.1 Current situation

There are few designs which work on the concept of flex sensors and accelerometers but the cost of which is so high that it is out of scope of ordinary people. It is not popular among the hard of hearing and speaking society due to its cost and efficiency.

1.2 Critical Issues

Some of the critical issues attached with the design of a Sign Language Translator are: Remove the linguistic barrier between people unaware of sign languages and the person who has command on it. It is preferred that the device is lightweight and sufficiently portable. The database containing signs and their meaning should be easily modifiable. It should be convenient and easy to use. It should be efficient.

2. Problem Definition
This section presents the proper problem definition along with the specifications and constraints. Need statement: The need statement is generated keeping the following requirements in mind: Simplifying communication between people hard of hearing and speaking and person with no disability. Enforcing a sense of confidence and equality as the users are able to express and present their views. Reducing complexity of daily life activities. Statement:- To design a device which can translate sign language to text and speech.

2.1 Specification and Constraints

User friendly with regard to operations, management and also replacement of parts Portable and compact design Elegant design for easy usage Proper ventilation for electronic components

2.2 Area of Research

The following areas have been considered for designing the Sign language translator mechanism: Need specification of the hard of hearing and speaking community.

 Embodiment design with view of it being compact, portable and user friendly. MATLAB and its toolboxes. Algorithms for image compression, pattern matching in images, image and video processing. Circuits and materials involved.

2.3 Methodology Research and data collection

Research and data collection phase was carried out during a visit to Viklang Seva Bharti Kendra, Chetna, Jabalpur. There we interacted with our Primary users (The hard of hearing and speaking community) and also secondary users (teachers and parents) in order to collect data. Need identification In this phase, all the data and the interviews taken was assembled and need statement was formulated. Exploration of current scenario Using internet and the help of faculty and research scholars of IIITDMJ we explored the current scenario with respect to technological advances in this field.

3. CONCEPT GENERATION :
The following are the concepts of mechanism to achieve aesthetically pleasing form and fulfilling the product's objectives. The concepts are evolved after intense brainstorming, analyzing different hand sketching and virtual modelling.

3.1: SLT in LCD mode

This converts sign language into text, reducing the communication gap between HI and a normal person. Uses camera for capturing sign language after which it processes and displays through LCD or SPEAKER. Features High efficiency Low portability High accuracy Moderate cost Clear output

Fig: 3.1 SLT in LCD Mode

3.2: SLT in Speaker mode

This converts sign language into speech eliminating the need to read through LCD. Uses camera for capturing sign language then processes it using image processing technique and uses speaker for output. Features High Efficiency Limited portability. Eliminates the need to read

Fig3.2: SLT in Speaker Mode

3.3: Frequency based vibrating alarm

This device will vibrate for any emergency signal or unusual sound. Features Very handy Can be used as a wrist band Low complexity Low maintenance. Especially useful in case of emergency.

FIG 3.3: Frequency based vibrator

3.4:-Mobile Sign Language Translator

This uses the processor of the phone instead of that of the laptop. Small Portable device converting sign language into text. light and handy, Uses a camera to capture images and has also a speaker for output. Low efficiency Low speed. Low accuracy

FIG 3.4: Mobile SLT

4. CONCEPT EVALUATION & SELECTION :

Final concept is developed from the concepts stated above and comparison between them is done on the basis of various factors which below :-

concepts

cost

efficiency

complexity

speed

portability

Concept1
concept2 concept3 concept4

medium
medium low high

medium
medium medium low

high
high low high

high
high high low

medium
medium high high

Table 3.1: Concept selection and evaluation

After analyzing all the factors we realize that concept 1 and 2 can be best incorporated in one design. Thus the final design is a combination of concept 1 and 2. The design is more compact and multifunctional with high speed and efficiency. It can give vocal as well as textual output.

On the basis of the table above the final concept developed is:

Fig 4.1: Final Concept

5.Embodiment Design
We have now brought the engineering design process to the point where a set of concepts has been generated and evaluated to produce a single concept or small set of concepts for further development. We have divided the embodiment design in three activities:

5.1 Product Architecture

Product architecture deals with the arrangement of physical components and devices into different groups called modules.

Fig 5.1: Estimated list of items to be arranged

Our product architecture consists of two parts: Modular architecture Integral architecture 5.1.1 Modular architecture Our design consists of three modules. Module 1-camera and stand We are using camera for capturing sign language and hand gestures. The camera will be mounted on a stand to adjust height and orientation of camera.

Fig 5.2: Sketch of camera Module 2-LCD with stand We are using LCD to display the output after processing the acquired image. The LCD is mounted on a stand.

Fig 5.3: Sketch of LCD

 Module 3-speaker Speaker is also used to display the output in the form of speech.

Fig 5.4 Sketch of speaker

Fig 5.5: sketch of stand

5.1.2 Integral architecture The integral part of our design is PCB which communicates with LCD, processor and matlab. PCB consist of one micro controller, rs232 module for USART communication.

Fig5.6 sketch of PCB

Fig5.7 orientation of PCB

5.1.3 Assembling of component and devices Assembling of devices is done in different levels. Levels are increased keeping in mind different criteria such as: compact assembling minimum length of wiring mass distribution 5.1.4 Different levels of assembling Level 1(suitcase type assembling) All components and devices are assembled in a suitcase.

Fig5.8 assembling in suitcase

Level 2(more compact design) This level focuses on more compact form of assembling.

Fig5.9 more compact assembling

5.2 Configuration Design

In configuration design we establish the shape and general dimensions of components and devices. Based on assembling the list of standard devices and components are as follows: DEVICES and shape PCB MODEL DIMENSION (cm) AVR 17X12X2 It consists of 40 pin IC ATmega16 which includes pins for serial communication, and 1 port for LCD. It also includes DB9 for serial to USB conversion. Voltage requirement-9V. SPECIFICATIONS

LCD

20X4 Jumbo LCD

Module size : Backlight 14.6X6.25X2 LED(blue). 16 pin LCD with brightness adjusting option. Voltage requirement-5V. Radius 2.75 Thickness 3.8 Quantum QHMPL Webcam 25 Megapixels, High Resolution Webcam with 6 Light and

Camera

Quantum 495LM webcam

Speaker

TAG DP 100TF 1 channel speaker

11X8X3.5

Battery

Li-Po battery

7X5X3

Microphone, USB 2.0 High Speed Most Advanced Features, LED Brightness Control. Capture Video with Audio or Still pictures, Snap Shot Button, Driverless Can work without loading Drivers built - in lithium battery, 5 watts, 2 - 3 hours play back time. Small and easy to carry, built in amplifier, stereo sound and extra bass. 9V supply.

Table 5.1: Standard Devices Used Components used connecting wires USB port DB9 LEDs Aluminium wires Switches descriptions For connecting devices with PCB For serial to USB conversion For USART transmission For indicating status of devices For flexible stand For on/off

Table 5.2: Standard components used

The most critical items in the module description are the description of the interfaces and the modelling of interactions between neighbouring modules. There are four types of interactions possible between component modules spatial, energy, information, and material. In our design interaction is of information flow. Information flow between modules often takes the form of signals to control the products operation or feedback relative to that operation. Sometimes these signals must branch out to trigger multiple functions simultaneously.

Fig 5.10: Signal flow

The shape and arrangements of components and devices is described in the below figure.

Fig 5.11: Assembling of component (top view)

Fig 5.12: Side view

5.3 Parametric Design

Parametric designing is about determining the exact values, dimensions, or tolerances of the components or component features that are deemed critical-to-quality. This aspect of design is much more analytical than conceptual or configuration design. The objective of parametric design is to set values for the design variables that will produce the best possible design considering both performance and cost.

5.3.1 CAD Model BASE- The base of the casing consist of various slots and spacing for electronic components . ELECTRONIC COMPONENT : Camera Speaker LCD Battery Printed Circuit Board Camera Stand Base :

Speaker Camera

Camera Stand

Fig 5.13:Case

DRAWER : The drawer consists of the spacing for Printed Circuit Board (PCB) and Battery. BATTERY

PCB

Fig 5.14: Drawer LID : The lid or the closing flap has the LCD embedded on its surface which displays text to the front .

LCD

Fig 5.15: Lid

Final Design Of Casing

Fig 5.16: Final casing

Compact Design

Fig 5.17 Compact design FINAL DIMENSION OF THE PRODUCT 25X23X4.5 CM

The design of the case is such that it is easily portable and optimum weight distribution.

Dimensions of case

Fig 5.18: Dimension of the case

Design of Electronic Components

CAMERA

Speaker

Fig 5.19: Speaker and camera

Material analysis:

Hence, the material used for case manufacturing is LEXAN POLYCARBONATE which will cost approx. Rs. 1100/-

5.3.2 PCB Design The circuit consists of a PIC16F877A as the microcontroller which is a 40 pin IC. To this microcontroller a 16X2 LCD (green) is connected along with a serial transmitter, a power supply circuit and an oscillator to regulate the clock pulses of the IC.

Fig 5.20: PCB Circuit 5.3.3 Cost Analysis The cost involved with the standard electronic components and devices have been tabulated below. Sr.no. 1 2 3 4 5 6 Devices/components PCB LCD Camera Speaker Connecting wires Casing material Table 3: Cost Analysis Cost 500/700/400/700/50/1100/-

6. Conclusion
We have designed a sign language translator to cater the basic needs of people hard of hearing and speaking. The device will help greatly in uplifting these people and imparting them with choice and freedom of communication with everyone. The device that we have proposed is robust, efficient and portable.

7. Future Scope
With advancement in technology the size of the design can be further reduced, With advancement in technology its efficiency can be increased.