IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. BME-33, NO.

10, OCTOBER 1986

985

(a)
L4-

(b)
12~~~~~~~~~~4-L

10

-50

50mm

Fig. 4. Averaged waveform over 1000 responses (a) and corresponding matched filter output (b).

cord. In all of the above examples the latency was determined after the averaged response was passed through the matched filter and in each case the latency measured is correct. Fig. 4 shows an example of an averaged signal and corresponding matched filter output. The measurement bandwidth was 1 kHz for which the matched filter gives an improvement in SNR of 2.4 times over ensemble averaging alone [8]. Hence, for a signal averaged over 1024 responses, although still noisy when seen on the CRT, the peak and latency are detected accurately since the effective number of responses averaged following matched filtering is approximately 2500, i.e., 1024 x 2.4. As seen in Fig. 3, for levels T8-T6 and below, the noise is nearly averaged out for 1024-2048 averages. Therefore, with the matched filter an average over 512-1024 would result in an SNR sufficient for determining latency and amplitude parameters accurately. Hence, the objective of reducing processing time for a given measurement accuracy is achieved.
CONCLUSIONS evoked potentials are obtainable from a Spinal somatosensory skin-surface recording site, but the low SNR of these signals necessitates the use of signal processing techniques to reliably detect the signals and retrieve useful information. Ensemble averaging of the recorded responses followed by matching filtering provides optimal SNR improvement for signal detection. An independent system for collection and processing of SEP's was realized. It implements the signal processing regimen and raises an alarm condition if an unacceptable change in latency, amplitude, or waveshape is detected. The system was found to perform well in experimental trials, and the main objective of reducing processing time for a given detection performance has been successfully achieved. It remains to test the monitor in a clinical environment to determine its performance under operating room conditions and its reliability in detecting significant changes in recorded SEP's. ACKNOWLEDGMENT The collaboration of E. Mprin, whose related research provided the data files for system evaluation, is appreciated. REFERENCES
[1] K. H. Chiappa, Evoked Potentials in Clinical Medicine. New York: Wiley, 1983. [2] S. J. Jones, M. A. Edgar, and A. 0. Ransford, "Sensory nerve conduction in the human spinal cord: Epidural recordings made during scoliosis surgery," J. Neurol., Neurosurg. Psych., vol. 45, p. 446, 1982. [3] G. L. Engler, N. I. Spielholz, W. N. Bernhard, F. Danziger, H. Merkin, and T. Wolff, "Somatosensory evoked potentials during Harrington instrumentation for scoliosis," J. Bone Joint Surg., vol. 60A, no. 4, June 1978. [4] S. J. Jones, M. A. Edgar, A. 0. Ransford, and N. P. Thomas, "A system for the electrophysiological monitoring of the spinal cord during operations for scoliosis," J. Bone Joint Surg., vol. 65-B, no. 2, pp. 134-139, Mar. 1983. [5] R. Q. Cracco, "Spinal evoked response: Peripheral nerve stimulation in man," Electroenceph. Clin. Neurophysiol., vol. 35, pp. 379-386,

tion," in Proc. 2nd Int. Conf Rehab. Eng., Ottawa, Ont., Canada, June 1984, pp. 83-86. [7] J. E. Hall, C. R. Levine, and K. G. Sudhir, "Intraoperative awakening to monitor spinal cord function during Harrington instrumentation and spine fusion," J. Bone Joint Surg., vol. 60-A, no. 4, pp. 536-553, June 1978. [8] V. S. Jasrotia and P. A. Parker, "Matched filters in nerve conduction velocity estimation," IEEE Trans. Biomed. Eng., vol. BME-30, pp. 1-9, Jan. 1983. [9] R. Gopalan, P. A. Parker, and R. N. Scott, "Microprocessor based on-line system for monitoring spinal cord integrity," in Proc. ISMM Int. Symp. Mini and Microcomput. Appl., San Francisco, CA, June 1984, pp. 6-10. [10] J. P. C. deWeerd, "Facts and fancies about a posteriori Wiener filtering," IEEE Trans. Biomed. Eng., vol. BME-28, no. 3, pp. 252257, 1981. [11] E. Carlton and S. Katz, "Is Wiener filtering an effective method of improving evoked potential estimation," IEEE Trans. Biomed. Eng., vol. BME-27, no. 4, pp. 187-192, 1980. [12] R. E. Kearney, "Evaluation of the Wiener filter applied to evoked EMG potentials," Electroenceph. Clin. Neurophysiol., vol. 38, pp. 533-534; 1975. [13] D. J. Doyle, "Some comments on the use of Wiener filtering for the estimation of evoked potentials," Electroenceph. Clin. Neurophysiol., vol. 38, pp. 533-534, 1975. [14] J. P. C. deWeerd, "A posteriori time-varying filtering of averaged evoked potentials," Biol. Cybern., vol. 41, p. 211-222, 1981. [15] Woody, "Characterization of an adaptive filter for the analysis of variable latency neurolelectric signals," Med. Biol. Eng., vol. 6, no. 6, pp. 539-553, 1967. [16] J. P. C. deWeerd, "Time-varying filtering (TVFth): What it does and how it works," Nicolet Potentials, pp. 7-8 and 28, Spring 1984. [17] R. E. Ziemer and W. H. Tranter, Principles of Communications-Systems, Modulation and Noise. Boston, MA: Houghton Mifflin, 1976. [18] E. Morn, "Spinal evoked potential monitoring," M.Sc.E. thesis, Univ. New Brunswick, Fredericton, N.B., Canada, Dec. 1984.

A Prototype Portable Electronic Speaking Aid for the Nonvocal Handicapped

MICHAEL J. WALSH AND L. C. WESTPHAL

Abstract-A new type of speaking aid for the vocally handicapped is described. The portable device has as key elements a speech synthesizer, a hand-worn touch-sensitive command device, and appropriate interfacing built around a microprocessor. The user, a nonvocal person with reasonable control of at least one hand, uses an augmented variation of finger spelling to command the synthesizer to output phonemes. A prototype is described and early testing is discussed.

I. INTRODUCTION A number of individuals, for reasons varying from congenital defects through laryngeal cancers to accidents, are unable to vocalize. This can and often does severely restrict their interactions with other people, simply because of the difficulty of communicatManuscript received December 18, 1985. The authors are with the Department of Electrical Engineering, University of Queensland, St. Lucia, Queensland, Australia 4067. IEEE Log Number 8609981.

1978. [6] M. R. Nuwer, "Evoked potential monitoring of spinal cord func-

0018-9294/86/1000-0985$01.00 1986 IEEE

986

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. BME-33, NO. 10, OCTOBER 1986

ing with them. This situation has lead to the marketing of a range of aids [1]: "artificial larynxes," some of them mechanical in nature, are sometimes useful; several printer-type devices (e.g., Canon Communicator) provide high-technology versions of the pad and pencil for some of the afflicted; speech synthesizers controlled by keyboards or computers help others, especially those with little need to be mobile but need or desire to send audible messages. The electronic synthesizer approach appears to be a clear favorite in the popularity stakes, if one may judge from the number of "home-brew" systems which are featured in the media. Certainly, a competent electrical engineering student can quickly assemble a system consisting of a synthesizer subsystem such as the Votrax Type'N' Talk plus an inexpensive home computer (e.g., [2]); such talking terminals have been reported for several years [3]. In them, Fig. 1. System configuration for speaking aid. a simple program allows the user to either type words to be spoken, type phonemes to be sounded, or select from a menu of messages to be spoken. The last seems especially prominent for use when the user is profoundly physically handicapped by, for example, FINGER CONTACTS accident or cerebral palsy; a simple switch activated in one of several ways (limb movement, tongue, breath, etc.) can be used to control the menus and hence the synthesizer [4]. The popularity of devices with synthesizers combined with the variety of one-of systems of this type leads us to consider the apparent need for a rethinking of the design of speech aids. It appeared that there is a need for speaking aid which, in addition to the usual engineering criteria of robustness and low cost, would feature flexibility and portability. The flexibility could be provided by making the speech phoneme based and having the user select WRIST CONTACT the phonemes directly, rather than via the intermediary at a textFig. 2. Details of prototype handpiece. to-speech program, while portability can be obtained by using battery power and low-power CMOS circuitry. An important consideration was that the device be perceived, after a training period, as in real high easy to use and without a need to memorize spelling tricks or look quality time. One of our software challenges is to develop phofor all speech, including some coarticulation of the at a prompter. The resulting system is unique, although its only unusual indi- nemes, but even rough 60 allophone speech is very intelligible at vidual component other than its software is the hand-wom inter- this stage. The circuitry for amplification and the loudspeaker are face, and patents are pending for it. Its place in the array of aid conventional. requirements for generating commands to the synThe speech devices is perhaps indicated by considering approximations which could be configured from commerical devices if portability were thesizer were estimated as reasonable for manual use and have so so. To estimates, the synthesis apsacrificed: we might suggest a MicroWriter one-hand keyboard far proved to be manual summarize the about choices from 60 possibilities, proach requires plus Votrax Type 'N' Talk, with the keyboard used to generate about 26, chosen at a nominal 10 or so phonemes/allophonesi.e., per phoneme commands as one approximation. The Canon Commu- second, the approximate ratio of speech phoneme production. This nicator is portable but nonvocal and keyboard oriented. Locus ten per second rate of hand position adjustment is roughly the charSystem Engineering, Melboume, Australia, has available a system typist and using a special keyboard to command phonemes; it is portable, but acter-typing rate of a competent notes uponis well below the peak rate at which musicians can play their by its keyboard-oriented nature is very slow for conversational use. two factors of number of phonemes to be available instruments. The and rate at which The remainder of this paper describes our prototype system and they should be commanded had several implications for the intersome preliminary testing of it. The system itself is described in Section II. Section III presents a description of how the device is face design. used. B. User Interface An important goal of the project was to develop a convenient II. SYSTEM OVERVIEW user interface. Immediately eliminated were keyboards, because of Our proposed system may be seen as having three modules, as their need for two hand or hand-plus-keyboard-support operation, in Fig. 1: a speech synthesizer with speaker, a user interface, and and word pronouncers, for which the requirements that inputs be support circuitry including power supply and processor. The con- spelled words were judged inappropriate for children and inconcept is simple and direct. It uses commercial hardware except for venient in conversations. We in fact briefly considered three opthe custom user interface; software is unique to the concept. These tions: joystick, EMG monitoring, and finger switches. The second, subsystems are described in the following sections. which may have long-term potential for the profoundly handicapped, was rejected after preliminary testing as requiring longA. Speech Synthesis term development, as were variations such as eye movement senOur early test bed system had a first generation synthesis speech sors. Preliminary tests showed that the third option was preferred chip and thus had the capability of producing 60 different allo- to the first by several persons, and it is also more amenable to onephones. This was adequate for early text-to-speech converters and hand use by a standing mobile user. intelligible but very monotone and synthetic-sounding for our purFollowing the above reasoning and testing, a strap-on handpiece pose. The prototype contains a third generation Silicon Systems was designed, constructed, and tested (Fig. 2). Contacts are SS1263 chip, which is capable of very high quality speech. All of touched by the thumb tip, the wrist, and by the finger bases. The its flexibility is utilized by the aid when producing prestored latter, somewhat unusual, feature derives from experience with a phrases, but to keep the user burden to a reasonable level, only 60 test bed system with a finger-tip contact system. It has three moallophones plus volume level and pitch are selectable by the user tivations: the fingertips are not blocked by the handpiece from doing

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. BME-33, NO. 10, OCTOBER 1986

987

Fig. 4. Prototype speaking aid. Handpiece is on user's right hand. Box contains all other components. User's left hand holds one of the prototype's circuit boards.

FROM INTERFACE

Fig. 3. Hardware block diagram.

other tasks; the folding and unfolding of the fingers is closer to a sign language than fingertip movements are, and thus the handpiece may be easier to use; contacting the switches requires quite positive action by the user, thus avoiding errors due to inadvertent signaling. An improved version of the prototype handpiece, which is both easier to customize for the user and more visually appealing, is now under development. Use of the handpiece is straightforward. There is one contact for each finger, three for the thumb, and one for the wrist. The wrist contact is used for inflection control, and the combinations of thumb contacts and contacts due to finger deflections select the operating mode of the system-e.g., output of allophones, output of prestored phrases, input of new phrases to storage-and the particular E. Packaging allophone, phrase, etc., to be spoken. The combinations are softCurrently the device, including its 10 cm speaker, NiCad batware selectable, but in the current version the basic mode selection teries, and two circuit boards, is packaged in an aluminum box is performed by the thumb and the parameter selection is made by measuring 6 x 11 x 30 cm and weighs about 1300 g. The battery the fingers together with the thumb. In the allophone mode, the charger is an external voltage supply. A wire leads from the box thumb parameters tend to select between vowels and consonants to the handpiece. The device is currently worn with a shoulder and the finger configurations are loosely based upon a simple sign strap, rather like a camera. language. Users grasp the concept of the handpiece quickly, alFinal packaging is still to be worked out as various tradeoffs of though some practice is needed to use the device. speaker size, number of batteries, and circuit board size and layout are tested. A preliminary indication is that the speaker should be C. Interface Circuitry mounted near the user's throat rather than on the hip, and this is a The interface circuitry between the handpiece and the synthe- concept to be tested. sizer is shown in block diagram form in Fig. 3. It is built around a 6303 microprocessor, 16 kbytes of random access read-write III. METHOD OF USE memory (RAM), and 16 kbytes of read-only memory (ROM). Analog signal conditioners for the handpiece, battery charger support Use of the device is perhaps best understood by referring first to circuitry, and a real-time clock are also included. Standard design Fig. 4, which shows the prototype being worn by a user, and then practices were followed. In the prototype, the circuitry is mounted to Fig. 2, showing the handpiece. on a pair of 10 by 16 cm boards and connections are via the Speed The device operates in several modes, some of which are: paWire system. Power is supplied by NiCad batteries, which provide rameter adjust for volume control, input for storage of phoneme for up to 8 h of continuous use or several days of intermittent use. strings are phrases, output of stored phrases, and output of pho-

D. Software The software for the system falls into two categories: system type software and user-generated phrases. The former is undergoing development, but carries out the following basic functions: 1) system control 2) battery monitoring 3) scanning and interpretation of keyboard inputs 4) 1/0 functions. This software is contained in the system ROM and is being developed on DEC VAX/750 computer using a cross assembler. Testing is done on the prototype by down-loading software into RAM and debugging before storage in ROM; The user will be able to create his own phrases for the device and enter them using the handpiece. Such phrases and other system data are stored in the RAM, which is powered at all times to protect the data from destruction. All control ofthe aid, e.g., volume control, on/off, speech entry, and control functions is via the handpiece.

988

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. BME-33, NO. 10, OCTOBER 1986

neme strings. Since it is the last of these which is unique, we describe it briefly. Suppose the user wishes to say "hello." With some experience with the device, he (the device is incapable of generating female voice pitches at present) will know that he needs the four phonemes "h," "'e," "1," and "oh." He will then successively enter commands using the finger positions as follows: 1) 111 1111 (to wake up the system) 2) 000 1010- (first phoneme "h") 3) 010 1000 (second phenome "e") 4) 000 0001 (third phoneme "1") 5) 011 0010 (fourth phoneme "oh") 6) 111 1011 (to turn off the system). The finger positions are given such that each digit indicates whether a particular contact is to be touched 1 or not 0, with the most significant bit of the position indicating the state of contact 6 and the least significant bit indicating the state of contact 0, e.g., 000 0001 means that only contact 0 is to be touched. Our experience has been that the speed of keying is not a major problem, and the timing in the device allows for some slowness of input. Although the keying may be changed by changing the system software, the above scheme is not arbitrary. Our present approach is to assign the most frequently used phonemes to the simplest finger positions.
IV. TESTING AND DEMONSTRATIONS A series of tests involving handicapped people will begin soon. To date, we have made trials with healthy people, who adjust quickly to the concept and sound of the device and after a few minutes can learn to command in-built phrases to be pronounced. One tester with about 20 h of experience can say any phrase at a slow rate and ca,n perfect a phrase in 2 min; this person observes that, although the device has its failings, it is a considerable improvement over the alternatives of note writing and sign language. A user of the early test bed system could generate 20 phrases after 2 h of practice. Based upon this informal testing, which shows that a few hours' practice yields some competence with the device, we estimate that several weeks of use would be needed to acquire pro-

The device has also been demonstrated to a number of rehabilitation experts, who have been encouraging about its potential. A number of engineers who have seen it have also suggested that its underlying concept, i.e., a portable microprocessor with hand-worn user input device and ROM-resident program generated on a host computer, is applicable to a number of other uses, some of which will be investigated. In the immediate future, work will concentrate on systematic testing of the prototype with handicapped people. The tests will establish 1) acceptability and utility of the concept 2) methods of teaching its use 3) need for changes. Concurrently, we will be investigating other uses for the devices as a general interface. We expect that, with little modification other than to system software, it could be used for foreign languages, wheelchair control, and computer interfacing, among several possibilities.
V. SUMMARY

This paper has described a unique approach, now in engineering prototype stages, to providing a speaking aid for people with some physical capability and a sufficient intellectual capacity. We have mentioned other speaking aids to emphasize the differences, discussed the device and its operation, and mentioned early testing. System refinement is continuing , and field testing will begin soon.
REFERENCES
[1] G. C. Vanderheiden and K. Grilley, Non-Vocal Communication Techniques and Aids for the Severely Physically Handicapped. Baltimore, MD: University Park Press, 1976. [2] D. R. Willett, "SPEECH-scrolling plus encoding communications aid," B. Eng. thesis, Univ. Queensland, St. Lucia, Queensland, Australia, 1985. [3] W. Myers, "Personal computers aid the handicapped," IEEE Micro, vol. 2, pp. 26-40, Feb. 1982. [4] A. Thomas, "Communication devices for the nonvocal disabled," IEEE Computer, vol. 14, pp. 25-30, Jan. 1981.

ficiency.