DIGITAL MEDIA DESIGN

DIRECTED STUDY
PAPER

KAREN KIELY

0643564

NOVEL INPUT DEVICES

 CONTENTS

● INTRODUCTION

● LITERATURE (in order of discussion)

• A Brief History of Human-Computer Interaction Technology

• Of Mice and Menus: Designing the User-Friendly Interface

• Touch Sensing Input Devices

• Teaching Old Mice New Tricks: Innovations in Computer Mouse Design

• Touch&Type: A Novel Pointing Device for Notebook Computers

• Soap: A Pointing Device That Works in Mid-Air

● PERSONAL STATEMENT

● REFERENCE LIST
 INTRODUCTION

From the list of available topics, I chose 'Novel Input Devices'. I chose this because I
felt that I would find the field interesting. We all use input devices every time we access a
computer or other device. We cannot carry out as many activities on a computer without
using input devices like a mouse or a keyboard. Because of this, I decided to focus on the
mouse as an input device. It is the most common and most recognised input device in use
today. Concentrating on the 'novel' aspect of the title, I researched into new improvements
and developments in the mouse as well as generally in human-computer interaction. This
provided the basis of my project from which I could stray into other related topics like
multi-touch.

LITERATURE

A Brief History of Human-Computer Interaction Technology

B.A. Myers

Introduction
This article by Brad Myers is one in which he discusses the numerous interactions,
applications, software tools and up-and-coming areas of human-computer interaction. It stays true
to the title and provides a concise history of HCI under a number of headings.
HCI today is seen everywhere when we use computers and other technologies. Myers
documents a topic which spans a huge area. The article is one which provides the reader with an
abundance of information on the topic of HCI.
The article's purpose is to change people's thinking that important HCI research takes place
at industry level. Myers states many people believe that if HCI research in universities is not
supported, industry will continue in its place. In reality, this is not true.
In my own opinion, I found the article an interesting read which aptly documented the
research carried out over the years in the field of human-computer interaction.

Critique
Myers begins this article by declaring that “all software written today employs user interface
toolkits and interface builders”. He then begins to state that the article will focus on summarising
the important research developments in HCI technology.
I felt that this introduction was exactly what an introduction should be. It was clear, concise
and not technically-written. It didn't contain many technical details and just summarised the purpose
of the article. It was one of the best introductions I came across while reading each paper.
Focusing on the section entitled 'Direct manipulation of graphical objects', I found this
interesting although in the context of my entire research, it provided repetition. Here, Myers simply
discussed the Sketchpad. However, this was only a fraction of the section, and Myers went on to
focus on AMBIT/G and Light Handles. The descriptions of these programs didn't contain much
detail so I was a little in the dark on how they worked etc. A little more attention to detail on Myers'
part wouldn't have done amiss. Although, having said that, I did do some substantial research in the
Sketchpad so that is the reason why I didn't understand much on the other programs.
Myers' discussion of the mouse is not very significant in the context of the whole article. It
is a tiny paragraph containing little information. The most interesting point I found was that the first
commercial mouse appeared in 1981 as part of the Xerox Star. Having read many papers on
Engelbart and others' work on the mouse, I felt as if the mouse would have appeared commercially
earlier.
 Myers continues in the same way into a section on windows, explaining when windows
were developed and when they commercially appeared as well as developments from tiled to
overlapping windows. Continueing in the same vein through the sections on drawing programs, text
editing, spreadsheets, hyper-text and CAD, Myers chronicles the inventions, developments and
commercial appearances of such programs and applications. These sections, including that on
windows, although not essential to my understanding of the history of the mouse, provided some
interesting reading.
Rounding up the article with descriptions of up-and-coming areas, Myers writes about
gesture recognition, three dimensionality and multimedia as well as many other aspects. These were
highly interesting although at times tough to read. I feel this is down to the fact that the programs
and applications discussed were new and unusual. I found it easy to read on topics like the mouse as
it is such an everyday device that I understand whereas this was not so for these topics. Even so,
when I could understand, I found these exciting new prospects for technology in the future and even
in the present.
In the discussion at the end of the article, Myers argues that industry will not be able to
carry on research in the field and research in universities must carry on. He goes on to explain that
“university, corporate and government- supported research continue and be well-supported”. I can
see his point as he explains that without sufficient academic funding, there would be “fewer
doctoral graduates” and in turn, less graduates to work in the top-notch research labs.

Summary
Having read this article and Myers' views on the research in HCI, I feel I know much more
about the problems behind the research process. He makes it clear that people should realise that
research is carried out in university more so than in industry.
Although this article was not very helpful in my research of the mouse, it did provide
interesting reading in the sections regarding different aspects of HCI. Myers', at most times, easy-
to-understand language was the article's high-point. However, in some places, although very
minimal, the article was a bit technically detailed, I still found it interesting and somewhat easy-to-
read.

Of Mice and Menus: Designing the User-Friendly Interface

T. S. Perry, J. Voelcker

Introduction
Perry and Voelcker discuss throughout this article the development of the mouse and other
aspects of HCI like windows and icons. The article contains a detailed section on the mouse and
documents Engelbart's development of the device. The development of the mouse after Engelbart
by companies such as Xerox and Apple is also discussed. Windows, icons and windows are also
discussed in detail.
The main topic of the article is important as windows, icons and menus obviously provide a
huge basis for human computer interaction today. The mouse is an almost essential input device
whose development throughout history is documented in a detailed manner in this paper. It provides
the reader with an insight into the history of things we use everyday.
The article's detailed and interesting analysis of the mouse as an input device as well as
components like windows appealed to me as it was easy-to-read and understand. It contained
information on a very interesting topic and provided an insight into the humble beginnings of many
of the components we take for granted today.
Critique
The writers begin this article by explaining the time frame in which certain technologies
were invented, researched, developed and distributed. Perry and Voelcker provide background
information on Ivan E. Sutherland's Sketchpad program as an introductory section. This provides
some detail on the beginnings of the GUI which I had already read about in Sutherland's thesis.
They go on to highlight the differences between the light pens of the 1960's and mice,
declaring that mice overcome some downfalls associated with the light pen. For example, the
mouse is used on a physical desktop while a light pen is used by the user lifting the arm from the
table. This got tiring after a certain time, so mice were seen as more advantageous.
A list of definitions is included in the first page, containing the meanings of words such as
bitmap, graphical user interface, raster display and clicking. The concepts of some of the first mice
are discussed in detail, including descriptions of the mechanics and technology behind them. In
different sections on buttons and balls, Perry and Voelcker analyse the workings of mice regarding
these aspects. A pictorial flow chart chronicling companies such as Xerox and Apple's research into
mice, windows, icons and menus is provided.
Further discussions of windows, menus and icons are found in the article. The article is
concluded by a paragraph on lawsuits to “establish who owns what”.
The introduction to the article is highly detailed, and almost too much so for an introduction.
An easy-to-read and generalised introduction would have appealed to me more. Having said that,
the information provided is interesting. The writers explain how prototypes incorporating windows,
mice, icons and menus were distributed in the 1970's, going into detail, explaining “more than 1200
Altos were built and tested” by Xerox. Definitive facts like this, in my own opinion, would prove
more useful in another section of the paper. I feel the introduction should have been more
simplistic.
The section of the article entitled 'Mouse Tales' was, I found, a very useful section. First,
highlighting the disadvantages with the light pen, the writers instilled the faith in the reader that the
mouse is a more useful device than it predecessors. Reading further, the writers begin to describe
the mechanics of the first mice. This was, at times, hard-to-read due to some technical terms used
but was not a great problem. The attention-to-detail was definitely an advantage and not a downfall
of the style of writing. The section 'Going On The Ball' further incorporated technical terms which
at times were hard-to-understand.
'Button Wars' was an interesting section, condemning Apple's decision to produce a mouse
with one single button. I found the analysis on user-based experiments on various versions of the
mouse highly engrossing. The contrast put across by the writers between mice with varying
numbers of buttons was interesting. Multiple button mice (generally mice with more than two
buttons) were concluded as being confusing for a user. Despite the fact millions of Apple single
button mice are/were in use, William English, a computer engineer who helped Douglas Engelbart
develop the first mouse, is mentioned as having stated one button mice were 'a bad idea'. This
analysis of buttons and the user opinions of them interested me as it had a human factor in it.
The sections entitled 'Window with view', 'Pop, pull and tear' and 'Iconography' each
documented early research and developments in windows, menus and icons, respectively. The
windows section didn't appeal to me as much as the others. It simply documented how windows
developed over the years, from Sketchpad to overlapping windows. I felt it didn't contain any
exciting or unusual point, unlike in the other sections. 'Pop, pull and tear', I felt, contained some
interesting points. Research in the field of drop-down and single bar menus is discussed. User
experiments showed how users would benefit from having a single bar menu at the top of a window
as opposed to the bottom. This is why tool bar menus are located at the top of a window today. A
supplement to menus, which I found interesting to read about, is key combinations, a “return to the
earliest of user interfaces”. 'Iconography' details the beginnings of the concept of the icon. It
contained an interesting point which stood out for me. On the NeXT computer system, deleted
objects are placed in a Black Hole as opposed to a trash can on a Macintosh. William English
argues that the Black Hole idea is a confusing concept which users who don't read science fiction
would find unusual and hard-to-understand.

Summary
Having read this article, I felt as if it contained useful information on different aspects of
human-computer interaction. Perry and Voelcker focused on well-chosen topics as we all make use
of mice, windows, menus and icons today.
At times, the article was very technically written and in places was hard-to-understand. I had
to re-read at some points. However, the exciting sections in the paper and the points made for great
reading.
I felt the points made on the user experience of different interfaces appealed to me most. I
enjoyed learning about what users found easy-to-use and vice versa. These were some of the most
interesting points I found in the paper.
Overall, it was a well-written papers with an exciting focus, containing information on a
widely-known topic.

Teaching Old Mice New Tricks: Innovations in Computer Mouse Designs

Shumin Zhai & I. Scott MacKenzie

Introduction
This article is a short and simple one where Zhai and MacKenzie discuss the mouse, its
human factors and some new “tricks” that are being incorporated into new models of mice.
The topic of this article, namely the development of the mouse in recent years and in years
to come, is a very interesting and important one. The writers aptly describe the new trends in mouse
design technology. The three “tricks” they document seem like interesting slants on the technology
of the mouse today.
The simplicity of the article and its language help portray the new “tricks” in an appealing
manner. I enjoyed the article as it was easy-to-read and had an exciting content.

Critique
This article is introduced with some fascinating statements which set the tone of the paper.
Zhai and MacKenzie state that the mouse is one of the most frequently handled devices in many
people's lives today. They also use interesting points to put across the message that the mouse is a
large part of modern society, such as one which tells us that the term 'mouse' is now accepted by
reputable dictionaries. They then go on to discuss the early history of the mouse which I had
already researched in a number of articles.
In the section on “Human Factors of The Mouse”, the writers argue that the mouse is
superior to other input devices like touch pads, trackballs and pointing sticks. Dividing this section
into four parts, they discuss in equal measure the 'Easy Control Mode', 'Button Integration',
'Flexibility of Using Different Muscle Groups' and 'Easy Device (Re)acquisition' of the mouse.
These separate sections contain interesting facts on why the mouse is so useful so long after its
invention.
Following a short paragraph on 'Models for Mouse Research and Evaluation', Zhai and
MacKenzie discuss the three new trends for mice: 'The Third Dimension and Six Degrees of
Freedom', 'Making a Mouse Feel Able' and 'Second Stream of Input for Zooming and Scrolling'.
The conclusion ends the article with a statement that it is possible to “teach old mice new tricks”.
Personally, I found the article very intriguing. The style of writing of the entire piece
appealed to me, more so than with any of the other articles I studied. In the section on human
factors, the writers argue that the mouse often outperforms other devices when used in experiments
for pointing purposes. They put across clearly the point that the mouse is an almost indispensable
input device that is so widely used that other devices pale in comparison. They continue to explain
the downfalls of other devices like trackballs. They make clear the point that dragging and dropping
is a difficult procedure with this device but not so with the mouse. This point interested me as it
showed that the mouse is certainly a much superior device than others on the market today, despite
its being on the market for more than twenty years now.
In the section on new tricks of the mouse, the designs interested me. The different ideas
were innovative and seemed exciting. 'Making a Mouse 'Feel Able'' is an interesting paragraph
describing how classes of feel may be incorporated into a mouse. Vibration and 'force feedback' can
be incorporated and the writers describe one such design developed by Immersion Comp. This
section was exciting and although short and not very detailed, still put across the idea in an effective
manner.
The writers then go on to discuss new methods for zooming and scrolling. They argue that,
perhaps, two-handed input would remedy the problem a user would have of having to switch
between scrolling and pointing. They then tell us that of all the new mouse 'tricks' mentioned,
scrolling is probably the most useful. I agreed with this as the paragraph on scrolling was the
longest and contained the most detail, which obviously shows that this is an important 'trick'. As
well as this, I can see the problem with scrolling and pointing more so than the problems put across
in the other paragraphs.
In the concluding section, Zhai and MacKenzie round up the article simply and efficiently,
putting across appropriate points which easily portray the points put across in the paper.

Summary
This was one of my favourite articles I read on the topic of input devices. It was extremely
easy-to-understand. The content was interesting as it described new innovations in mouse design. It
was exciting and the writers put across all their points well.

Touch&Type: A Novel Pointing Device For Notebook Computers

W. Fallot-Burghardt, M. Fjeld, C. Speirs,
S. Ziegenspeck, H. Kreuger, T. Läubli

Introduction
The Touch&Type device is one which, again, incorporates touch-sensing into an existing
device. Here, the keyboard is being used as a touch pad as well as a device on which to type. This
article describes a user-study using a prototype device There is also a technically detailed article
available.
The user-centred process of testing and prototyping and the subsequent description which is
detailed in this article is important to my understanding of touch-sensitive input devices. It gives the
reader an insight into how a potential user perceives a new product, especially since similar
products have already been on the market for twenty or thirty years.
I found this article very interesting as the concept of a keyboard with touch-sensors
integrated into it seems very exciting. The paper was very simplistic in language and easy-to-read.

Critique
The article begins with an interesting statement, declaring that computer users today are
reluctant to use alternative, built-in pointing devices due to the widespread use of the mouse. The
concept of Touch&Type, the writers go on to say, is that it is a novel input device which, they
consider, is superior than any conventional pointing device. The study described in the paper is a
comparison between the Touch&Type device, a mouse and a conventional touch pad.
 Throughout the introduction of the article, the writers put across points highlighting the
ineffectiveness of touch pads, pointing sticks and mice with keyboards. They make valid and
interesting points which provide a sturdy base on which to develop the concept and research of this
new input device.
In the 'Touch&Type Approach' the writers describe the layout of the Touch&Type device,
creating an image for the reader with each detail mentioned. They proceed to explain the expected
advantages of the device. Moving on to the 'User Study' section, the writers inform the reader how
they went about examining the performance of the three devices (mouse and keyboard, touch pad
and Touch&Type) and the kind of tasks used in the examination. A series of graphs and charts
document the results. Two lists show what the users found to be advantages and disadvantages of
the device. The article finishes up with a summary and outlook where the writers explain the
shortcomings of the device as well as documenting the aspects in which it outshone the other
devices.
Personally, I found the abstract and introduction to this article very interesting and
informative, giving me a detailed foundation before delving into the concept of Touch&Type. The
abstract, using simple language and describing effectively, gave a great insight into the basics of the
device. The introduction, showing the disadvantages to existing input devices, gave detailed
explanations regarding the other devices. I knew then, after reading such a clear and concise
introduction and proceeding to read further, the downfalls of existing devices and that this new
concept will be superior.
The writers go on to discuss the Touch&Type device in a detailed manner. The attention to
detail yet simplicity of language was one aspect of the text I found particularly effective. The
expected advantages of the device are discussed, with an explanatory table accompanying. This
makes the text all the more clearer to the reader, which I felt was very useful. The 'User Study'
section appeared to be the core of the article, describing the writers' experiments with prototype
devices. The results section, complete with tables and graphs, was clear and concise. I found this
section helpful in my research and reading as the writers accurately and simply put across the
results of the experiment in an easy-to-read fashion.
In the summary, the writers argue that one of the shortcomings in the user experience results
is down to the fact that users have much more prior experience with using mice, keyboards and
touch pads. The linkage between the summary and the introduction is seen here. The discussion at
the beginning of the article on the disadvantages of existing pointing devices mirrors this analysis
that these same pointing devices provided a shortcoming in the experiment. This brings the article
together and links the beginning and the end.

Summary
Overall, I found this article on the Touch&Type system highly interesting and exciting. It's a
new slant on touch-sensing and input devices. Although the mouse will stay in prominent use due to
its reliability and popularity, Touch&Type seems an exciting new prospect that, should it be
developed and mass produced, would aid easy integration between man and machine. The article,
overall, was produced in a very effective manner which I found very interesting and fascinating.

Soap: A Pointing Device That Works in Mid-Air

P. Baudisch, M. Sinclair, A. Wilson

Introduction
This article on the pointing device 'Soap' is one which describes and demonstrates the actual
device. The writers explain the concept behind the device and include lots of images to portray the
exact concept of the device. They put across the advantages of Soap efficiently and well.
The importance of this paper and the device on which it is based is very significant.
Researchers seem to always be trying to find ways of incorporating new methodologies into
existing devices. This article is one which grabbed my attention as the concept of having a mouse
working in mid-air is very innovative and exciting. There would be no need for a physical desktop
nearby.
I found the article very exciting and different. The writers put across their points concisely
so the reader fully understands the workings of the device and the experiments carried out during its
development.

Critique
The paper's abstract and introduction, well written and concise, put across the main points
that provide the fundamentals for understanding the topic. The writers explain how other input
devices, such as joystick-based presenter tools and game controllers, can be utilised in mid-air
whereas the mouse can't, thus making a mouse “half an input device”. A mouse is unusable without
a desk or mouse pad.
The writers then go on to explain the device in a paragraph entitled 'Soap'. They explain the
diagrams that accompany the paper. The device consists of two parts, the hull and the core. The
core contains an optical sensor, like in a normal optical mouse. The hull surrounds it. Any relative
motion between the hull and the core is picked up by the device and translated into on-screen
movement. They argue that the device is more suited to interacting with large screens as it can
produce very large motions and are “useful for getting across longer distances”.
The writers then go on to describe the various user tests they put Soap up to. These were
'Controlling Microsoft Windows on a wall display', 'Soap as a presenter tool' and 'Playing first
person shooter games'. The article is concluded with a paragraph on related work.
This article, in my opinion, was one of the more exciting and interesting. Its concise
documentation of the workings of the device appealed to me. It was easy to understand and
comprehend, despite the fact I have never seen or used Soap.
This paper was one of the more easy-to-understand papers and I never found myself
struggling to understand what the writers were trying to put across to the reader. The concept of the
device is fairly comprehendable and it seems like an easy device to manipulate and utilise.
I really liked the idea of a pointing device that works in mid-air. This would be down to one
reason and that is that it would be such a different experience to using a normal mouse or touch pad.
Having said that, refering back to the paper on CHECK PAPER! It is obvious that the user's hand
would get tired and/or sore after using such a device. For that reason, I believe that the device is not
as useful as it may seem. However, the information put across by the writers on the usage of Soap
as a pointing device for wall display or as a presenter tool is certainly valid. For these situations,
personally, I believe that Soap would benefit the user.
One strong point of this article is the use of images and diagrams. Putting such helpful and
relevant images in the article helps the reader's understanding of the device a great deal. Personally,
I found this to be the case. I understood the concept of Soap a little easier than if the article had no
diagrams or images. Images of the user-experiments appealed to me most. These showed the device
being used and provided some sort of reiteration that the device is effective and helpful in such
cases as presenting and gaming.

Summary
To conclude, I found this article on Soap most interesting. The article content appealed to
me as it is an exciting and innovative development in the field of HCI and input devices.
I don't see Soap being used as a commercial product soon or if it is, it may only benefit
people who will use it as a presenting tool or gaming controller. A similar device can already be
seen in the game controller of the Nintendo Wii. Perhaps this is the beginnings of such devices
being used by the consumer.
Overall, the article was well written and easy-to-understand. The content was exciting and
the images helped the article a great deal.
 Personal Statement on the Topic

As I have explained in my introduction, I chose the topic of novel input devices because
input devices are widely used today. I felt the topic would be interesting, as the word 'novel' would
allow me to research into new and exciting ways of using input devices.
I decided to focus on the mouse as it is such a popular and indispensable input device. Novel
ways of using the mouse interested me a great deal and I felt that researching into new innovations
in the mouse would be a very exciting prospect.
My aims beginning the project were to learn more about human-computer interaction and
novel input devices as well as the relationship between touch sensing and existing input devices and
how touch sensing can improve a regular input device like the mouse.
I went about my research by using both the internet and the library. I made great use of
websites such as Google Scholar,. ACM Digital Library and IEE Electronic Library. I found the
university library quite poorly stocked for my topic of choice. I found only one book on the topic of
input devices and at that, it was not a very helpful one. I choose to omit it from the documentation
of my research in this paper.
Having read a number of papers covering topics as diverse as history of HCI and new mouse
models, I feel like I have learned a lot about human-computer interaction, among other things.
Touch sensing was another slant on the topic I took and the papers I read on this were particularly
interesting.
The papers I found most useful in my research were:

• Soap: A Pointing Device That Works In Mid-Air

• Teaching Old Mice New Tricks: New Innovations in Computer Mouse Design
• Touch&Type: A Novel Pointing Device for Notebook Computers

These were the papers that I understood mostly. I also found them highly interesting. Their
content was easy-to-understand. The papers were well written. The topic of each appealed to me,
especially with Soap and Touch&Type as these were new mouse designs which I hadn't ever heard
of or used before.
As for the actual general topic of my study, personally I found the topic highly interesting.
The studies I found relating to the topic of novel input devices were plentiful and the ones into
which I choose to go into more detail were exciting and great to read.
Studying the history of HCI and the work of Douglas Engelbart, I have a great respect for
him as, essentially, the inventor of the mouse. I read and agreed with a point made in one paper that
he is not appropriately rewarded as an inventor as not many people know his name.
I feel novel input devices and in particular new mouse models have a place in technology in
the future. Although the mouse may never be replaced, at least in the near future, new innovations
in its design tell me that perhaps one day novel mouse designs may appear on the commercial
market. In particular, I found the Touch&Type device very exciting and maybe more commercial-
friendly than Soap, for example. It's only a matter of time before researchers in this field develop a
new device that is worthy enough of replacing the mouse. Having said that, I feel this new device
will still be based on the mouse. It may only be a simple mouse with a new tweak in its design to
make it more efficient for the user of the future.
To conclude, I'd like to say I enjoyed researching into the mouse's design and its history and
predecessors. It was an exciting topic and I'm glad I chose it.
 REFERENCE LIST

Baudisch, P., Sinclair, M., Wilson, A. (2006) 'Soap: A Pointing Device That Works in Mid-Air',
Symposium on User Interface Software and Technology, October 2006, 43-46

Fallot-Burghardt, W., Fjeld, M., Speirs, C., Ziegenspeck, S., Kreuger, H., Läubli, T. (2006)
'Touch&Type: A Novel Pointing Device For Notebook Computers', ACM International Proceeding
Series vol. 189, 465-468

Myers, B. A. (1998) 'A Brief History of Human Computer Interaction Technology.' ACM
interactions, 5(2), 44-54

Perry, T. S., Voelcker, J. (1989) 'Of Mice and Menus', PC/Computing, 2(4), 126-132

Zhai, S., & MacKenzie, I. S. (1998) 'Teaching old mice new tricks: Innovations in computer mouse
design' Proceedings of Ergon-Axia '98 - the First World Congress on Ergonomics for Global
Quality and Productivity, 80-83.