Anda di halaman 1dari 29

By

Mohammad Bintahir
Gunjan Patel

Multi-touch and multi-modal technologies have existed


since 80s
Only recently has the technology been cost efficient and
robust enough to allow full implementation of environments
where these technologies can be utilized, cheaply and
effectively.
The recent developments of multi-touch technology such as
the Apple iPhone, Microsoft Thingsight and Jeff Hans
Touch wall to name a few examples, show the increasing
need, demand, use and popularity of these devices.
The combination of these events and the needs for a truly
versatile multi-touch and multi-modal hardware has
prompted the development of this project.

The objective of this project is to design and prototype an


implementation of a hardware device that will allow for the
detection of multi-modal inputs. The device will allow for multitouch and multi-user interaction with a defined focus to create a
stable modular hardware prototype based on a discrete
distributed sensor array.

Implement a method of expanding the digital workflow


environment in such a way that natural intuitive human
interactions can be developed and implemented to facilitate
collaboration of multiple users and workflows together
seamlessly.
The user can manipulate, share and transfer digital content in
ways not associated with the use of traditional personal
computers.
Touch-based interaction offers several advantages over current
WIMP based interfaces, especially when it comes to the use of
physical objects or widgets in interactive scenarios.
Traditional multi-point and multi-user collaboration systems
typically rely on projections and cameras.
While camera based gesture recognition methods are most
commonly used, they often suffer from integral flaws such as
lighting conditions, occlusion, and field of view

This capstone introduces a new sensing architecture, which will


facilitate new and emerging interactive models, it is based on
Discrete Distributed Sensors.
The method of sensing proposed will accurately track the position
of the users inputs in 2-Dimentional space and will be able to
calculate the pressure of the input.
In its elemental design it will be constructed using a basic voltage
divider circuit shown below.

Below is Level 0 of the Discrete Distributed Sensor Matrix, where


IR signal as input with power of 5V DC and output is a two
dimensional matrix containing IR intensity data across the
module.

IR Input Signal
Discrete Distributed Sensor
Matrix
Power, 5V DC

Sensor Data Output Signal

This is Level 1 cascaded architecture used to detect multi-modal inputs


based on IR signal.
The First Stage 16x8 array of IR Sensor s are used as the Input sources
that provide a relative Vout that based off the level of IR intensity that the
sensor can detect.
Second Stage is the Analog to Digital conversion step that converts the
Sensor Data Output Signal into a Quantized Data output signal.
The Final Stage is where the data is processed and a digitized signal is
sent to the computer to allow for blob analysis.

This is an overview of Level 2 design of how the Discrete


Distributed sensor matrix prototype looks like.
16 x 8 Infrared Sensors
ADC
ADC
ADC
ADC
ADC

ADC
ADC
ADC
ADC
ADC
ADC
ADC
ADC

SPI

Microcontroller Unit

ADC

USB

Figure below represents the SPI BUS Design that allows for
communication between the ADC and MCU SPI Master
represents the MCU and SPI Slave represents the ADC.

SPI BUS Design

SPI
Master

SCLK
MOSI
SS
MISO1
MISO2
MISO3

SCLK
MOSI
SS
MISO

SCLK
MOSI
SS
MISO

SCLK
MOSI
SS
MISO

SPI
Slave

SPI
Slave

SPI
Slave

This prototype consist of 32 discrete distributed sensor matrix form, and it was
tested successfully.

Final design consists of same essential voltage


divider circuit shown previously in 16 x 8 sensors
arrays.
The total prototype will consist of 128 sensors
Parallel data processing and acquisitions allows for
faster and more responsive detection of inputs
Prototype is designed in a 16:9 format that will
allow easy scalability on traditional LCD screens.

Prototype: Sensor Schematic

Prototype: MCU Motherboard Schematic

This is the final design of 16x8 discrete distributes sensor matrix, where total

of 128 IR sensors are used. It is two board design and its currently mounted
with microcontroller unit.

This is the front view of 128 IR sensors


soldered on PCB.

This is 2nd PCB, where ADC and capacitors are


soldered.


1.
2.
3.

Software consist of three critical components:


Microcontroller code
Blob analysis and tracking on the computer
Implementing interface design

Essentially the microcontroller code is


responsible for polling each individual sensor
and extrapolating IR intensity from input
source.
We do this by implementing unique row and
column address where at each pass the
microcontroller sets high each column and
polls each sensor by accessing the channels 1 to
8 of the ADC to obtain 2-D array of 12-bit
sensor data.

When the first frame of the data has been


captured in this 2-D array, it is used as a static
reference for threshold calculations. Where
each subsequent frame is compared to this first
frame. The resulting 2-D array is our
compensated sensor data.
This compensated sensor data is than
transmitted via USB to computer.

The resulting data received from the MCU is


than instructed into a low resolution, high
frame rate video feed.
This video feed is analyzed by using computer
vision techniques.
The technique, that was implemented is based
of the principle of growing regions. Where each
sensor value above predefined user controlled
threshold is noted and its position is recorded.

These sensor values are than grouped if


adjacent sensor values are also above the
specified threshold level.
Each of these groups is than assigned an ID
and its XY co-ordinates are noted.
The resulting identification is now tracked and
can be utilized for third party interfacing
applications.

The tracked IDs and positions can now be used


to develop applications such as this:

Have a nice Day

Anda mungkin juga menyukai