+

Google Goggles
By Sayantan Ghosh

+
What is Google Goggles ?

Google Goggles is a visual

search
application that lets a user

submit a search query by taking a

picture.
It

uses

computer

vision

techniques to recognize objects in a

image and then return information
on it or do some further processing
on it.

+
What all can it currently do?

User case demo

I know what this isbring it into my phone and tell me

more

I dont know what this istell me

Barcodes

Landmarks

Text

Books

Art Work

Wine

Logos

Contact information

SUDOKU

+
How does it do it?
It

uses multiple techniques to recognize the image.

First

it tries to identify the object by some object recognition

algorithms and compares it against Google's already quite
large database of images on almost everything.

To

assist the search it tries to find text in the image using

optical character recognition to get a better idea of what the
object might be.

It

also uses GPS to figure out where you are to filter results
received for landmarks to those that are relevant to your
location.

+
Object Recognition and Search
There are many algorithms out there for Object Recognition
Edge Detection
Scale Invariant Feature Transform
For Text
OCR (Optical Character Recognition)
For Images
Feature Detection (SIFT, SURF, PCA-SIFT)
Search By color
Visual Similarity Search

Life of a goggles query

Highly schematic

Highly parallel

+
Issues
Google

goggles was initially so sensitive that it could in many

cases find a person through an image and return a link to a
blog leading up to him.Google realized the privacy issues in
this and now controls the results so as to not recognize
people.

Whether

it still keeps that information for itself or not ?Who

knows ?
Google

goggles is still in its experimental phase and not all

results are accurate.

Google

Goggles doesnt work well with pets,furniture and

clothing.

Barriers
The faster Goggles responds, the more usage we have
Its good to respond fast even (especially) when we can't
give the right answer

Delays
Client delays
Camera focusing: 1.5 s
Picture acquisition: 2 s
Image (re) encoding.
Network delays
Goals at odds with Universal visual search as
Barcode and place recognition very different

Drawbacks

Supports only android and IOS mobiles

Pictures captured in the frame must be clear

Search results may not be precise

What is coming next in Google goggles?

Expanding the corpus of recognized objects

Using search quality techniques to improve relevancy

Increasing the Goggles footprint

Enabling 3rd party developers to create interesting applications based on

computer vision

+
Future

Google

is working to provide the ability to search for leaves

and plants which will also give the ability to tell weather a
plant/leaf is poisonous.

More

augmented reality like providing a 3D rendering of the

object in the image youre shooting.There are other Apps
that do provide this functionality.

More

accuracy and better search results and more

applications of this technology.
Use a front facing camera to detect if the user is happy

Demo

Questo un messaggio di prova

References
Google Goggles Main Page

Google Goggles by Symbioun Technologies Inc

Presentation on Google Goggles by Sahil Handa

Search by Sight: Google Goggles by David Petrou

Thank you.