Geoffrey Hinton talks about Deep Learning, Google and Everything

1 of 4

http://www.kdnuggets.com/2014/12/geoffrey-hinton-talks-deep-learning...

KDnuggets
Data Mining, Analytics, Big Data, and Data Science
Subscribe to KDnuggets News | Follow
@KDnuggets

| Contact

Data Mining Software

News
Jobs
Academic
Swamped with TEXT Data ? Simplify your analysis! PolyAnalyst from Megaputer
Companies
Courses
Datasets
Data Mining Course
Education
KDnuggets
Home News 2014 Dec Opinions, Interviews, Reports Geoffrey Hinton talks about Deep Learning, Google and Everything ( 14:n32 )
Meetings
Latest News
Polls
Webcasts
KDnuggets News 15:n22, Jul
15: Deep Learning, Empiricism, &
Perfect Girl; 50+ Cheat Sheets; R vs
Python
Macy's: VP, Marketing Data
Science and Analytics
Top KDnuggets tweets, Jul 7-13:
Deep Learning and the Triumph of
Empiricism
Upcoming Webcasts on Analytics,
Big Data, Data Science - Jul 14 and
beyond
R, Python users show surprising
stability, but strong regional
differences
NYU MS in Business Analytics
NYU MS in Business Analytics
for Professionals - Learn More

Stevens BI & Analytics Graduate Program,

Learn MORE

Top Stories Last week

Most Viewed Last Week

1. Deep Learning and the Triumph of
Empiricism
2. Top 20 R packages by popularity
3. R vs Python for Data Science: The
Winner is
4. Top 20 R Machine Learning and
Data Science packages
5. Emacs for Data Science
6. Data Science and Big Data: Two
very Different Beasts
7. 9 Must-Have Skills You Need to
Become a Data Scientist

15/7/2015 17:04

Geoffrey Hinton talks about Deep Learning, Google and Everything

2 of 4

http://www.kdnuggets.com/2014/12/geoffrey-hinton-talks-deep-learning...

KDnuggets
Most Shared Last Week
Data Mining,
Big the
Data,
and Data
1. Deep Analytics,
Learning and
Triumph
of Science
SubscribeEmpiricism
to KDnuggets News | Follow
@KDnuggets
2. Data Science and Big Data: Two
very Different Beasts
Data
Software
3. 10 Mining
Key Tips
for Entry-Level
News
Analytics Professionals
Jobs
4. The Definitive Guide to doing Data
Academic
Science for Social Good
Companies
5. Can deep learning help find the
Courses
perfect girl?
Datasets
6. Standardizing the World of Machine
Data
Mining Web
Course
Learning
Service APIs
Education
7. Dataiku Data Science Studio Meetings
intuitive solution for data
Polls
professionals
Webcasts

| Contact

Geoffrey Hinton talks about Deep Learning, Google and Everything

100

31

Tweet

100

Previous post

Next post
Tags: Deep Learning, Geoff Hinton, Google, Neural networks, Reddit, Yann LeCun

A review of Dr. Geoffrey Hintons Ask Me Anything

on Reddit. He talked about his current research and

his thought on some deep learning issues.

By Ran Bi, Dec 2014

There is no doubt that Geoffrey Hinton is one of the top thought leaders in artificial intelligence. He is a professor at University of Toronto, and recently
joined Google as a part-time researcher.
When it comes to deep learning, we can see his name almost everywhere, such as in Back-propagation, Boltzmann machines, distributed representations,
time-delay neural nets, dropout, deep belief nets, etc.
My aim is to discover a learning procedure that is efficient at finding complex structure in large, high-dimensional datasets and to show that
this is how the brain learns to see Geoff Hinton.
Since Geoffrey Hinton became a researcher in Google, we did not hear much from him. His AMA (Ask Me Anything) on Reddit provided an excellent
opportunity to his fans. Here are some hot topics discussed on his AMA.
Current Research
The main thing I have been working on is my capsules theory.
Currently we use an unstructured layer of artificial neurons to model a cortical area. We tried it because it is easy to program and it turned out to work
very well. But Dr. Hinton wants to replace unstructured layers with groups of neurons called capsules that are a lot more like cortical columns.
Hinton also commented on Szegedy et al.s paper where the continuity and stability of deep neural networks were questioned (See my post Does Deep
Learning Have Deep Flaws?). However, Hinton said his capsule theory may not be fooled as easily.
Work at Google
Dr. Hinton joined Google last year after Google acquired his startup DNNresearch Inc.
When asked about his most successful things so far at Google, he said that
One big success was sending my student, Navdeep Jaitly, to be an intern at Google. He took a deep net for acoustic modeling developed by
two students in Toronto (George Dahl and Abdel-Rahman Mohamed) and ported it to Googles system.
Deep learning and How Our Brains Work
Geoff Hinton discovered how the brain really works. Once each year for the last 25 years.
This is a popular joke about Dr. Hinton and his theories (watch the video). Dr. Hinton made a couple of points about Deep Learning and how our brain
works on his AMA.

15/7/2015 17:04

Geoffrey Hinton talks about Deep Learning, Google and Everything

3 of 4

http://www.kdnuggets.com/2014/12/geoffrey-hinton-talks-deep-learning...

I think the success of deep learning gives a lot of credibility to the idea that we learn multiple layers of distributed representations using stochastic
gradient descent. However, I think we are probably a long way from understanding how the brain does this.
The brain does complex tasks like object recognition and sentence understanding with surprisingly little serial depth to the computation. So artificial

KDnuggets

neural nets should do the same.

Data Mining,
Analytics,
Big
Data,
and DataNeural
Science
It is very
impressive
that
DeepMinds
Turing Machine can get an RNN to invent a sorting algorithm. Its the first time Ive believed that deep
Subscribe
to KDnuggets
Follow
@KDnuggets
| Contact
learning
would be News
able to| do
real reasoning
in the not too distant
future.
Personal life
Data Mining Software
Dr. Hinton
News shared his life stories which have the greatest influences on his thinking.
Jobs
From a very young age I was convinced that many of the things that the teachers and other kids believed were just obvious nonsense. Thats
Academic
great
training for a scientist and it transferred very well to artificial intelligence.
Companies
Courses
He also enjoys watching Fry and Laurie.
Datasets
Data
Course
Course
onMining
Coursera
Education
Meetings
Dr. Hinton
started a course Neural Networks for machine learning on Coursera, which introduces artificial neural networks and its application. The
Polls
course
is a good start to learn deep learning. However, the course was offered two years ago. Someone asked if there are any changes on the course if
Webcasts
Hinton
redid it today.
Dr. Hinton said that he would split it into a basic course and an advanced course. In the advanced course, he would put a lot more about RNNs especially
for things like machine translation and also include reinforcement learning in the course.
Other Interesting Opinions
I think that the most exciting areas over the next five years will be really understanding videos and text. I will be disappointed if in five years time
we do not have something that can watch a YouTube video and tell a story about what happened.
The pooling operation used in convolutional neural networks is a big mistake and the fact that it works so well is a disaster.
I think the long-term future (of machine learning) is quite likely to be something that most researchers currently regard as utterly ridiculous and
would certainly reject as a NIPS paper.
I think answering questions about pictures is a better form of the Turing test. Methods that manipulate symbol strings without understanding them
(like Eliza) can often fool us because we project meaning into their answers.
More AMAs on Reddit of Machine Learning and Deep Learning stars:
AMA: Michael Jordan
AMA: Yann LeCun, and
AMA: Joshua Bengio.
Related:
Deep Learning important resources for learning and understanding
Does Deep Learning Have Deep Flaws?
Will Deep Learning take over Machine Learning, make other algorithms obsolete?
Previous post

Next post

Most popular last 30 days

Most viewed last 30 days

1. Top 20 Python Machine Learning Open Source Projects - Jun 1, 2015.
2. Which Big Data, Data Mining, and Data Science Tools go together? - Jun 11, 2015.
3. Popular Deep Learning Tools - a review - Jun 18, 2015.
4. R vs Python for Data Science: The Winner is ... - May 26, 2015.
5. 9 Must-Have Skills You Need to Become a Data Scientist - Nov 22, 2014.
6. Top 10 Data Analysis Tools for Business - Jun 13, 2014.
7. Top 10 Data Mining Algorithms, Explained - May 21, 2015.
8. Top 10 Machine Learning Videos on YouTube - Jun 23, 2015.

Most shared last 30 days

1. Which Big Data, Data Mining, and Data Science Tools go together? - Jun 11, 2015.
2. Why Does Deep Learning Work? - Jun 23, 2015.

15/7/2015 17:04

Geoffrey Hinton talks about Deep Learning, Google and Everything

4 of 4

http://www.kdnuggets.com/2014/12/geoffrey-hinton-talks-deep-learning...

3. Popular Deep Learning Tools - a review - Jun 18, 2015.

4. Top 20 R Machine Learning and Data Science packages - Jun 24, 2015.
5. Deep Learning and the Triumph of Empiricism - Jul 7, 2015.
6. Data Science and Big Data: Two very Different Beasts - Jul 6, 2015.
Top 10 Analytics,
Machine Learning
on YouTube
Data7.Mining,
Big Data,Videos
and Data
Science - Jun 23, 2015.
8. In Machine Learning, What is Better: More Data or better Algorithms - Jun 17, 2015.
Subscribe to KDnuggets News | Follow
@KDnuggets
| Contact

KDnuggets

Data Mining Software

News
Jobs
Academic
Companies
Courses
Datasets
Data Mining Course
Education
Meetings
Polls
Webcasts

Why lock yourself in?

Try KNIME
the Open Analytics
Platform
KDnuggets Home News 2014 Dec Opinions, Interviews, Reports Geoffrey Hinton talks about Deep Learning, Google and Everything ( 14:n32 )

2015 KDnuggets. | About KDnuggets

15/7/2015 17:04