:23'Problems in

Systems Neuroscience
Edited by J. Leo van Hemmen Terrence J. Seinowski

UTATIONAL
computational Neuroscience
Series Editors
Michael Strylcer
Terrence J. Sejnowski

Biophysics of Computation
Chistof ICoch

23 Problems in Systems Neuroscience

Edited by J. Leo van Hemmen and Terrence J. Sejnowslci
23 Problems in Systems
Neuroscience

Edited by

J. Leo van Hemmen

Terrence J. Sejnowslci

OXFORD
UNIVERSITY PRESS
OXFORD
UNIVERSITY PRESS

Oxford University Press, Inc., publishes works that further

Oxford University's objective of excellence
in research, scholarship, and education.
Oxford New Yorlc
Aucldand Cape Town Dar es Salaam Hong ICong ICarachi
ICuala Lumpur Madrid Melbourne Mexico City Nairobi
New Delhi Shanghai Taipei Toronto
With offices in
Argentina Austria Brazil Chile Czech Republic France Greece
Guatemala Hungary Italy Japan Poland Portugal Singapore
South Korea Switzerland Thailand Turkey Ukraine Vietnam

Copyright O 2006 by Oxford University Press, Inc.

Published by Oxford University Press, Inc.
198 Madison Avenue, New Yorlc, New Yorlc 10016

Oxford is a registered trademark of Oxford University Press

All rights reserved. No part of this publication may be reproduced,
stored in a retrieval system, or transmitted, in any form or by any means,
electronic, mechanical, photocopying, recording, or otherwise,
without the prior permission of Oxford University Press.

Library of Congress Cataloging-in-Publication Data

23 problems in systems neuroscience / edited by Leo Van Hemmen and Terrence Sejnowslti.
p, cm.
Includes bibliographical references and index.
ISBN-13: 978-0-19-514822-0
ISBN-10: 0-19-514822-3
1. Neurobiology. 2. Biological systems. I. Title: Twenty three problems in systems
neuroscience. 11. Hemmen, J. L, van (Tan Leonard). 111. Sejnowslci, Terrence J.
(Terrence Joseph)
QP355.2.Al4 2005
573.8'6--dc22 2005003411

Printed in the United States of America

on acid-free paper
 Preface

As long as a branch of science offers an abundance of problems, so long is

it alive; a lack of problems foreshadows extinction or the cessation of
independent development.
-David Hilbert in Paris, 8 August 1900

When invited in 1899 to give the opening address at the second International
Congress of Mathematicians (ICM) in Paris the following year, David Hilbert
(1862-1943), one.of the greatest mathematicians of the last century and the
father of Hilbert space, was facing the beginning of a new century. Henri
Poincar6, who had a similar task in addressing the first ICM in Zurich, extolled
the virtues of what had previously been accomplished in pure mathematics.
Rather than focus on the glories of the past, Hilbert chose to make a break
with tradition and instead emphasize what was not known. At the ICM on
August 8, 1900, he outlined twenty-three key open mathematical problems
as challenges to mathematicians for the twentieth century. These problems
were influential in shaping the direction of mathematics.
To understand Hilbert's motivation, let us retrace his reasoning in the
introduction to his lecture entitled Mathematical problems1:

The German text appeared in 1900 directly after the Paris meeting in Nachr. Kgl. Ges. d. Wiss. zu
GdZtingen, math.-phys. Klnsse 3, 253-297. It can also be found in volume 3 of Hilbert's collected works
(1935. 250-329). The translation above is by Dr. Mary Winston Newson and was published in 1902
in Bull. Arne,: Math. Soc. 8: 4 3 7 4 7 9 : it reappeared in 2000 in Bull. Amer. Math. Soc. (new series) 37:
407-436.
 History teaches the continuity of the development of science. We know that
every age has its own problems, which the following age either solves or
casts aside as profitless and replaces by new ones. If we would obtain an idea
of the probable development of mathematical knowledge in the immediate
future, we must let the unsettled questions pass before our minds and look
over the problems which the science of to-day sets and whose solution we
expect from the future. To such a review of problems the present day, lying
at the meeting of the centuries, seems to me well adapted. For the close of a
great epoch not only invites us to look back into the past but also directs our
thoughts to the unknown future.
The deep significance of certain problems for the advance of mathema-
tical science in general and the important role that they play in the work of
the individual investigator are not to be denied. As long as a branch of
science offers an abundance of problems, so long is it alive; a lack of prob-
lems foreshadows extinction or the cessation of independent development.
Just as every human undertaking pursues certain objects, so also mathe-
matical research requires its problems. It is by the solution of problems that
the investigator tests the temper of his steel; he finds new methods and new
outloolts, and gains a wider and freer horizon.
It is difficult and often impossible to judge the value of a problem cor-
rectly in advance; for the final award depends upon the grain which science
obtains from the problem. Nevertheless we can ask whether there are
general criteria which mark a good mathematical problem. An old French
mathematician said: "A mathematical theory is not to be considered com-
plete until you have made it so clear that you can explain it to the &-st man
whom you meet on the street." This clearness and ease of comprehension,
here insisted on for a mathematical theory, I should still more demand for a
mathematical problem if it is to be perfect; for what is clear and easily
comprehended attracts, the complicated repels us.

As for systems neuroscience, why add more, if-mutatis mutandis-it has all
been said so clearly a century ago? The same text, with "mathematical sci-
ence" replaced by "systems neuroscience," applies to the twenty-first century.
The complexity of the brain and the protean nature of behavior remain the
most elusive area of science and also the most important. There is no single
brilliant individual who could serve as the David Hilbert of systems neu-
roscience, so we have invited twenty-three experts from the many areas of
systems neuroscience to formulate one problem each.
We thank the Max Planck Society for hosting a wonderful meeting on
"Problems in Systems Neuroscience" in Dresden on September 4 to 8, 2000,
that formed the core of the authors in this collection. We have asked several
other colleagues to contribute to the twenty-three challenging questions in
systems neuroscience that are included. It took Hilbert two years to get
his problems published in English. It took us even longer to get the twenty-

vi Preface
three different essays collected together in this volume. Although it is not
comprehensive, and it is, like Hilbert's original list of problems, idiosyncratic,
nonetheless it may serve as a source of inspirations for future explorers of the
brain.
We have organized the chapters into five themes that range from evolution
to qualia, each theme posed as a general question. Although each of the
chapters was independently written and can be read separately, there are
some common issues that reflect contemporary concerns.

J. Leo van Hemmen

Terrence J. Sejnowski

Preface vii
 Contents

Contributors xiii

PART I How Have Brains Evolved?

1. Shall We Even Understand the Fly's Brain? 3

Gilles Laurent

2. Can We Understand the Action of Brains in Natural

Environments? 22
Hermann wagner and Bernhard Gaese

3. Hemisphere Dominance of Brain Function-Which Functions

Are Lateralized and Why? 44
Giinter Ehret

PART II How Is the Cerebral Cortex Organized?

4. What Is the Function of the Thalamus? 65

S. Murray Sherman

5. What Is a Neuronal Map, How Does It Arise, and What Is

It Good For? 83
J. Leo van Hemmen

6. What Is Fed Back? 103

Jean Bullier
PART I l l How Do Neurons Interact?

7. How Can the Brain Be So Fast? 135

WuIfram Gerstner

8. What Is the Neural Code? 1 4 3

C. van Vreeswijk

9. Are Single Cortical Neurons Soloists or Are They Obedient

Members of a Huge Orchestra? 1 6 0
Tal Icenet, Amos Arieli, Misha Tsodylzs, and Amiram Grinvald

10. What Is the Other 8 5 Percent of V 1 Doing? 182

Bruno A. Olshausen and David J. Field

PART IV What Can Brains Compute?

11. Which Computation Runs in Visual Cortical Columns? 2 1 5

Steven W , Zucker

12. Are Neurons Adapted for Specific Computations? Examples

from Temporal Coding in the Auditory System 245
C. E. Carr, S. Iyer, D. Soares, S. IGzlluri, and J. Z. Simon

13. How Is Time Represented in the Brain? 266

Andreas V. M. Herz

14. How General Are Neural Codes in Sensory Systems? 283

David McAlpine and Alan R. Palmer ,

15. How Does the Hearing System Perform Auditory Scene

Analysis? 303
Georg M. Iclump

16. How Does Our Visual System Achieve Shift and Size Invariance? 322
Laurenz Wiskott

PART V Organization of Cognitive Systems

17. What Is Reflected in Sensory Neocortical Activity: External

Stimuli or What the Cortex Does with Them? 343
Henning Scheich, Frank W. Ohl, Holger Schulze, Andreas Hess,
and Andre' Brechmann

x Contents
18. Do Perception and Action Result from Different Brain Circuits?
The Three Visual Systems Hypothesis 367
Giacomo Rizzolatti and Vittorio Gallese

19. What Are the Projective Fields of Cortical Neurons? 394

Terrence J. Sejnowslzi

20. How Are the Features of Objects Integrated into Perceptual

Wholes That Are Selected by Attention? 406
John R Reynolds

21. Where Are the Switches on This Thing? 423

L. F. Abbott

22. Synesthesia: What Does It Tell Us about the Emergence

of Qualia, Metaphor, Abstract Thought, and Language? 432
V. S. Ramachandran and Edward M. Hubbard

23. What Are the Neuronal Correlates of Consciousness? 474

Francis C. Crick and Christof Koch

Author Index 49 1

Subject Index 505

Contents xi